Since I'm a persistence person and not a UI person, I am more interested in what is on the other side of the Phoenix/Elixir/BEAM VM server cluster: the database/persistence. I understand that currently apps mostly talk to PostgreSQL, like a Rails/Django app would?
Then, if something modifies the data in the database not by using the Phoenix app, the Phoenix app would not find out until it loaded the values from the database. And what would prompt it to do that?
But if your entire active state can fit in RAM in a cluster of BEAM VMs, you might turn the BEAM VM cluster itself into a distributed database, in which the only way to modify the data is by talking to the Phoenix app (using Liveview, regular REST HTTP API or something else, maybe even postgresql protocol). If this is the case, the app server can guarantee that no state change could have happened by something else modifying the database or whatever, and a client receiving updates via websocket would be sure that it has a complete and accurate picture of the data.
Of course, you would need snapshot isolation (versioning of tuples in RAM) and you would need to store the transaction log durably on disk and you would need to garbage collect (postgresql vacuum) the no longer needed records. Snapshot export and "live update pushing" to a traditional SQL database would be desirable for BI and stuff. But basically, if a modern distributed database was also a good application server, it would be neat.
> Then, if something modifies the data in the database not by using the Phoenix app, the Phoenix app would not find out until it loaded the values from the database. And what would prompt it to do that?
Just like Rails, phoenix is the doorway to your application. If there are data changes happening that aren't through your application, then you're doing it wrong.
> If there are data changes happening that aren't through your application, then you're doing it wrong.
This is super-unrealistic. State mutation legitimately happens through many channels. The trick is signaling to the application layer that state has changed and either the app needs to reload it or update it.
In the case of GP, I would either build in a web callback that can be used by outside processes, or put a message on a queue like SQS that can be consumed by the application. This isn't a phoenix thing so much as an enterprise app design thing.
> This is super-unrealistic. State mutation legitimately happens through many channels. The trick is signaling to the application layer that state has changed and either the app needs to reload it or update it.
13 years in professionally, many companies, many contracts, and many projects later I've yet to see a system where it wasn't the case. Small or large (16k/rps at the large end). Startup and "enterprise".
And why wouldn't it be the case? Why not have your data flow through a "central" business logic layer?
Some applications use a database which is shared by multiple applications, and you are not allowed to change the models (i.e. you can't ALTER TABLE). You code your application knowing that data can be changed by others. Your application coordinates with other applications through the database. Using separate services that each have their own database is not required and incurs overhead.
Postgres can do a form of pub/sub with triggers on table change events, pg_notify. For anything more complicated get Postgres to publish all its changes using Change Data Capture to a Materialize instance. With Materialize, every query can push live updates to an app server, whenever the results change. Debounce a little and push the html, ideally don’t overwrite something the user is editing.
The Materialize folks made a blog post/demo of the materialize->app server->browser pipeline in Python+JS you could follow along with. They have a Postgres CDC implementation too, so hook it all up and it will go.
> But if your entire active state can fit in RAM in a cluster of BEAM VMs, you might turn the BEAM VM cluster itself into a distributed database
It's hard to imagine an active state that couldn't fit into RAM in a cluster of BEAM VMs. I've run clusters with thousands of nodes, with some nodes having 768 GB of ram. With today's servers, 4 TB of ram per node is approachable. The new pg module avoids pg2's dependence on (cluster) global locks that limited effective cluster size.
Of course, you have to want to do it, and mnesia is mostly key-value, so I don't think you'd have a good time if you need other kinds of queries (but I could be wrong). And you need to have readily partitioned data if you want your cluster to be operable; having all the nodes in one mnesia schema makes a lot of things hard.
Then, if something modifies the data in the database not by using the Phoenix app, the Phoenix app would not find out until it loaded the values from the database. And what would prompt it to do that?
But if your entire active state can fit in RAM in a cluster of BEAM VMs, you might turn the BEAM VM cluster itself into a distributed database, in which the only way to modify the data is by talking to the Phoenix app (using Liveview, regular REST HTTP API or something else, maybe even postgresql protocol). If this is the case, the app server can guarantee that no state change could have happened by something else modifying the database or whatever, and a client receiving updates via websocket would be sure that it has a complete and accurate picture of the data.
Of course, you would need snapshot isolation (versioning of tuples in RAM) and you would need to store the transaction log durably on disk and you would need to garbage collect (postgresql vacuum) the no longer needed records. Snapshot export and "live update pushing" to a traditional SQL database would be desirable for BI and stuff. But basically, if a modern distributed database was also a good application server, it would be neat.