You need an interface, usually a GUI, that lets them add or edit stuff.
If you modelled the data reasonably, the screen presentations are mappable back to the database format, so you just need a bidirectional way to go between them.
If you need to decorate that interface with some relatively static data, it doesn’t get much harder. You manually get the data from somewhere, keeping track of how often it gets updated. If changes are occasional or a little more frequent, you can put that data into a config file, then maybe try to add in some update automation or screens to make updating it easier.
The game gets a lot harder, though, when you split the program in two with a client/server type arrangement. If the communication is super reliable, it's just a little extra error handling to support rainy days. If the communication is not reliable, then there are a series of hard trade-offs you need to make when handling failures.
In some cases, if the front can’t reach the back, it can effectively freeze. But in some cases, it needs to continue on as normal, then carefully resync later when the backend is available again. What it shouldn’t do is show the user old, stale data; if changes in the backend went unnoticed, that just causes chaos. It also shouldn’t clobber new data that was changed in the backend without being updated.
This gets super complicated if there are lots of users and they are changing stuff all of the time. And bumps up to a huge problem if there are lots of users and lots of backends. Keeping that type of arrangement in sync is right at the edge of our knowledge.
Internally, you want to try and wrap it somehow. That is, you have user data, static data, and remote data. You put that in a layer underneath any of the computations and presentations.
The different types of data are polymorphic, but you wire in the worst-case error handling at the top for everything. You do that because data accessibility problems are essentially interface problems, not data ones. That is, the interface asks for some data, but it is told to wait. Or maybe it is told to drop what it asked for earlier and update it with something newer. The problem isn’t reading data; rather, it is keeping a small window of data in the client synchronized with some other source of data, which means that the wiring has to include any and all of the means to detect that it is out of sync and also to fix the circumstances, as well as support two-directional communication.
So we can play with the idea that the client sends an event to ‘ask’ for data; the server sends an event to ‘show’ a specific version of that data.
If you’re going to go right over the edge, then you might wire up a client as sets of widgets that send ‘ask’ events for specific data entries. Later they get ‘show’ events and respond. At some point, they can also send ‘update’ events for changes to that data. They do this on all ‘sets’ of interdependent data rather than individual fields.
Then, we get a scenario where two clients send a bunch of ‘ask’ events and get back ‘show’ events. The users change stuff; the first one to hit Save gets updated, but then the other client receives an ‘update’ event. For each widget on their screen with modified data, it can show both the new value and the changed value on the screen and ask the second user to pick, or back off.
We get another scenario where the user grabbed the data via a bunch of ‘asks’ followed by ‘shows’. Then the server went down. So on Save, they get some visible indication that the ‘update’ was sent, but the code never gets the ‘show’ back again. The user is notified that their changes are pending and not saved. Again, it can ask the user to address the situation. It can even provide a means to temporarily hold their changes until later.
In this way, we can reframe the problem not in terms of logic to write, but rather in terms of just keeping a small pool of data synchronized with a much larger pool. This type of abstraction is data-driven and is far easier to understand than lots of twisted nested error handling dumped all over the code. You can explore the essence of a problem and find simple machinery in order to mitigate it.
We can even extend this idea to really difficult scenarios. The program displays integrated data from a huge number of different sources. The widgets only display derived data; underneath this, the derivation does a series of ‘asks’, then does recomputes based on the incoming ‘shows’. It pumps back a ‘show’ to the above widget. For some sources, using the very last ‘show’ that is cached is allowable; for some, it is not. Daily calcs, for example, can’t use day-old stuff, but weekly calcs can use stuff up to six days old. We wire in that data retention config at the source, and then everybody on top can only do the right thing. It can’t be ignored; it can’t be overridden. If the data far below, buried in some deep derived data, won’t update, then nobody on top will either.
That type of strict handling is necessary because different people will each handle the same problem in a different way. That freedom begets chaos, which ultimately confuses the users, diminishing the value of the code, not enhancing it. It would be better to be consistently wrong than to have a random mix of correctness. At least then it could be fixed quickly in just one place.
The idea, though, is that we can step back to look at the way the data flows around, or doesn’t. If we change our perspective away from coding and towards just keeping different things in sync with each other, it becomes a lot easier to solve this as a general problem, rather than keep specifically solving it over and over again for each frustrating little instance. Good abstractions aren’t special; they are just stronger ways to twist your perspective a little when a bunch of related problems are causing similar trouble, in order to do a better job solving them all at once. We just need to dig down to the fundamental problems, not the symptoms on top.
If you need to decorate that interface with some relatively static data, it doesn’t get much harder. You manually get the data from somewhere, keeping track of how often it gets updated. If changes are occasional or a little more frequent, you can put that data into a config file, then maybe try to add in some update automation or screens to make updating it easier.
The game gets a lot harder, though, when you split the program in two with a client/server type arrangement. If the communication is super reliable, it's just a little extra error handling to support rainy days. If the communication is not reliable, then there are a series of hard trade-offs you need to make when handling failures.
In some cases, if the front can’t reach the back, it can effectively freeze. But in some cases, it needs to continue on as normal, then carefully resync later when the backend is available again. What it shouldn’t do is show the user old, stale data; if changes in the backend went unnoticed, that just causes chaos. It also shouldn’t clobber new data that was changed in the backend without being updated.
This gets super complicated if there are lots of users and they are changing stuff all of the time. And bumps up to a huge problem if there are lots of users and lots of backends. Keeping that type of arrangement in sync is right at the edge of our knowledge.
Internally, you want to try and wrap it somehow. That is, you have user data, static data, and remote data. You put that in a layer underneath any of the computations and presentations.
The different types of data are polymorphic, but you wire in the worst-case error handling at the top for everything. You do that because data accessibility problems are essentially interface problems, not data ones. That is, the interface asks for some data, but it is told to wait. Or maybe it is told to drop what it asked for earlier and update it with something newer. The problem isn’t reading data; rather, it is keeping a small window of data in the client synchronized with some other source of data, which means that the wiring has to include any and all of the means to detect that it is out of sync and also to fix the circumstances, as well as support two-directional communication.
So we can play with the idea that the client sends an event to ‘ask’ for data; the server sends an event to ‘show’ a specific version of that data.
If you’re going to go right over the edge, then you might wire up a client as sets of widgets that send ‘ask’ events for specific data entries. Later they get ‘show’ events and respond. At some point, they can also send ‘update’ events for changes to that data. They do this on all ‘sets’ of interdependent data rather than individual fields.
Then, we get a scenario where two clients send a bunch of ‘ask’ events and get back ‘show’ events. The users change stuff; the first one to hit Save gets updated, but then the other client receives an ‘update’ event. For each widget on their screen with modified data, it can show both the new value and the changed value on the screen and ask the second user to pick, or back off.
We get another scenario where the user grabbed the data via a bunch of ‘asks’ followed by ‘shows’. Then the server went down. So on Save, they get some visible indication that the ‘update’ was sent, but the code never gets the ‘show’ back again. The user is notified that their changes are pending and not saved. Again, it can ask the user to address the situation. It can even provide a means to temporarily hold their changes until later.
In this way, we can reframe the problem not in terms of logic to write, but rather in terms of just keeping a small pool of data synchronized with a much larger pool. This type of abstraction is data-driven and is far easier to understand than lots of twisted nested error handling dumped all over the code. You can explore the essence of a problem and find simple machinery in order to mitigate it.
We can even extend this idea to really difficult scenarios. The program displays integrated data from a huge number of different sources. The widgets only display derived data; underneath this, the derivation does a series of ‘asks’, then does recomputes based on the incoming ‘shows’. It pumps back a ‘show’ to the above widget. For some sources, using the very last ‘show’ that is cached is allowable; for some, it is not. Daily calcs, for example, can’t use day-old stuff, but weekly calcs can use stuff up to six days old. We wire in that data retention config at the source, and then everybody on top can only do the right thing. It can’t be ignored; it can’t be overridden. If the data far below, buried in some deep derived data, won’t update, then nobody on top will either.
That type of strict handling is necessary because different people will each handle the same problem in a different way. That freedom begets chaos, which ultimately confuses the users, diminishing the value of the code, not enhancing it. It would be better to be consistently wrong than to have a random mix of correctness. At least then it could be fixed quickly in just one place.
The idea, though, is that we can step back to look at the way the data flows around, or doesn’t. If we change our perspective away from coding and towards just keeping different things in sync with each other, it becomes a lot easier to solve this as a general problem, rather than keep specifically solving it over and over again for each frustrating little instance. Good abstractions aren’t special; they are just stronger ways to twist your perspective a little when a bunch of related problems are causing similar trouble, in order to do a better job solving them all at once. We just need to dig down to the fundamental problems, not the symptoms on top.
No comments:
Post a Comment
Thanks for the Feedback!