The recommended security setup is use a different server for the checksums.
Makes sense but I am already having major server creep with just about everything wanting its own server.
How are you managing checksum storage?
Checksums are generated on the fly by the server, and sent to the client. The client then calculates a hash on the assets it has. If the hashes are different, the client downloads the assets from the server.
The server calculates them on the fly because some assets can be modified while the server is running.
This is not about security, it is about ensuring the client has assets that are not corrupt. There is nothing that will stop users from modifying assets if they really want to, all users have to do is modify the client to not perform the check. Hashes should not be used for security, they should simply be used to check for corrupt files. If you are worried about people modifying assets from a security standpoint, you need to re-evaluate why modified assets have a security impact.
1 Like
As I understand it, the CheckedInputStream just returns whatever value it calculates for that given file after it completes reading the stream. Which would be a different value if there was an error.
How are you retrieving the value from the server?
Maybe I should clarify what at first glance seems like such a simple question.
You say you calculate the checksum on the fly and send it to the client.
This implies to me you have a rest end point configured specifically for calculating the checksum and sending back via a response.
Do you set it in the body of the response or use a header?
As I am only downloading files the client knows it needs in advance, I have the option of using the response ETag header for the requested resource, even though this seems to be considered not the thing to do.
Edit: As I already know the file the client is after, I can set the the header server side prior to streaming the file.
I can read the ETag from the client and do my checking, retry or cleanup right after the file is written to disk.
Just curious how you do it.
My process for getting assets to the client:
- Client performs an http request to the server
http://<ip>:<port>/client.assets - Server calculates hash for all assets, and creates an assets list file in the format of:
ASSET_LIST=<Number of assets>
<asset file>=<asset last modified timestamp>=<asset hash>
<asset file>=<asset last modified timestamp>=<asset hash>
<asset file>=<asset last modified timestamp>=<asset hash>
<asset file>=<asset last modified timestamp>=<asset hash>
...
For example:
ASSET_LIST=101
assets/Interface/CC/CC.fxml=1607707754276=1457137033
assets/Interface/CC/MorphListCell.fxml=1607707754278=2325421817
assets/Interface/Chat.fxml=1607707754281=1094231430
assets/Interface/circle.png=1607707754282=3826605755
...
-
Client checks to see if any of the assets are older, or have a different hash than the server’s asset, or if the server has an asset it does not have.
-
Client downloads any assets it decided were missing or different using the http server.
EDIT: You will note that this is not a very clean endpoint on the server as far as how the data is structured. It is on the todo list to make this a standard rest api using json.
For errors, if the server has an error in calculating the hash, we have larger problems.
Our http server (which is built into the game server) has several endpoint specifically for handling getting assets to the client.
Do you know if there’s a downside to me using the ETag to store the desired checksum in the response of the file being streamed?
I think that would be fine. That is how web browsers determine if the http assets (images and the such) are newer on the server using a timestamp. It would be the same idea, just using a hash instead.
Ok, this is done.
I use @tlf30 methods from that class on the server, setting the ETag when the resource is requested.
I made a minor change to the streams on the client to be CheckedInputStream and CheckedOutputStream as my setup reads files from HTTP and the local file system using the same class. If its a local file thats just been read in, I use the clients CheckedInputStream to determine the local checksum, and if its an HTTP, I read the ETag.
Now I am down to writing the retry, cleanup code.
Thanks to all for the input, made all the difference.
1 Like