TCP parsing storage has been reworked

TCP session parsing storage has evolved for the second time.

The first time was to be able to have TCP sessions with thousands of HTTP communications inside: I had to remove the list of items from the parsing block.

This time was to be able to have parallel parsing of TLS and HTTP (or another) protocol without risking concurrency update.

Change

I managed to update the structure only on the parsing phase, in Redis, without changing the long term storage in Elasticsearch. This avoided many rework on the UI and indices!

Now Parsing is stored separately than the session in Redis, and then joined in the application level.
This slows down a bit the retrieval of TCP sessions and the parsing speed.
But as always, it is done in bulk, with a right focus on performance, and effect is barely noticeable!

Process

This change has been a great challenge. As it is not often that you change the underlying storage of you main process.
There is a high risk of regression and a nightmare of troubleshooting.

I nevertheless managed to do this in less than a week delay! Thus comforting me in the quality of the code and application structure 😀

The main and most important part was to do reverse engineering of the current state, document it, and then design the evolution. 💪

More?

Want to know more? Contact me.