Architecture upgrade

Yesterday, ย Spider encountered a major architecture upgrade:

  • I moved to one index by resource type
    • To get ready for ES 6
    • To improve shards balance in the cluster
    • To reduce IO access because smaller volume resources are separated from big ones
    • To improve ES aggregations speed
  • I introduced a poller between Redis and ElasticSearch for the four main resources
    • The load constraint is now only focused on two microservices and only on Redis
    • The load on ES is smoothed
  • I added some in memory cache on Whisperer configuration access from microservices to reduce drastically unnecessary calls
  • I have now 20 microservices in my cluster, with many being multi instantiated ๐Ÿ˜Ž

Streesmart instance has been upgraded with a complete data purge (sorry).

Let’s look how it behaves. It should be much more stable!

2 Replies to “Architecture upgrade”

  1. After one week without any reboot needed or any DB purge. I can tell that the system is more stable ๐Ÿ™‚
    But still, I have time outs happening with my Redis instances not being fast enough to absorb packets spikes…

    Instead of increasing hardware to be able to handles spikes only, next step is to limit client throughput to absorb spikes on client side!
    Stay tuned ๐Ÿ˜‰

  2. For now system handles 800/900 indexation/s in ES in peaks lasting for more than 30 min. About 600 packets/s. Not so bad.
    I think we can do more with same hardware, as this is only with one unique poller by resource!

    And that’s only serialization. Redis handles twice more packets saves/s in its cache when on peak. It is still not enough, but I have ideas to improve.

    I got 300 million records in DB for 3 days, and it’s still fast.

Leave a Reply

Your email address will not be published. Required fields are marked *