Yesterday, Spider encountered a major architecture upgrade:
- I moved to one index by resource type
- To get ready for ES 6
- To improve shards balance in the cluster
- To reduce IO access because smaller volume resources are separated from big ones
- To improve ES aggregations speed
- I introduced a poller between Redis and ElasticSearch for the four main resources
- The load constraint is now only focused on two microservices and only on Redis
- The load on ES is smoothed
- I added some in memory cache on Whisperer configuration access from microservices to reduce drastically unnecessary calls
- I have now 20 microservices in my cluster, with many being multi instantiated 😎
Streesmart instance has been upgraded with a complete data purge (sorry).
Let’s look how it behaves. It should be much more stable!