Automated maintenance
Concept
Spider includes some automated maintenance jobs that are scheduled and run automatically to tend to the system behavior.
List of jobs
| Job | Description | Periodicity |
|---|---|---|
| forceMergeJob | Performs a forcemerge action to some indices that have low cardinality and high update frequency. | Each 45 min |
| purgeJob | Removes useless data from the system. | Each day |
| backupJob | Backup active configuration to be able to restore it in case of big outage, or for a redeployment of Spider. | Each day |
| cleanWhisperersJob | Removes deleted whisperers from Teams and Customers resources. | Each day |
Execution
These jobs are run by the maintenance service, at start and after the configured delay.
The configuration of jobs is made through standard system configuration and may be overridden by adding extraConfiguration key in the service part of the values.yaml file.
The default configuration is listed here: Maintenance.
Details
Force merge job
Some indices are storing data that is updated too frequently for Elasticsearch to perform its segment merges.
This slows down searches on these data.
To cope with this, Spider performs regular forcemerge action on the following indices:
| Index | Content |
|---|---|
| spider-active-hosts-streaming | Stores the hosts resolution (IP -> Name) sent by Whisperers. Records might be updated every min. |
| spider-whisp-status | Stores the Whisperers status. Records are updated every 20s. |
| spider-session | Stores the user sessions. Records are updated every few minutes. |
Purge job
Some deleted or stale data stay in the system for auditability or recovery.
This job removes them after a while.
| Data | Index | Condition |
|---|---|---|
| Users | spider-customers | User deleted more than 6 month ago |
| Whisperers | spider-whisp | Whisperers deleted more than 6 months ago |
| Whisperers status | spider-whisp-status | When modified more than 1d ago |
| Controllers | spider-controllers | Controllers deleted more than 6 months ago |
| Attachments | spider-attachments | Attachments ended more than 6 months ago |
| Links | spider-links | Links created more than 3 months ago |
Backup job
Configuration data is backed up to a S3 compatible storage in case of disaster, or to reload the configuration in a new deployment of Spider.
| Data | Index | Condition |
|---|---|---|
| Users | spider-customers | ACTIVE customers |
| Users UI settings | spider-guisettings | - |
| Whisperers | spider-whisp | ACTIVE whisperers |
| Teams | spider-teams | ACTIVE teams |
| Controllers | spider-controllers | ACTIVE controllers |
| Attachments | spider-attachments | ATTACHED attachments |
Clean Whisperers job
This job:
- Loops on all teams and users
- Removes from their whisperers list
- Whisperers deleted more than 1 week ago
- Whisperers that have been purged
- Updates the teams and users
This job will trigger emails when it is updating teams to inform administrators that the whisperers have been deleted.
In the editor field, the user maintenance is provided for auditability.
Logs are provided to explicitly list the changes performed.