Skip to main content

Installing Spider

caution

The following instructions are for Swarm installation process.
This process is not actively tested or used anymore.

Please inform Floocus before using them.

Prerequisites​

Credentials​

To setup Spider, you need several tokens:

All these tokens come together with your License key when you purchase a Spider license.
They all have a limited validity duration, linked to the license own TTL.

All files referenced in this documentation are placed in Spider GIT repository: gitlab.com/spider-analyzer/spider.

Tools​

Tools to run setup scripts:

  1. Setup tools
    • Docker
    • make
    • sed
  2. Docker Swarm or docker-compose (>=3.7) must be installed and accessible
    See 0_infra/0_spider server setup.md
  3. Elasticsearch cluster is ready and setup
    See 0_infra/1_es server setup.md
    Metricbeat settings file on ES nodes will be available in 2_environment/2_config/metricbeat/metricbeat-es.yml, once configuration is generated
  4. The node from which you run installation is ready (Docker, Make, Git, Gitlab access)

Optional dependencies​

S3 bucket​

Spider may back-up regularly its configuration resources:

  • Users + User settings
  • Teams
  • Whisperers

To allow backup of data, you need to provide access to a S3 - or compatible - bucket. Address and credentials have to be provided in the setup.yml.

SMTP server​

Spider sends mails at various times (configurable):

  • For alerting
  • During user creation workflow
  • When a user profile changes
  • At team creation or updates
  • ...

To send a mail, Spider needs access to a SMTP server, with credentials to provide in setup.yml.

Configuration​

  1. Clone the repository
    git clone git@gitlab.com:spider-analyzer/spider.git
    1. Use the token given by Floocus to connect
  2. List available versions and pick the one you want
    make listAvailableVersions
  3. Run make new-install
  • Creates a new branch not to pollute master, and to be able to save it on your own
  • It is advised to create a folder X_local for any specific local setup file
  • Command asks you if you want a specific version (tag) or the latest
  1. Create a setup.yml file from setup-template.yml
  • In same folder
  • Use template own inline documentation
  • This file is ignored by git by default as it will contain private information
  • But you may change this and commit your branch on your own servers if you wish. By adding a different remote server on Git configuration.

Sample (simplest) configuration file:

version: 2022.12.13
endpoint:
  publicPath: http://spider.mycompany.com
stores:
  elasticsearch:
    _default:
      name: Elasticsearch Cluster
      node: http://elasticsearch.local:9200
  redis:
    _default:
      name: Redis cache
      node: redis.local
customers:
  supportEmails:
    - admin@mycompany.com
smtp:
  email: spider@mycompany.com
  server: smtp.local
  port: 25

Installation​

Installation process:

Once configuration is set, you run a one line command that installs Spider on the selected target.
This one runs a set of commands under-the-hood:

  • config - Generates configs from templates
    • Requires Docker to run the script
  • keypair - Setup new keypair for security
    • Requires Docker to run the script
  • elasticsearch - Spins how elasticsearch store
    • Except for Swarm as it is expected outside Swarm
  • db - Setup Elasticsearch indices
  • admin - Setup first admin account
  • demo-data - Inject demo data for Demo target
  • And then the command to deploy the services

The commands are running setup scripts from Spider setup Docker image.
Thus you need first to login to Spider docker registry.

Connect to Spider docker registry​

Connect to the registry with Floocus provided token.

docker login registry.gitlab.com

One line startup commands​

  • Local - make d-local-setup
    • Sets up Spider locally with docker-compose
  • Demo - make d-demo-setup
    • Sets up Spider locally with docker-compose and some demo configuration
  • Swarm - make d-swarm-setup
    • Sets up Spider in a Swarm cluster

Update​

You may list available versions and pick the one you want:
make listAvailableVersions

Once you chose a version (latest one ;-) ), you run a one line command to update your setup.

It update your repository and all template files. You may then have to add new values in your setup.yml to adjust with new features.

One line update commands​

  • Local - make d-local-update
  • Demo - make d-demo-update
  • Swarm - make d-swarm-update

Commands run under the hood​

The online command runs the following process, with some adaptations regarding the target:

  • update
    • Updates the branch
    • Command asks you if you want a specific version (tag) or the latest
  • config
    • Update configuration from new templates, and inform of changes
    • If new parameters are required in the setup.yml file, you'll be notified
  • db
    • To update indices (if needed) - idempotent
  • deploy
    • Updates the cluster
    • If service configuration has changed, services will restart automatically

(Option) Run make diff to see the changes, file by file.