How to develop plugins

Development tooling

The plugin is a plain javascript file, associated to a json to manage metadata and configuration.

The javascript file may be bundled with any available bundler on the market:

• webpack
• parcel
• ...

tip

It is interesting to have the plugin sourcemap available, at least when developing!

How to test / debug

Make your plugin available to Spider

To test / debug a plugin, the best way us to:

• Have a local HTTP server exposing both:
  • plugin manifest: {nameOfPlugin}.json
  • plugin javascript file: {nameOfPlugin}.js
• Expose it under a HTTPS domain
  • Use nginx for instance
  • Have the domain name added to your /etc/hosts file - or use localhost
  • Use a SSL certificate signed by a CA
  • Have this CA public key configured as a trusted host in your browser

Why HTTPS?

Spider is most often exposed in an HTTPS domain. An, in fact, an HTTPS website may ONLY download HTTPS exposed javascript files.

This HTTPS setup is a bit cumbersome, but you'll have to do it only once.

Below are step-by-step instructions to perform it under a Linux host, using a docker-compose file to expose the web server, and certificates already configured for you!

Step-by-step instructions

Plugins development helpers are present in https://gitlab.com/spider-analyzer/plugins/certificate-decoder repository.

They allow:

• Exposing the plugin as HTTPS
• Fast rebuild the plugin on any change
• Source map exposure for debugging

Load the HTTPS CA in your browser

Install the localCA certificate authority in your own Browser.

• In Chrome, go to chrome://settings/certificates
• In Authorities tab, import the file ./server/localCA.pem

Launch development web server

Requirements: You need docker and docker-compose set up.

1. Launch the development stack

• docker-compose up -d

This runs:

• A NGINX server with CORS managed
• A small HTTP server managed by parcel bundler, that exposes the plugin files published by live build using parcel serve

The plugin is accessible at:

• https://localhost/{nameOfPlugin}.js

Load the plugin

You may then import the plugin in Spider Settings details, Plugins tab.

• https://localhost/{nameOfPlugin}.json

Result:

note

You must have created the manifest in ./dist folder

Reloading on changes

Default setup does not support hot reloading of the bundle.
I could not make it work 😅

However, it still rebuilds the plugin on any file change! 😀

• No need to run npm run build every time.
• A change in file triggers parcel, that rebuilds the bundle in real time.
• This bundle is not optimised, it is in development mode.

You then have to reload it manually on Spider:

• Either using the reload icon in the plugin tab
• Or by simply reloading the page in the browser ;)

Exposing plugins

Once built, the plugins may be either:

• exposed by yourselves
• uploaded to Spider Plugins Store

Expose a plugin by ourselves

You may expose the plugin javascript and manifest files on your own managed servers.
The only important point is to configure CORS to allow cross site request originated from your Spider domain.

Plugins Store

Spider Plugins Store allow you to expose your plugin in a highly available and easy way, directly from Spider itself.

Uploading

On Spider UI, upload the manifest and plugin code:

note

To upload a plugin on the UI, your account requires a specific right

When uploading manifest and code, Spider extracts information from them to confirm you what you are uploading:

• @type
• @id
• @version
• name & description of the plugins
• name & description of the defined parameters

Spider tells you if there is an existing plugin in the store and its current metadata.

Spider also tells you if a mismatch is found between:

• Existing and new version, on:
  • @type
  • name
  • description
• New manifest and new code, on:
  • @type
  • @id
  • @version

tip

To let Spider find the type, id and version in {code}.js file, you should use injectPlugin at the end of the file, with the real 'strings' as parameters (not variables)

injectPlugin({
  id: 'my-own-plugin',
  type: 'http-headers-decode-plugin',
  version: '1.0',
  func: myOwnPlugin,
  errorMessage: 'Error executing my great plugin!'
});

Storage and privacy

Depending on your setup, the plugin is stored either:

• Locally in a dedicated service in Spider cluster
• Remotely in AWS Ireland S3 service and exposed by AWS API gateway

In the second case, the plugin is still kept private, but it is available from all instances of Spider in your organisation.
Indeed, only Spider instances using licenses bought from the same organisation will see these plugins.

Installing

Once in the store, installing a plugin is one click away for users.