openapi: 3.0.0 info: description: >- --- # Description Spider API allows you to: - configure your system (teams, users, agents) - search, browse and compute statistics on the resources created while parsing the network communications - upload new communications - and even write a new agent! You may search through Packets, Tcp Sessions, Http Communications, download them, analyse them with ElasticSearch power, rebuild the content... These are the services behind the GUI. In fact, all Spider processing is done using these APIs. --- ## How to start? 1. Create a [Service Account](https://spider-analyzer.io/docs/operating/apiIntegration/) on the UI 1. Start by authenticating with `POST /customer/v1/service-accounts/sessions` to get your JWT token to use in all further calls. 2. Click 'Authorize' button to register your token. 3. Then use any customers authorized API. Ex: - `POST /web-read/v1/http-com/_search` to search for HTTP communications - `GET /web-read/v1/http-com/{id}/res/body/` to get the response body of a communication 4. When using a team, you may have to get the team token to access its whisperers - `GET /teams/v1/teams/{id}/user-token/` --- version: 2.14.0 title: Spider API contact: email: contact@floocus.com tags: - name: Common description: Common API for all services. - name: Alert description: Alert service. - name: Controllers description: Controllers are able to spawn Whisperers in a remote cluster. - name: Customers description: Users accounts. - name: Teams description: Teams of users. - name: Packets description: Network packets, as captured on the wire(less). ANSI layer 3. - name: Tcp sessions description: >- TCP sessions: consistent stateful communications of packets between 2 hosts. Can contain any kind of exchanges. ANSI layer 4. - name: Http communications description: >- HTTP communications: unitary communications used on the Web. ANSI layer 5-7. - name: PostgreSQL communications description: >- PostgreSQL communications: unitary communications using PostgreSQL protocol. ANSI layer 5-7. - name: Tls keys description: TLS keys used to encrypt TLS sessions. ANSI layer 5. - name: Whisp description: To control Whisperers, the probes sending sniffed communications to Spider - name: Ciphers description: To control Gociphers, the probes sending TLS keys to Spider - name: Hosts description: The list of Hosts detected by Whisperers - name: Whisp status description: Runtime metrics from Whisperers (status, speed, cpu...) - name: Ciphers status description: Runtime metrics from Gociphers (status, speed, cpu...) - name: Links description: Shareable links from Spider UI - name: Sessions description: Statistics from UI sessions - name: Jobs description: Traces of upload, download and purge Jobs - name: MailSender description: Service to send emails - name: GuiLogs description: Logs of errors generated on GUI - name: GuiSettings description: User settings for GUI - name: Plugins description: Plugins store backend or proxy - name: Monitor description: Access monitoring metrics and logs from Spider - name: Pollers description: Pollers statistics paths: /{service}/: get: tags: - Common summary: Get micro service info description: |- ### Description Give basic info, can be used for healthcheck of service ### Access - No identification parameters: - in: path name: service schema: $ref: '#/components/schemas/ServiceNames' description: Service name required: true responses: '200': content: text/plain: schema: type: string example: Spider Customer description: The micro service basic information. /{service}/v1/stats/api/: get: tags: - Common security: - Bearer: [] summary: Get Api stats description: |- ### Description Give metrics informations for HTTP Api of this service ### Access - Admin - Application - User parameters: - in: path name: service schema: $ref: '#/components/schemas/ServiceNames' required: true responses: '200': content: application/json: schema: $ref: '#/components/schemas/ApiStats' description: Api stats /{service}/v1/stats/cb/: get: tags: - Common security: - Bearer: [] summary: Get circuit breakers stats description: >- ### Description Give circuit breakers informations for downstream connections of this service ### Access - Admin - Application - User parameters: - in: path name: service schema: $ref: '#/components/schemas/ServiceNames' required: true responses: '200': content: application/json: schema: $ref: '#/components/schemas/CbStats' description: Circuit breakers stats /{service}/v1/stats/process/: get: tags: - Common security: - Bearer: [] summary: Get process stats description: |- ### Description Give process metrics for this service ### Access - Admin - Application - User parameters: - in: path name: service schema: $ref: '#/components/schemas/ServiceNames' required: true responses: '200': content: application/json: schema: $ref: '#/components/schemas/ProcessStats' description: Process stats /{parser}/v1/stats/parsing/: get: tags: - Common security: - Bearer: [] summary: Get parsing stats description: |- ### Description Give metrics informations for parsing ### Access - Admin - Application - User parameters: - in: path name: parser schema: $ref: '#/components/schemas/ParserNames' required: true responses: '200': content: application/json: schema: $ref: '#/components/schemas/ParsingStats' description: Parsing stats /alert/metrics/: get: tags: - Alert summary: Get metrics description: >- ### Description Give metrics information collected by alerting probes in Prometheus format. ### Access - Free responses: '200': content: text/plain; version=0.0.4: schema: example: > # HELP spider_logs_errors_per_min Tells how many error logs per min have been emitted over the last minutes # TYPE spider_logs_errors_per_min gauge spider_logs_errors_per_min 5 # HELP spider_whisperers_status_per_min Tells how many whisperers status are received per min # TYPE spider_whisperers_status_per_min gauge spider_whisperers_status_per_min 126 # HELP spider_elasticsearch_disk_free_space_bytes Tells Elasticsearch nodes free storage # TYPE spider_elasticsearch_disk_free_space_bytes gauge spider_elasticsearch_disk_free_space_bytes{node="elasticsearch-es-default-0"} 116.267578125 spider_elasticsearch_disk_free_space_bytes{node="elasticsearch-es-default-3"} 297.740234375 spider_elasticsearch_disk_free_space_bytes{node="elasticsearch-es-default-1"} 558.912109375 spider_elasticsearch_disk_free_space_bytes{node="elasticsearch-es-default-2"} 345.0810546875 description: Alert metrics /alert/health/: get: tags: - Alert summary: Get health description: |- ### Description Give summary of probes status ### Access - Free - Exposed responses: '200': content: applications/json: schema: type: object properties: license: type: object properties: name: type: string description: Name of license expires: type: string format: date description: Expiration date for this license endpoint: type: string format: uri description: Endpoint Spider is exposed probes: type: object additionalProperties: type: object properties: status: type: string enum: - ACTIVE - INACTIVE - UNKNOWN description: | Status of the probe * ACTIVE - issue, * INACTIVE - everything is right, * UNKNOWN - probe did collect data statusSince: type: string format: date-time description: Last time the status changed lastChecked: type: string description: Last time the probe ran example: license: name: MyCompany build factory expires: '2024-11-30' endpoint: https://spider.mycompany.com probes: tooManyLogs: status: INACTIVE statusSince: '2024-01-28T16:18:01.127Z' lastChecked: '2024-01-28T21:39:02.365Z' noNewStatus: status: INACTIVE statusSince: '2024-01-28T16:18:01.128Z' lastChecked: '2024-01-28T21:39:01.869Z' description: Probes status /controls/v1/controllers/: post: tags: - Controllers security: - Bearer: [] summary: Create a new Controller description: |- ### Description Create a new controller, and associate it to the owner customer. ### Access - User, with controllers creation rights - Admin requestBody: description: Controller creation request content: application/json: schema: type: object properties: customer: type: string description: System Id of the customer. name: type: string description: Name of the controller to create. required: - customer - name example: customer: YOD66VZ54Jih name: Upload required: true responses: '201': description: Controller created headers: Location: schema: type: string description: '@id of the created Controller' '400': $ref: '#/components/responses/TooBig' '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '406': $ref: '#/components/responses/WrongAccept' '411': $ref: '#/components/responses/NoParameter' '415': $ref: '#/components/responses/WrongBodyContentType' '422': $ref: '#/components/responses/InvalidInputs' /controls/v1/controllers/_search: post: tags: - Controllers security: - Bearer: [] summary: Search for Controllers description: |- ### Description Search for controllers ### Rules If client is not admin, the search will be limited to the Controllers owned by this customer or shared with him, directly or by the team. ### Access - Admin - User requestBody: $ref: '#/components/requestBodies/ConfSearches' responses: '200': $ref: '#/components/responses/Collection' '400': $ref: '#/components/responses/TooBig' '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '406': $ref: '#/components/responses/WrongAccept' '411': $ref: '#/components/responses/NoParameter' '415': $ref: '#/components/responses/WrongBodyContentType' '422': $ref: '#/components/responses/InvalidInputs' /controls/v1/defaultConfig: get: tags: - Controllers summary: Get default Configuration for Controllers description: |- ### Description Get default configuration. ### Access - User - Admin security: - Bearer: [] responses: '200': description: The default configuration headers: eTag: schema: type: string description: eTag of the resource state '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '404': $ref: '#/components/responses/NotFound' '406': $ref: '#/components/responses/WrongAccept' /controls/v1/controllers/{id}: parameters: - in: path name: id schema: type: string description: Internal id of Controller required: true get: tags: - Controllers summary: Get a Controller description: |- ### Description Get a controller's details. ### Access - User owning the controller - The own controller - User being shared access to this controller - User of the team being shared access to this controller - Customer application - Admin security: - Bearer: [] responses: '200': description: The requested Controller content: application/json: schema: $ref: '#/components/schemas/Controller' headers: eTag: schema: type: string description: eTag of the resource state '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '404': $ref: '#/components/responses/NotFound' '406': $ref: '#/components/responses/WrongAccept' patch: tags: - Controllers security: - Bearer: [] summary: Change controller's name or shared users description: >- ### Description Updates a customer. You can: - Update controller's name - Change sharing settings: teams, users and users rights on this controller ### Rules - Can do any change: - Admin - User owning the controller - User being shared access to this controller with share rights can: - Add or remove users from the sharing settings - User being shared access to this controller with rights change rights can: - Change rights of users in the sharing settings ### Access - User wning the controller - User of a team having shared access to the controller - User being shared access to this controller with config, share or rights modification rights - Admin parameters: - in: header name: If-Match description: eTag of previous state of the controller schema: type: string required: true requestBody: description: Json patch with the changes required: true content: application/json-patch+json: schema: $ref: '#/components/schemas/JsonPatch' responses: '204': description: Controller has been updated headers: eTag: schema: type: string description: eTag of the resource state '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '404': $ref: '#/components/responses/NotFound' '406': $ref: '#/components/responses/WrongAccept' '411': $ref: '#/components/responses/NoParameter' '412': $ref: '#/components/responses/ConcurrencyError' '415': $ref: '#/components/responses/WrongBodyContentType' '422': $ref: '#/components/responses/InvalidInputs' '428': $ref: '#/components/responses/MissingETag' delete: tags: - Controllers summary: Delete a Controller description: |- ### Description Delete a controller. - Put it to technicalStatus DELETED. ### Access - User wning the controller - User being shared access to this controller and with delete right - Admin security: - Bearer: [] responses: '204': $ref: '#/components/responses/NoContent' '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '404': $ref: '#/components/responses/NotFound' '406': $ref: '#/components/responses/WrongAccept' /controls/v1/controllers/{id}/apikey/: post: tags: - Controllers security: - Bearer: [] summary: Create/Replace the controller API key description: >- ### Description Create/Replace the controller API key used for controller connection to Spider. The API key is a public/private key pair. - The public key is stored in controller's settings. - The private key is sent in response to this call. Any call to this API (if authorized) will overwrite the previous API key, and the controller will not be able to use the previous one. A connected controller will be disconnected when its current JWT token will expire. The API key is taken as a configuration AT START of controllers, and need a restart to be changed. ### Output The API can supports two outputs: - application/json: - Provides a file with the private key, Spider's URL, and the controller's id This file is the only expected configuration file at controller start. - application/x-pem-file - Provides only the private key as a PEM file ### Access - User wning the controller - Admin parameters: - in: path name: id schema: type: string description: System id of Controller required: true responses: '201': description: Controller API key headers: Content-Disposition: schema: type: string description: Attachment and filename Cache-Control: schema: type: string description: Private content: application/json: schema: type: object properties: controller: type: string description: Id of Controller spiderConfigURI: type: string description: URI of Spider to get configuration privatePem: type: string description: Private PEM used for secured connection example: controller: abcdefghijklmnopqrstuv spiderConfigURI: >- https://spider.streetsmart.global/controls/v1/controllers/abcdefghijklmnopqrstuv/config?view=client privatePem: | -----BEGIN RSA PRIVATE KEY----- line1 line2 line3 ... -----END RSA PRIVATE KEY----- application/x-pem-file: schema: type: string example: | -----BEGIN RSA PRIVATE KEY----- line1 line2 line3 ... -----END RSA PRIVATE KEY----- '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '404': $ref: '#/components/responses/NotFound' '406': $ref: '#/components/responses/WrongAccept' /controls/v1/controllers/{id}/apikey/signature/: post: tags: - Controllers security: - Bearer: [] summary: Generate an API key signature with the private key of the controller description: >- ### Description This endpoint is for testing purposes only: - It allows testing the API key or the controllers API without a controller - It generates a valid signature for the controller to call the configuration endpoint - However, IT REQUIRES YOUR PRIVATE KEY IN INPUT - After testing, please, reset your API key ### Output - The signature for this controller, timestamp and private API key - A validation of this signature with the controller registered public API key ### Access - User owning the controller - Admin parameters: - in: path name: id schema: type: string description: System id of Controller required: true - in: query name: timeStamp description: Timestamp to use in signature schema: type: string format: date-time required: true - in: query name: instanceId description: InstanceId to use in signature schema: type: string required: true requestBody: content: application/x-pem-file: schema: description: The controller RSA private key, as a PEM type: string example: |- -----BEGIN RSA PRIVATE KEY----- ... -----END RSA PRIVATE KEY----- responses: '201': description: Controller signature and validation content: application/x-pem-file: schema: type: object properties: signature: type: string format: base64 description: Signature of the controller matchPubKey: type: boolean description: True if the signature - the private key - is valid '400': description: No API key in controller config content: application/json: schema: $ref: '#/components/schemas/Error' '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '404': $ref: '#/components/responses/NotFound' '406': $ref: '#/components/responses/WrongAccept' '411': $ref: '#/components/responses/NoParameter' '415': $ref: '#/components/responses/WrongBodyContentType' '422': $ref: '#/components/responses/InvalidInputs' /controls/v1/controllers/{id}/config/: parameters: - in: path name: id schema: type: string description: System id of Controller required: true get: tags: - Controllers security: - Bearer: [] summary: Get Controller configuration description: > ### Description Get the configuration of this Controller Usages: - Called by controller at start (or before token expiration) with their API key - Called by controllers regularly to check for configuration change - Called by UI to get configuration ### API key The first call from controllers is made by: - building a json payload with current time and controller id, - then signing it with SHA256 algorithm using controller's private key ``` const timeStamp = moment().toISOString(); const info = { timeStamp, controllerId }; const privKey = new NodeRSA(privatePem); const signature = privKey.sign(Buffer.from(JSON.stringify(info)), 'base64'); ``` - and then calling this API with specific headers: ``` Spider-TimeStamp: timeStamp Spider-Signature: signature //base 64 encoded ``` ### Output - The configuration - A JWT token to use on further calls in Spider-Token header - If no token provided, or if called from a Customer - A Customer may call to get the configuration of one of its controllers and use the generated token to upload data (as on the UI) ### Access - The own controller - User owning the controller - User being shared access to this controller - User of the team being shared access to this controller - Admin parameters: - in: header name: Spider-Signature schema: type: string format: base64 description: Signature of the call by the Controller, with its API key required: false - in: header name: Spider-Timestamp schema: type: string format: date-time description: >- Provided with API key in first Controller call to get its JWT token with the config required: false responses: '200': description: Controller configuration, with attachments list headers: eTag: schema: type: string description: eTag of the resource state (the generated config) Spider-Token: schema: type: string description: JWT token for the controller '400': description: Request timestamp is too old - when connecting with API key content: application/json: schema: $ref: '#/components/schemas/Error' '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '404': $ref: '#/components/responses/NotFound' '406': $ref: '#/components/responses/WrongAccept' '500': description: Controller does not have an API key - when connecting with API key content: application/json: schema: $ref: '#/components/schemas/Error' patch: tags: - Controllers security: - Bearer: [] summary: Change Controller's config description: >- ### Description Updates a Controller configuration. ### Rules - Can do any change: - Admin - User owning the Controller - User being shared access to this Controller with config rights can: - Change configuration settings - User being shared access to this Controller with share rights can: - Change sharing settings ### Access - User owning the Controller - User being shared access to this Controller with config or share rights - Admin parameters: - in: header name: If-Match description: eTag of previous state of the Controller's config schema: type: string required: true requestBody: description: Json patch with the changes required: true content: application/json-patch+json: schema: $ref: '#/components/schemas/JsonPatch' responses: '204': description: Controller's configuration has been updated headers: eTag: schema: type: string description: eTag of the resource state '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '404': $ref: '#/components/responses/NotFound' '406': $ref: '#/components/responses/WrongAccept' '411': $ref: '#/components/responses/NoParameter' '412': $ref: '#/components/responses/ConcurrencyError' '415': $ref: '#/components/responses/WrongBodyContentType' '422': $ref: '#/components/responses/InvalidInputs' '428': $ref: '#/components/responses/MissingETag' /controls/v1/controllers/{id}/namespaces: parameters: - in: path name: id schema: type: string description: System id of Controller required: true get: tags: - Controllers security: - Bearer: [] summary: Get namespaces of the cluster where the Controller is description: > ### Description Get the list of namespaces names from the cluster. ### Output - An array of string Returns 404 if the controller is not connected. ### Access - User owning the controller - User being shared access to this controller with attach right - User of the team being shared access to this controller with attach right - Admin responses: '200': description: List of namespaces content: application/json: schema: type: array items: type: string '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '404': $ref: '#/components/responses/NotFound' '406': $ref: '#/components/responses/WrongAccept' /controls/v1/controllers/{id}/namespaces/{namespace}/collection/{collection}: parameters: - in: path name: id schema: type: string description: System id of Controller required: true - in: path name: namespace schema: type: string description: Name of the namespace we want the object from required: true - in: path name: collection schema: type: string enum: - pods - statefulsets - deployments - cronjobs description: Collection we are interested in required: true get: tags: - Controllers security: - Bearer: [] summary: >- Get list of objects of the requested namespace and collection from the cluster description: > ### Description Get the list of objects from the cluster. ### Output - An array of items Returns 404 if the controller is not connected. ### Access - User owning the controller - User being shared access to this controller with attach right - User of the team being shared access to this controller with attach right - Admin responses: '200': description: List of namespaces content: application/json: schema: type: array items: type: object properties: name: type: string children: type: object description: Recursive list of children objects with types properties: name: type: string objectType: type: string children: type: object example: - name: alert children: - objectType: replicasets name: alert-855f6f6548 children: - objectType: pods name: alert-855f6f6548-vbsfp - name: capture-status-poller children: - objectType: replicasets name: capture-status-poller-5695d8487c children: - objectType: pods name: capture-status-poller-5695d8487c-cflqp '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '404': $ref: '#/components/responses/NotFound' '406': $ref: '#/components/responses/WrongAccept' /controls/v1/controllers/{id}/attachments/: parameters: - in: path name: id schema: type: string description: System id of Controller required: true post: tags: - Controllers security: - Bearer: [] summary: Create a new attachment description: >- ### Description Create a new attachment, asking, by it, to spawn a Whisperer to each linked Pod - The attachment is first saved in DB, and it will be fetched by the Controller next time with its configuration. - Then, an attachment request is sent to the Controller for an immediate attachment (if Controller is connected) ### Access - User owning the controller - User being shared access to this controller with attach right - User of the team being shared access to this controller with attach right - Admin requestBody: description: Attachment creation request content: application/json: schema: type: object properties: whisperer: type: string description: Whisperer Id to attach. namespace: type: string description: Namespace where the workload is. collection: type: string description: Collection of the workload. enum: - pods - statefulsets - daemonsets - deployments - cronjobs item: type: string description: Name of the workload. agent: type: string description: Agent to attach. enum: - whisperer - gossiper required: - whisperer - namespace - collection - item required: true responses: '201': description: Attachment created headers: Location: schema: type: string description: '@id of the created Attachment' '400': $ref: '#/components/responses/TooBig' '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '406': $ref: '#/components/responses/WrongAccept' '411': $ref: '#/components/responses/NoParameter' '415': $ref: '#/components/responses/WrongBodyContentType' '422': $ref: '#/components/responses/InvalidInputs' /controls/v1/controllers/{id}/attachments/{attachment}/containers/{container}/logs: parameters: - in: path name: id schema: type: string description: System id of Controller required: true - in: path name: attachment schema: type: string description: System id of the Attachment required: true - in: path name: container schema: type: string description: Container id of the Whisperer required: true get: tags: - Controllers security: - Bearer: [] summary: Get last logs of the attached Whisperer description: > ### Description Get last logs of the attached Whisperer. ### Access - User owning the controller - User being shared access to this controller with attach or monitor right - User of the team being shared access to this controller with attach or monitor right - Admin responses: '200': description: List of log events content: application/json: example: - time: '2024-12-01T09:52:51.791Z' level: 30 msg: Gossiper starting. module: main version: 7.2.1 '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '404': $ref: '#/components/responses/NotFound' '406': $ref: '#/components/responses/WrongAccept' /controls/v1/controllers/{id}/attachments/{attachment}/: parameters: - in: path name: id schema: type: string description: System id of Controller required: true - in: path name: attachment schema: type: string description: System id of the Attachment required: true delete: tags: - Controllers security: - Bearer: [] summary: Detach the attachment description: > ### Description Ask for the attachment to be terminated. The Whisperers connected to the worload linked to this attachment will terminate the next time they check their status. ### Access - User owning the controller - User being shared access to this controller with attach right - User of the team being shared access to this controller with attach right - Admin responses: '200': description: Ok '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '404': $ref: '#/components/responses/NotFound' '406': $ref: '#/components/responses/WrongAccept' /controls/v1/controllers/{id}/attachments/_search: parameters: - in: path name: id schema: type: string description: System id of Controller required: true post: tags: - Controllers security: - Bearer: [] summary: Search for Attachments description: >- ### Description Search for attachments ### Rules If client has limited access to namespaces, the search will be limited to the Attachments the user can access, directly or by the team. ### Access - Admin - User owning this Controller - User being shared access to this Controller - User of the team being shared access to this Controller requestBody: $ref: '#/components/requestBodies/DataSearches' responses: '200': $ref: '#/components/responses/Collection' '400': $ref: '#/components/responses/TooBig' '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '406': $ref: '#/components/responses/WrongAccept' '411': $ref: '#/components/responses/NoParameter' '415': $ref: '#/components/responses/WrongBodyContentType' '422': $ref: '#/components/responses/InvalidInputs' /controls/v1/controllers/{id}/whisperers/: parameters: - in: path name: id schema: type: string description: System id of Controller required: true get: tags: - Controllers security: - Bearer: [] summary: Get the Whisperers managed by this Controller description: > ### Description Returns the list of Whisperers that the Controller knows and manages. ### Access - User owning the controller - User being shared access to this controller with attach or monitor right - User of the team being shared access to this controller with attach or monitor right - Admin responses: '200': description: List of whisperers content: application/json: example: - name: spider-whisperer-qcc4tpvythic6fvayf2vtw-1693774449 namespace: spider-system pod: job-688b6ddc85-swnx9 attachment: '@id': MGO-9NowQzay09lPEytyFg '@type': Attachment controller: local-controller whisperer: qCC4TpvyThic6FVAYf2VTw item: job namespace: spider-system collection: deployments status: ATTACHED creator: ILWAjWPGQLm3Qf2eSPCAUA dateCreated: '2023-09-03T20:54:09.116Z' expires: '2023-09-03T22:54:09.116Z' editor: local-controller dateModified: '2023-09-03T20:54:10.491Z' whisperer: qCC4TpvyThic6FVAYf2VTw state: running: startedAt: '2023-09-03T20:54:09Z' '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '404': $ref: '#/components/responses/NotFound' '406': $ref: '#/components/responses/WrongAccept' /controls/v1/controllers/{id}/whisperers/sidecars: parameters: - in: path name: id schema: type: string description: System id of Controller required: true get: tags: - Controllers security: - Bearer: [] summary: Get the Whisperers deployed as Sidecars seen by this Controller description: > ### Description Returns the list of Sidecar Whisperers that the Controller knows. ### Access - User owning the controller - User being shared access to this controller with monitor right - User of the team being shared access to this controller with monitor right - Admin responses: '200': description: List of whisperers content: application/json: example: - '@id': spider-system.deployments.hosts.qCC4TpvyThic6FVAYf2VTw namespace: spider-system kind: deployments name: hosts whisperer: qCC4TpvyThic6FVAYf2VTw agent: Gossiper instances: - '@id': spider-system.hosts-649c985c8f-9shvn.hosts-whisperer pod: hosts-649c985c8f-9shvn container: hosts-whisperer node: 192.168.1.56 status: state: running startedAt: '2024-11-30T22:54:18Z' '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '404': $ref: '#/components/responses/NotFound' '406': $ref: '#/components/responses/WrongAccept' /controls/v1/controllers/{id}/whisperers/sidecars/{key}/logs: parameters: - in: path name: id schema: type: string description: System id of Controller required: true - in: path name: key schema: type: string description: >- @id of the sidecar whisperer container as returned by GET /whisperers/sidecars required: true get: tags: - Controllers security: - Bearer: [] summary: Get the last logs as Sidecar Whisperer description: > ### Description Returns the last logs of the Sidecar Whisperer ### Access - User owning the controller - User being shared access to this controller with monitor right - User of the team being shared access to this controller with monitor right - Admin responses: '200': description: List of log events content: application/json: example: - time: '2024-12-01T12:45:11.363Z' level: 30 msg: Gossiper starting. module: main version: 7.2.1 '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '404': $ref: '#/components/responses/NotFound' '406': $ref: '#/components/responses/WrongAccept' /controls/v1/controllers/{id}/gociphers/: parameters: - in: path name: id schema: type: string description: System id of Controller required: true get: tags: - Controllers security: - Bearer: [] summary: Get the Gociphers known by this Controller description: > ### Description Returns the list of Gociphers that the Controller knows. ### Access - User owning the controller - User being shared access to this controller with monitor right - User of the team being shared access to this controller with monitor right - Admin responses: '200': description: List of Gociphers content: application/json: example: - '@id': spider-system.local-gocipher namespace: spider-system kind: daemonsets gocipher: local-gocipher controller: local-controller instances: - '@id': spider-system.local-gocipher-wv6f6 pod: local-gocipher-wv6f6 container: local-gocipher node: 192.168.1.56 status: state: running startedAt: '2024-12-01T12:42:28Z' '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '404': $ref: '#/components/responses/NotFound' '406': $ref: '#/components/responses/WrongAccept' /controls/v1/controllers/{id}/gociphers/{key}/logs: parameters: - in: path name: id schema: type: string description: System id of Controller required: true - in: path name: key schema: type: string description: '@id of the Gocipher container as returned by GET /gociphers' required: true get: tags: - Controllers security: - Bearer: [] summary: Get the last logs of the Gociphers description: > ### Description Returns the last Logs of the Gocipher. ### Access - User owning the controller - User being shared access to this controller with monitor right - User of the team being shared access to this controller with monitor right - Admin responses: '200': description: List of log events content: application/json: example: - time: '2024-12-01T12:42:28.398Z' level: 30 msg: Gocipher starting. module: main version: 1.3.2 linuxKernel: 6.8.12 code: GCPH-MAIN-001 '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '404': $ref: '#/components/responses/NotFound' '406': $ref: '#/components/responses/WrongAccept' /controls/v1/controllers/{id}/network-usage/: parameters: - in: path name: id schema: type: string description: System id of Controller required: true post: tags: - Controllers security: - Bearer: [] summary: Save Network Usage description: |- ### Description Save Network Usage to make it accessible for analysis ### Access - The Controller - Admin requestBody: description: Network Usage content: application/json: schema: type: object required: - controller - minute - nodes - items properties: controller: type: string minute: type: string format: date-time nodes: type: integer minimum: 0 items: type: array items: type: object required: - client - server - ingress - egress properties: client: type: object required: - ip - actor properties: namespace: type: string name: type: string ip: type: string kind: type: string actor: type: string node: type: object properties: name: type: string ip: type: string podCIDRs: type: array items: type: string instance: type: object properties: name: type: string kind: type: string server: type: object required: - ip - actor - port properties: namespace: type: string name: type: string ip: type: string kind: type: string actor: type: string node: type: object properties: name: type: string ip: type: string podCIDRs: type: array items: type: string instance: type: object properties: name: type: string kind: type: string port: type: integer minimum: 0 ingress: type: object required: - packetsCount - totalBytes properties: packetsCount: type: integer minimum: 0 totalBytes: type: integer minimum: 0 egress: type: object required: - packetsCount - totalBytes properties: packetsCount: type: integer minimum: 0 totalBytes: type: integer minimum: 0 required: true responses: '201': description: Network usage created '400': $ref: '#/components/responses/TooBig' '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '406': $ref: '#/components/responses/WrongAccept' '411': $ref: '#/components/responses/NoParameter' '415': $ref: '#/components/responses/WrongBodyContentType' '422': $ref: '#/components/responses/InvalidInputs' /controls/v1/controllers/{id}/network-usage/_search: parameters: - in: path name: id schema: type: string description: System id of Controller required: true post: tags: - Controllers security: - Bearer: [] summary: Search for Network Usage description: >- ### Description Search for Network Usage ### Rules If client has limited access to namespaces, the search will be limited to the Attachments the user can access, directly or by the team. ### Access - Admin - User owning this Controller - User being shared access to this Controller - User of the team being shared access to this Controller requestBody: $ref: '#/components/requestBodies/DataSearches' responses: '200': $ref: '#/components/responses/Collection' '400': $ref: '#/components/responses/TooBig' '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '406': $ref: '#/components/responses/WrongAccept' '411': $ref: '#/components/responses/NoParameter' '415': $ref: '#/components/responses/WrongBodyContentType' '422': $ref: '#/components/responses/InvalidInputs' /controls/v1/whisperers/{id}/attachments/: parameters: - in: path name: id schema: type: string description: System id of a Whisperer required: true get: tags: - Controllers security: - Bearer: [] summary: >- Get the Attachments associated to this Whisperer, from all know attachments description: > ### Description Returns the list of Attachments linked to a Whisperer. The service will calls all referenced and connected Controllers to get the status of the Whisperers. ### Access - User owning the controller - User being shared access to this controller with attach or monitor right - User of the team being shared access to this controller with attach or monitor right - Admin responses: '200': description: List of attachments and whisperers instances content: application/json: example: - '@id': MGO-9NowQzay09lPEytyFg '@type': Attachment controller: local-controller whisperer: qCC4TpvyThic6FVAYf2VTw item: job namespace: spider-system collection: deployments status: ATTACHED creator: ILWAjWPGQLm3Qf2eSPCAUA dateCreated: '2023-09-03T20:54:09.116Z' expires: '2023-09-03T22:54:09.116Z' editor: local-controller dateModified: '2023-09-03T20:54:10.491Z' whisperers: - name: spider-whisperer-qcc4tpvythic6fvayf2vtw-1693774449 namespace: spider-system pod: job-688b6ddc85-swnx9 whisperer: qCC4TpvyThic6FVAYf2VTw state: running: startedAt: '2023-09-03T20:54:09Z' '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '404': $ref: '#/components/responses/NotFound' '406': $ref: '#/components/responses/WrongAccept' /controls/v1/controllers/{id}/logs/: parameters: - in: path name: id schema: type: string description: System id of a Controller required: true get: tags: - Controllers security: - Bearer: [] summary: Get the last logs of the Controller description: > ### Description Returns the last logs of the Controller. ### Access - User owning the controller - User being shared access to this controller with monitor right - User of the team being shared access to this controller with monitor right - Admin responses: '200': description: List of log events content: application/json: example: - name: controller hostname: local-controller-57f9559cfd-vbldj pid: 1 level: 30 msg: 'Controller local-controller starting. Version: 1.6.0.' time: '2024-12-01T12:42:16.553Z' v: 0 '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '404': $ref: '#/components/responses/NotFound' '406': $ref: '#/components/responses/WrongAccept' /customer/v1/sessions/: post: tags: - Customers summary: Connect a customer description: >- ### Description Connects a customer and returns a JWT token When successful, a cookie is set in the answer, containing the refresh_token. The cookie is http only, secure, with strict domain and path. ### Rules - After many unsuccessful attempts, the email will be blocked for some time. ### Access - No identification required requestBody: description: The email/password for connection required: true content: application/json: schema: type: object properties: email: description: Email type: string password: description: Password type: string format: password required: - email - password responses: '201': description: Session created / User connected content: application/json: schema: type: object properties: customer: description: Customer id type: string token: description: JWT token to use for services calls type: string '400': $ref: '#/components/responses/TooBig' '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '404': description: User deleted or draft '406': $ref: '#/components/responses/WrongAccept' '411': $ref: '#/components/responses/NoParameter' '415': $ref: '#/components/responses/WrongBodyContentType' '422': $ref: '#/components/responses/InvalidInputs' delete: tags: - Customers summary: Logs out a customer description: |- ### Description Logs out a customer and clears the refresh token cookie ### Access - The cookie with the refresh token is expected in input responses: '204': description: Disconnected '401': $ref: '#/components/responses/Unauthenticated' '404': $ref: '#/components/responses/NotFound' '422': $ref: '#/components/responses/InvalidInputs' /customer/v1/sessions/from/code: post: tags: - Customers summary: Connect a customer using OIDC code flow description: |- ### Description Connects a customer using OIDC and returns a JWT token. Takes in input the `code` provided by the Identity Provider and the `name` of the IP, as set in configuration. ### Access - No identification required requestBody: description: The code and IP to get the tokens from required: true content: application/json: schema: type: object properties: code: description: Code received from the authorization_endpoint type: string provider: description: Identity provider name set in configuration type: string required: - code - name responses: '201': description: Session created / User connected content: application/json: schema: type: object properties: customer: description: Customer id type: string token: description: JWT token to use for services calls type: string '400': $ref: '#/components/responses/TooBig' '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '404': description: User deleted or draft content: application/json: schema: $ref: '#/components/schemas/Collection' '406': $ref: '#/components/responses/WrongAccept' '411': $ref: '#/components/responses/NoParameter' '415': $ref: '#/components/responses/WrongBodyContentType' '422': $ref: '#/components/responses/InvalidInputs' /customer/v1/sessions/for/service-account: post: tags: - Customers summary: Connect a service account using OAuth2 client-credentials flow description: >- ### Description Connects a service account and returns a JWT token. Takes in input the `client_id` and `client_secret` of the service account. * `grant_type` must be equals to "client_credentials" * when using `application/x-www-form-urlencoded` content type, client_id and client_secret may also be provided as a Basic Authorization header ### Access - No identification required requestBody: description: The service account credentials required: true content: application/json: schema: type: object properties: client_id: type: string client_secret: type: string grant_type: type: string enum: - client_credentials required: - client_id - client_secret - grant_type application/x-www-form-urlencoded: schema: type: object properties: client_id: type: string client_secret: type: string grant_type: type: string enum: - client_credentials required: - client_id - client_secret - grant_type responses: '201': description: Session created / Service account connected content: application/json: schema: type: object properties: access_token: description: JWT token to use for services calls type: string token_type: type: string enum: - Bearer expires_in: type: number description: Validity in seconds of the token '400': $ref: '#/components/responses/TooBig' '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '404': description: User deleted or draft '406': $ref: '#/components/responses/WrongAccept' '411': $ref: '#/components/responses/NoParameter' '415': $ref: '#/components/responses/WrongBodyContentType' '422': $ref: '#/components/responses/InvalidInputs' /customer/v1/sessions/token: get: tags: - Customers summary: Generate a new access token and refreshes the refresh token description: |- ### Description Rotate the access token and issue a new refres token for a user responses: '200': description: Session created / User connected content: application/json: schema: type: object properties: customer: description: Customer id type: string token: description: JWT token to use for services calls type: string '401': $ref: '#/components/responses/Unauthenticated' '404': description: User deleted or draft '406': $ref: '#/components/responses/WrongAccept' '409': description: Token already refreshed or invalid content: application/json: schema: $ref: '#/components/schemas/Error' '422': $ref: '#/components/responses/InvalidInputs' /customer/v1/customers/: post: tags: - Customers security: - Bearer: [] summary: Create a new customer description: >- ### Description Create a new customer. ### Rules - If a customer with same email already exists, creation is cancelled. - Depending on system settings, a self created customer (no token) will be: - Created as Draft but checked for all fields when admin activation is required - Created as Active when no activation is required #### Draft customer All fields are optional at start, but checked for correctness. - It then creates a DRAFT resource. - To update it to ACTIVE - Some fields are required. - Use PATCH. #### Active customer To create an ACTIVE customer, all mandatory fields must be set. Status must be set as ACTIVE. ### Access Either: - Self creation and no identification is required - Creation by someone else: - An admin user or user with users creation rights - Only a user with rights modification rights can create a user with rights - Only an admin can create an admin - Creation by a userTrainer, with trainee rights set. requestBody: description: The customers details content: application/json: schema: type: object description: Based on https://schema.org/person properties: email: type: string description: Customer's email. _password: type: string description: Customer's password. minLength: 6 _admin: type: boolean description: True if user is an administrator. address: type: object properties: addressCountry: description: The country. For example, USA. type: string addressLocality: description: The locality. For example, Mountain View. type: string addressRegion: description: The region. For example, CA. type: string postalCode: description: The postal code. For example, 94043. type: string streetAddress: description: The street address. For example, 1600 Ampitheatre Pkwy. type: string required: - addressCountry birthDate: description: Date of birth. type: string format: date honorificPrefix: description: An honorific prefix preceding a name such as Dr/Mrs/Mr. type: string givenName: description: The given name, the first name. type: string familyName: description: The family name, the last name. type: string nationality: description: Nationality. type: string jobTitle: description: The job title (for example, Financial Manager). type: string worksFor: description: Organizations'name that the person works for. type: string required: - email - _password - address - givenName - familyName - nationality example: email: example@gmail.com _password: YGIUHIdzzf!/85F _admin: true givenName: John familyName: Doe nationality: American address: addressCountry: France technicalStatus: DRAFT required: true responses: '201': description: Customer created headers: Location: schema: type: string description: '@id of the created customer' '400': $ref: '#/components/responses/TooBig' '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '406': $ref: '#/components/responses/WrongAccept' '409': description: Customer with same email already exists content: application/json: schema: $ref: '#/components/schemas/Error' '411': $ref: '#/components/responses/NoParameter' '415': $ref: '#/components/responses/WrongBodyContentType' '422': $ref: '#/components/responses/InvalidInputs' /customer/v1/service-accounts/: post: tags: - Customers security: - Bearer: [] summary: Create a new service account description: |- ### Description Create a new service account. #### Draft service account All fields are optional at start, but checked for correctness. - It then creates a DRAFT resource. - To update it to ACTIVE - Some fields are required. - Use PATCH. #### Active customer To create an ACTIVE service account, all mandatory fields must be set. Status must be set as ACTIVE. ### Access - Creation by someone else: - An admin user or user with service accounts creation rights - Only a user with rights modification rights can create a service account with rights - Only an admin can create an admin requestBody: description: The service account details content: application/json: schema: type: object properties: name: type: string description: type: string _password: type: string description: client_secret minLength: 6 _admin: type: boolean description: True if service account is an administrator. worksFor: description: Organizations'name that the person works for. type: string required: - name - _password example: name: Spider bot _password: YGIUHOIUHIOEUKBEZUICUYEGIIdzzf!/85F technicalStatus: ACTIVE required: true responses: '201': description: Service Account created headers: Location: schema: type: string description: '@id of the created service account' '400': $ref: '#/components/responses/TooBig' '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '406': $ref: '#/components/responses/WrongAccept' '411': $ref: '#/components/responses/NoParameter' '415': $ref: '#/components/responses/WrongBodyContentType' '422': $ref: '#/components/responses/InvalidInputs' /customer/v1/customers/_search: post: tags: - Customers security: - Bearer: [] summary: Search for Customers or Service Accounts description: |- ### Description Search for customers. _Also available by GET method on the collection_ ### Access - Admin - User with user or service account management rights - userTrainer (but may only search on HIS trainees) requestBody: $ref: '#/components/requestBodies/ConfSearches' responses: '200': $ref: '#/components/responses/Collection' '400': $ref: '#/components/responses/TooBig' '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '406': $ref: '#/components/responses/WrongAccept' '411': $ref: '#/components/responses/NoParameter' '415': $ref: '#/components/responses/WrongBodyContentType' '422': $ref: '#/components/responses/InvalidInputs' /customer/v1/customers/{id}: parameters: - in: path name: id schema: type: string description: Customer internal @id required: true get: tags: - Customers security: - Bearer: [] summary: Get a customer's or service account details description: |- ### Description Get a customer's details. - `_password` field is always stripped out. - Deleted customers are only visible by admins. - Depending on access rights, returns diverse representations. #### Clients getting full details - The own client - Admins - User s with users administration rights Response is: - Full content - Minus _* fields (password, admin...) #### Clients getting system details - Whisp application (to update linked whisperers) Response is limited to: - @id - email / name - rights - whisperers #### Clients getting shortened info - Any other client Response is limited to: - @id - email ### Access - Customer, expect when using public links - Admin - User with users management right - User impersonating the user to open - User with userTrainer rights opening one of its trainee - Whisp application (to update linked whisperers) responses: '200': description: The requested customer. content: application/json: schema: $ref: '#/components/schemas/Customer' headers: eTag: schema: type: string description: eTag of the resource state '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '404': $ref: '#/components/responses/NotFound' '406': $ref: '#/components/responses/WrongAccept' patch: tags: - Customers security: - Bearer: [] summary: Update customer's or service account details description: >- ### Description Updates a customer. You can: - Update customer details - Change password - Set new rights (and admin flag) - Change Whisperer names (for synchro) - Change its state from DRAFT to ACTIVE ### Rules - Patch must be done with resource previous eTag - eTag must be given in ifMatch header (* not authorized) - eTag must match current eTag - Technical fields are protected (@id, creator, dateCreated, @type) - For a customer to change its password or email, patch operations must include previous password value inside a test operation - `{ "op":"test", "path":"_password", "value":"xxx" }` - When customer changes email: - If a customer with same email already exists, operation is cancelled. - A confirmation email challenge is sent. The account is blocked at connection until the mail is confirmed. - A mail is sent to old email - When customer changes password: - A confirmation mail is sent - When a customer changes from Draft to Active - A information mail is sent - User cannot change its own Whisperers list - Admins can: - Update a customer's details ONLY when DRAFT - Also clients with creation right - Update rights - Also clients with rights admin right - Reinit password without providing old one - Also clients with password admin right - Set a customer as admin - Change a DELETED user back to ACTIVE - User details, once in ACTIVE state, can only be modified by own customer - Whisp & Maintenance services can: - Update associated whisperers - _password and _admin field cannot be removed, copied or moved - email field cannot be removed or moved ### Access - Admin - Own customer - Whisp service (to update linked whisperers) - User with rights administration right - User with user creation right - User with password change/reinit right parameters: - in: header name: If-Match description: eTag of previous state of the customer schema: type: string required: true requestBody: description: Json patch with the changes required: true content: application/json-patch+json: schema: $ref: '#/components/schemas/JsonPatch' responses: '204': description: Customer has been updated '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '404': $ref: '#/components/responses/NotFound' '406': $ref: '#/components/responses/WrongAccept' '411': $ref: '#/components/responses/NoParameter' '412': $ref: '#/components/responses/ConcurrencyError' '415': $ref: '#/components/responses/WrongBodyContentType' '422': $ref: '#/components/responses/InvalidInputs' '428': $ref: '#/components/responses/MissingETag' delete: tags: - Customers security: - Bearer: [] summary: Delete a customer or a service account description: >- ### Description Set a customer to DELETED status. When the customer has one to many own Whisperers of UPLOAD type, they are also deleted. ### Access - Admin - User s with delete customer right - User with userTrainer right deleting on of its trainee account responses: '204': description: Customer deleted (or already deleted) '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '404': $ref: '#/components/responses/NotFound' '406': $ref: '#/components/responses/WrongAccept' /customer/v1/customers/by/email/{email}: get: tags: - Customers security: - Bearer: [] summary: Get a customer info by its email description: |- ### Description *Same as GET /customer/v1/customers/{id}* operationId: getACustomerByEmail parameters: - in: path name: email schema: type: string description: Customer email required: true responses: '200': description: The requested customer. content: application/json: schema: $ref: '#/components/schemas/Customer' headers: eTag: schema: type: string description: eTag of the resource state '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '404': $ref: '#/components/responses/NotFound' '406': $ref: '#/components/responses/WrongAccept' /customer/v1/customers/{id}/user-token: parameters: - in: path name: id schema: type: string description: Customer internal @id required: true - in: query name: useUserRights schema: type: boolean description: Ask to use user's right in the token. Keep caller's right if false. required: false get: tags: - Customers summary: Get user token to impersonate this customer security: - Bearer: [] description: >- ### Description Generates a token for a user to be able to use another user whisperers and rights. ### Rules - User must not be deleted - Only an administrator may impersonate another administrator ### Output - Generate a new token with * The requested customer id in `impersonated` field. * The customer's whisperers * The customers rights (if `useUserRights` is 'true') - The token can be used to call any API - The services will behave as if the customer was calling, except that all traces and audit fields will be valued with the original caller's id. ### Access - User with impersonate right - User with userTrainer right impersonating one of its trainee - Admin responses: '200': description: User token content: application/json: schema: type: object properties: token: type: string '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '404': $ref: '#/components/responses/NotFound' '406': $ref: '#/components/responses/WrongAccept' /customer/v1/password/challenge/: post: tags: - Customers summary: Create a password challenge description: |- ### Description Create a password challenge to reinitialize a password. A token is created and sent by mail with a redirection link to the user. The redirection link: - launches an UI presenting a form to enter a new password - contains a unique token, valid once and a limited time ### Rules - Account with this email must exist and not be deleted ### Access - No identification required requestBody: description: The email of the account required: true content: application/json: schema: type: object properties: email: description: Email type: string redirectUrl: description: >- Base Url to construct the redirection link. Expected: Login UI endpoint. type: string required: - email - redirectUrl responses: '202': description: Request accepted, mail sent. '400': $ref: '#/components/responses/TooBig' '404': description: User deleted or not found content: application/json: schema: $ref: '#/components/schemas/Error' '406': $ref: '#/components/responses/WrongAccept' '411': $ref: '#/components/responses/NoParameter' '415': $ref: '#/components/responses/WrongBodyContentType' '422': $ref: '#/components/responses/InvalidInputs' /customer/v1/password/: post: tags: - Customers summary: Set a password after a password challenge description: |- ### Description Set a new password to a user with: - The challenge token - The user email - The password (plain) An email is sent to the user to inform him of password change. Connections error count is reset ;) ### Rules - Account with this email must exist and not be deleted - Token must still exists (not used, not too old) - Token must be associated to right email and user ### Access - No identification required requestBody: description: The new password of the account required: true content: application/json: schema: type: object properties: email: description: Email type: string token: description: Token sent in challenge type: string password: description: Password choosen by the user type: string required: - email - token - password responses: '204': description: Request accepted, password updated, mail sent. '400': $ref: '#/components/responses/TooBig' '404': description: User deleted or token not found content: application/json: schema: $ref: '#/components/schemas/Error' '406': $ref: '#/components/responses/WrongAccept' '411': $ref: '#/components/responses/NoParameter' '415': $ref: '#/components/responses/WrongBodyContentType' '422': $ref: '#/components/responses/InvalidInputs' /customer/v1/emailConfirmation/: get: tags: - Customers security: - Bearer: [] summary: Confirm user email description: >- ### Description Used in mails sent to users to confirm their email addresses. This is why it is a GET, not a POST. ### Rules - User must exist and not be deleted - The account confirmation token must exist - The token is linked to the right email and account - If user is in Draft status, an email is sent to admins for account activation - Confirmation page is shown ### Access - No identification required parameters: - in: query name: email schema: type: string description: Email required: true - in: query name: token schema: type: string description: Mail confirmation token required: true responses: '200': description: A confirmation page. content: text/html: schema: {} '204': description: Mail is already confirmed '404': description: User deleted or token not found content: application/json: schema: $ref: '#/components/schemas/Error' '406': $ref: '#/components/responses/WrongAccept' '422': $ref: '#/components/responses/InvalidInputs' /teams/v1/teams: post: tags: - Teams security: - Bearer: [] summary: Create a new Team description: |- ### Description Creates a new team and add the owner as the first full rights customer. ### Rules - A team with the same name must not exist ### Access - User with team create right - User with userTrainer right creating a training team - Admin requestBody: description: The team to create content: application/json: schema: type: object description: Extract of UI state properties: name: type: string description: Team's name. description: type: string description: Team's description. owner: type: object description: Owner of the team. properties: '@id': type: string description: Internal id of the customer. email: type: string description: Email of the customer. required: - name - owner required: true responses: '201': description: Team created headers: Location: schema: type: string description: '@id of the created team' '400': $ref: '#/components/responses/TooBig' '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '406': $ref: '#/components/responses/WrongAccept' '409': description: Team with same name already exists content: application/json: schema: $ref: '#/components/schemas/Error' '411': $ref: '#/components/responses/NoParameter' '415': $ref: '#/components/responses/WrongBodyContentType' '422': $ref: '#/components/responses/InvalidInputs' /teams/v1/teams/{id}: parameters: - in: path name: id schema: type: string description: Team internal @id required: true get: tags: - Teams security: - Bearer: [] summary: Get a team's details description: |- ### Description Get a team's details. - Depending on access rights, returns diverse representations. #### Clients getting full details - Admins - Customers from the team - Customers using this team Response is: - Full content #### Clients getting system details - Whisp application (to update linked whisperers) Response is limited to: - @id - name - whisperers #### Clients getting shortened info - Any other client Response is limited to: - @id - name ### Access - Customer - Admin - Whisp application (to update linked whisperers) responses: '200': description: The requested team. content: application/json: schema: $ref: '#/components/schemas/Team' headers: eTag: schema: type: string description: eTag of the resource state '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '404': $ref: '#/components/responses/NotFound' '406': $ref: '#/components/responses/WrongAccept' patch: tags: - Teams security: - Bearer: [] summary: Update team's details description: >- ### Description Updates a team. You can: - Update team's details - Add/remove customers - Set new customers rights - Add/remove whisperer names (for synchro) - Change team's settings ### Rules - Patch must be done with resource previous eTag - eTag must be given in ifMatch header (* not authorized) - eTag must match current eTag - Patch cannot be done on DELETED team - Technical fields are protected (@id, creator, dateCreated, @type) - Customers with share right can update customers list and access filters - Customers with settings right can update team settings - Customers with update right can update name, description - Whisp and Maintenance services can: - Update associated whisperers - Token cannot be changed with patch - A mail is sent to team administrators with changes made - After update, - Whisperers list for customers and access filters are cleaned from any removed whisperer from the team - User s list for access filters are cleaned from any removed user from the team ### Access - Admin - Whisp service (to update linked whisperers) - User with rights on team: share, whisperers, settings, update, rights parameters: - in: header name: If-Match description: eTag of previous state of the team schema: type: string required: true requestBody: description: Json patch with the changes required: true content: application/json-patch+json: schema: $ref: '#/components/schemas/JsonPatch' responses: '204': description: Customer has been updated '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '404': $ref: '#/components/responses/NotFound' '406': $ref: '#/components/responses/WrongAccept' '411': $ref: '#/components/responses/NoParameter' '412': $ref: '#/components/responses/ConcurrencyError' '415': $ref: '#/components/responses/WrongBodyContentType' '422': $ref: '#/components/responses/InvalidInputs' '428': $ref: '#/components/responses/MissingETag' delete: tags: - Teams security: - Bearer: [] summary: Delete a team description: |- ### Description Set a team to DELETED status. If the team is a training team, its own whisperers are deleted. ### Access - Admin - User s with update team right responses: '204': description: Team deleted (or already deleted) '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '404': $ref: '#/components/responses/NotFound' '406': $ref: '#/components/responses/WrongAccept' /teams/v1/teams/users/: post: tags: - Teams summary: Join a team security: - Bearer: [] description: |- ### Description Join a user to a team with: - The join token - The user @id - The user email An email is sent to the team admin to tell them about new user. ### Rules - The join token must belong to a team - Team must still be active - User must not be part of the team ### Access - Admin - User whose @id is in body requestBody: description: The new password of the account required: true content: application/json: schema: type: object properties: email: description: Email type: string token: description: Token sent in challenge type: string password: description: Password choosen by the user type: string required: - email - token - password responses: '204': description: Request accepted, team joined. '400': $ref: '#/components/responses/TooBig' '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '406': $ref: '#/components/responses/WrongAccept' '409': description: User already exists in team content: application/json: schema: $ref: '#/components/schemas/Error' '411': $ref: '#/components/responses/NoParameter' '415': $ref: '#/components/responses/WrongBodyContentType' '422': $ref: '#/components/responses/InvalidInputs' /teams/v1/teams/{id}/share-token: parameters: - in: path name: id schema: type: string description: Team internal @id required: true post: tags: - Teams summary: Add a share token to the team security: - Bearer: [] description: |- ### Description Set a share token to the team. The token is randomly generated. ### Rules - Team must not be deleted - A notification mail is sent to team admins ### Access - User being part of the team, with team update right - Admin responses: '204': description: Token created '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '404': $ref: '#/components/responses/NotFound' delete: tags: - Teams summary: Remove team share token security: - Bearer: [] description: |- ### Description Delete the team share token. ### Rules - Team must not be deleted - A notification mail is sent to team admins ### Access - User being part of the team, with team update right - Admin responses: '204': description: Token removed '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '404': $ref: '#/components/responses/NotFound' /teams/v1/teams/{id}/user-token: parameters: - in: path name: id schema: type: string description: Team internal @id required: true get: tags: - Teams summary: Get user token for this team security: - Bearer: [] description: >- ### Description Generates a token for the user to be able to use team's whisperers and rights. ### Rules - Team must not be deleted ### Access - User being part of the team - Admin responses: '200': description: User token content: application/json: schema: type: object properties: token: type: string '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '404': $ref: '#/components/responses/NotFound' '406': $ref: '#/components/responses/WrongAccept' /teams/v1/teams/by/name/{name}: get: tags: - Teams security: - Bearer: [] summary: Get a team info by its name description: |- ### Description *Same as GET /teams/v1/teams/{id}* operationId: getATeamByName parameters: - in: path name: name schema: type: string description: Team name required: true responses: '200': description: The requested team. content: application/json: schema: $ref: '#/components/schemas/Team' headers: eTag: schema: type: string description: eTag of the resource state '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '404': $ref: '#/components/responses/NotFound' '406': $ref: '#/components/responses/WrongAccept' /teams/v1/teams/_search: post: tags: - Teams security: - Bearer: [] summary: Search for Teams description: >- ### Description Search for teams. _Also available by GET method on the collection_ ### Rules - When not called by admin, will limit to teams of which the user belongs. ### Access - Admin - User of the team requestBody: $ref: '#/components/requestBodies/ConfSearches' responses: '200': $ref: '#/components/responses/Collection' '400': $ref: '#/components/responses/TooBig' '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '406': $ref: '#/components/responses/WrongAccept' '411': $ref: '#/components/responses/NoParameter' '415': $ref: '#/components/responses/WrongBodyContentType' '422': $ref: '#/components/responses/InvalidInputs' /pack-write/v2/packets/: post: tags: - Packets security: - Bearer: [] summary: Push packets description: |- ### Description Process a json payload of packets and: - Make them ready for parsing. - Save them (optional). ### Checks - Spider packets structure ### Access - Whisperers requestBody: description: Json payload with packets to analyse by Spider. required: true content: application/json: schema: type: array items: type: object properties: '@id': type: string description: Unique id of the packet in the system. '@type': type: string enum: - Packet version: type: string description: Version of the schema. enum: - '2.0' name: type: string description: Name of the packet (for display). whisperer: type: string description: Whisperer that captured the packet instanceId: type: string description: Instance id of the whisperer tcpSession: type: string description: Id of the Tcp session the packet is in timestamp: type: number format: double description: Unix timestamp of capture, with microseconds length: type: integer description: Size of the packet (size of rawPacket.buf buffer) protocols: type: object description: >- List of protocols used by this packet, keys are protocols name: TCP, UDP, IPv4... additionalProperties: type: object properties: src: type: string description: >- Source identifier in this protocol (ip address, tcp port, ...) dst: type: string description: Destination identifier direction: type: string description: Direction of the packet (only for TCP protocol) enum: - in - out index: type: integer description: >- Index of the packet in the Tcp session (only for TCP protocol) packetLot: type: integer description: >- Index of the packetLot - consecutive packets with data in same direction - in the Tcp session (only for TCP protocol) relativeSeq: type: integer description: >- Relative seq of the packet in this Tcp session (only for TCP protocol) relativeAck: type: integer description: >- Relative ack of the packet in this Tcp session (only for TCP protocol) hasData: type: boolean description: If the packet has Tcp data (only for TCP protocol) rawPacket: type: object properties: '@type': type: string buf: description: Packet content in base64 type: string header: description: Packet pcap header in base64 type: string link_type: description: Type of network physical link type: string enum: - LINKTYPE_ETHERNET - LINKTYPE_NULL - LINKTYPE_RAW - LINKTYPE_IEEE802_11_RADIO - LINKTYPE_LINUX_SLL example: '@id': >- ROlrqlFhTY2ayXIxTV2uZA.rd-srv508-bes.456285.172.16.102.72-43118-172.16.102.125-8080.1682747931-5 '@type': Packet version: '2.0' commonId: >- ROlrqlFhTY2ayXIxTV2uZA.456285.172.16.102.72-43118-172.16.102.125-8080.1682747931-5 name: 456285.172.16.102.72-43118-172.16.102.125-8080#5 whisperer: ROlrqlFhTY2ayXIxTV2uZA instanceId: rd-srv508-bes tcpSession: >- ROlrqlFhTY2ayXIxTV2uZA.rd-srv508-bes.456285.172.16.102.72-43118-172.16.102.125-8080.1682747931 timestamp: 1642625100.188968 length: 229 protocols: Linux SLL: src: 50:6b:8d:3a:97:undefined dst: null IPv4: src: 172.16.102.125 dst: 172.16.102.72 TCP: src: '8080' dst: '43118' direction: in packetLot: 1 index: 5 relativeSeq: 1 relativeAck: 0 hasData: true rawPacket: link_type: LINKTYPE_LINUX_SLL buf: '' header: '' responses: '201': description: Packets created '400': $ref: '#/components/responses/TooBig' '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '406': $ref: '#/components/responses/WrongAccept' '411': $ref: '#/components/responses/NoParameter' '415': $ref: '#/components/responses/WrongBodyContentType' '422': $ref: '#/components/responses/InvalidInputs' /pack-read/v1/packets/_search: post: tags: - Packets security: - Bearer: [] summary: Search for packets description: |- ### Description Searches or aggregation analysis on packets _Also available by GET method on the collection_ ### Rules Admin may search without specifying a whisperer ### Access - Admin or client - Customer - Limited access to the whisperers it has access to - Taking into account access filters - Taking into account time range for public links requestBody: $ref: '#/components/requestBodies/DataSearches' responses: '200': $ref: '#/components/responses/Collection' '400': $ref: '#/components/responses/TooBig' '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '406': $ref: '#/components/responses/WrongAccept' '411': $ref: '#/components/responses/NoParameter' '415': $ref: '#/components/responses/WrongBodyContentType' '422': $ref: '#/components/responses/InvalidInputs' /pack-read/v1/packets/{id}: get: tags: - Packets security: - Bearer: [] summary: Get a packet description: |- ### Description Get a packet ### Access - Admin - Customer - Limited access to the whisperers it has access to - Taking into account access filters - Taking into account time range for public links parameters: - in: path name: id schema: type: string description: Internal id of packet required: true responses: '200': description: The requested packet content: application/json: schema: $ref: '#/components/schemas/Packet' '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '404': $ref: '#/components/responses/NotFound' '406': $ref: '#/components/responses/WrongAccept' /pack-read/v1/packets/payload/tcp/: post: tags: - Packets security: - Bearer: [] summary: Aggregate TCP payload for provided packets (fallback) description: |- ### Description Build the tcp payload of the packets listed in input. _Also available by GET method_ ### Access - Admin - Customer - Limited access to the whisperers it has access to - Taking into account access filters - Taking into account time range for public links requestBody: description: List of packets identifiers required: true content: application/json: schema: type: array items: description: System id of packet type: string responses: '200': description: Tcp payload content: binary/octet-stream: schema: type: string format: binary '400': description: Could not find all packets / Request body too big content: application/json: schema: $ref: '#/components/schemas/Error' '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '406': $ref: '#/components/responses/WrongAccept' '411': $ref: '#/components/responses/NoParameter' '415': $ref: '#/components/responses/WrongBodyContentType' '422': $ref: '#/components/responses/InvalidInputs' /pack-read/v1/packets/by/list/: post: tags: - Packets security: - Bearer: [] summary: Get a collection of packets (fallback) description: |- ### Description Get the packets listed in input (by id). _Also available by GET method_ ### Access - Admin - Application (another service) - Customer - Limited access to the whisperers it has access to - Taking into account access filters - Taking into account time range for public links requestBody: description: List of packets identifiers required: true content: application/json: schema: type: array items: description: System id of packet type: string responses: '200': description: Collection of packets content: application/json: schema: type: array items: type: object '400': description: Could not find all packets / Request body too big content: application/json: schema: $ref: '#/components/schemas/Error' '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '406': $ref: '#/components/responses/WrongAccept' '411': $ref: '#/components/responses/NoParameter' '415': $ref: '#/components/responses/WrongBodyContentType' '422': $ref: '#/components/responses/InvalidInputs' /pack-read/v1/packets/of/tcpSession/: get: tags: - Packets security: - Bearer: [] summary: >- Get a packets of a Tcp Session between two indices (or from a index to the end) description: >- ### Description Get the packets of the {tcpSession} in input, from {indexStart} to {indexEnd} (opt). ### Access - Admin - Application (another service) - Customer - Limited access to the whisperers it has access to - Taking into account access filters - Taking into account time range for public links requestBody: description: Range of packets to receive required: true content: application/json: schema: type: object properties: tcpSession: type: string description: Id of the Tcp session the packet is in indexStart: type: integer description: >- Index above which the first packet to send must be (exclusive) indexEnd: type: string description: >- Index before which the last packet to send must be (inclusive), optional responses: '200': description: Collection of packets content: application/json: schema: type: array items: type: object '400': description: Request body too big content: application/json: schema: $ref: '#/components/schemas/Error' '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '406': $ref: '#/components/responses/WrongAccept' '411': $ref: '#/components/responses/NoParameter' '415': $ref: '#/components/responses/WrongBodyContentType' '422': $ref: '#/components/responses/InvalidInputs' /pack-read/v1/packets/payload/tcp/bulk/: post: tags: - Packets security: - Bearer: [] summary: Aggregate TCP payload for several group of packets description: |- ### Description Build the tcp payload of the list of packets groups listed in input. _Also available by GET method_ ### Access - Admin - Customer - Limited access to the whisperers it has access to - Taking into account access filters - Taking into account time range for public links requestBody: description: List of packets groups required: true content: application/json: schema: type: array items: type: object properties: requestId: description: Id of group of packets, internal to client type: string packetIds: type: array items: description: System id of packet type: string responses: '200': description: Group of Tcp payloads content: application/json: schema: type: array items: type: object properties: requestId: description: Id of group of packets, internal to client. From input. type: string packetIds: type: array items: description: System id of packet. From input. type: string data: description: Array of bytes. type: array items: type: integer errorCode: description: Error code in case of error (400, 403, 422, 500). type: string errorMessage: description: Error description. type: string '400': description: Could not find all packets / Request body too big content: application/json: schema: $ref: '#/components/schemas/Error' '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '406': $ref: '#/components/responses/WrongAccept' '411': $ref: '#/components/responses/NoParameter' '415': $ref: '#/components/responses/WrongBodyContentType' '422': $ref: '#/components/responses/InvalidInputs' /pack-update/v2/packets/parsed/: post: tags: - Packets security: - Bearer: [] summary: Set packets as parsed description: >- ### Description Set the packets of {tcpSession} before {maxIndex} as parsed. When parsed, they are removed from working memory in Redis. Depending of whisperer settings to save Packets: - Poller may synchronize them to ES, then remove them from Redis. - Poller may plainly remove them from Redis. - Pack Update may directly remove them from Redis, if already processed by Poller. This API is used in real time processing of packets to optimise Redis usage and speed of processing. ### Access - Admin - Application (another service) requestBody: required: true content: application/json: schema: description: Array of TCP sessions maxIndex that have been parsed type: array items: type: object properties: tcpSession: type: string description: System id of the TCP session owning the packets maxIndex: type: integer description: Maximum index that has been parsed (inclusive) responses: '201': description: Packets set as parsed. '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '406': $ref: '#/components/responses/WrongAccept' '411': $ref: '#/components/responses/NoParameter' '415': $ref: '#/components/responses/WrongBodyContentType' '422': $ref: '#/components/responses/InvalidInputs' /pack-update/v1/packets/purge/: post: tags: - Packets security: - Bearer: [] summary: Purge packets description: |- ### Description Create a asynchronous purging job of packets. ### Access - Admin - User asking to purge only on whisperers it owns of with purge right requestBody: description: List of packets identifiers required: true content: application/json: schema: description: Purge request type: object properties: whisperers: type: array items: description: System id of whisperers on which to purge type: string from: description: >- Start unix timestamp of purge window. Up to 6 decimals for microseconds. type: number format: double to: description: >- Stop unix timestamp of purge window. Up to 6 decimals for microseconds. type: number format: double responses: '202': description: Purge job created. headers: Location: schema: type: string description: '@id of the created job' '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '406': $ref: '#/components/responses/WrongAccept' '411': $ref: '#/components/responses/NoParameter' '415': $ref: '#/components/responses/WrongBodyContentType' '422': $ref: '#/components/responses/InvalidInputs' /pack-update/v1/packets/purge/{job}: get: tags: - Packets security: - Bearer: [] summary: Get purge packets progress description: |- ### Description Get purge progress ### Access - Admin - Customer parameters: - in: path name: job schema: type: string description: Internal id of job required: true responses: '200': $ref: '#/components/responses/PurgeProgress' '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '406': $ref: '#/components/responses/WrongAccept' '411': $ref: '#/components/responses/NoParameter' '415': $ref: '#/components/responses/WrongBodyContentType' '422': $ref: '#/components/responses/InvalidInputs' /tcp-write/v2/tcp-sessions/: post: tags: - Tcp sessions security: - Bearer: [] summary: Push Tcp sessions description: >- ### Description Stores TCP sessions and trigger parsing of payload according to Whisperers parsing configuration. ### Access - Whisperers requestBody: description: Tcp Session with packets id to analyse by Spider. required: true content: application/json: schema: type: array description: Array of Tcp sessions items: type: object properties: '@id': type: string description: System id of TCP session name: type: string src: description: Client host type: object properties: ip: description: IP address type: string format: ipv4 port: description: TCP port type: integer name: description: Hostname type: string dst: description: Server host type: object properties: ip: description: IP address type: string format: ipv4 port: description: TCP port type: integer name: description: Hostname type: string state: description: State of TCP session lifecycle type: string enum: - SYN_SENT - SYN_RECEIVED - ESTABLISHED - CLOSE_WAIT - LAST_ACK - CLOSED packetsCount: description: Count of packets in the sessions type: integer synTimestamp: description: Timestamp of SYN packet type: number format: double missedSyn: description: If whisperer missed SYN type: boolean connectTimestamp: description: Timestamp when connection was established type: number format: double firstTimestamp: description: >- Timestamp of first packet (different from SYN when missedSyn) type: number format: double lastTimestamp: description: Timestamp of last packet type: number format: double out: description: Out packets (responses from server) type: object properties: ip: description: IP headers total overhead type: integer tcp: description: TCP headers total overhead type: integer payload: description: TCP payload total size type: integer initialSeq: description: For seq number in this side of communication type: integer in: description: In packets (responses from server) type: object properties: ip: description: IP headers total overhead type: integer tcp: description: TCP headers total overhead type: integer payload: description: TCP payload total size type: integer initialSeq: description: For seq number in this side of communication type: integer example: - '@id': OOX8KrBXTaKpLHhy1is0ng==.1457182652.130872.1 whisperer: OOX8KrBXTaKpLHhy1is0ng== name: 1457182652.130872.1 state: ESTABLISHED src: ip: 192.168.1.22 port: 47566 name: web-write dst: ip: 5.135.41.230 port: 80 in: ip: 2480 tcp: 3976 payload: 168567 initialSeq: 237657365354 out: ip: 1400 tcp: 2376 payload: 4506 initialSeq: 237657365354 minInSeq: 1341521279 minOutSeq: 1877837644 packetsCount: 5 firstTimestamp: 1457182652.130872 synTimestamp: 1457182652.130872 connectTimestamp: 1457182653.1638 lastTimestamp: 1457182654.871751 missedSyn: false responses: '201': description: Tcp session created '400': $ref: '#/components/responses/TooBig' '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '406': $ref: '#/components/responses/WrongAccept' '411': $ref: '#/components/responses/NoParameter' '415': $ref: '#/components/responses/WrongBodyContentType' '422': $ref: '#/components/responses/InvalidInputs' /tcp-read/v1/tcp-sessions/{id}: get: tags: - Tcp sessions security: - Bearer: [] summary: Get a TCP session description: |- ### Description Get a TCP session ### Access - Admin - Customer - Limited access to the whisperers it has access to - Taking into account access filters - Taking into account time range for public links parameters: - in: path name: id schema: type: string description: Internal id of TCP session required: true responses: '200': description: The requested TCP session content: application/json: schema: $ref: '#/components/schemas/TcpSession' headers: eTag: schema: type: string description: eTag of the resource state '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '404': $ref: '#/components/responses/NotFound' '406': $ref: '#/components/responses/WrongAccept' /tcp-read/v1/tcp-sessions/_search: post: tags: - Tcp sessions security: - Bearer: [] summary: Search for TCP sessions (fallback) description: |- ### Description Searches or aggregation analysis on TCP sessions _Also available by GET method on the collection_ ### Rules May ask for an aggregation, with size:0 and no whisperers defined: - Admin - User with admin monitoring right ### Access - Admin - Customer - Limited access to the whisperers it has access to - Taking into account access filters - Taking into account time range for public links requestBody: $ref: '#/components/requestBodies/DataSearches' responses: '200': $ref: '#/components/responses/Collection' '400': $ref: '#/components/responses/TooBig' '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '406': $ref: '#/components/responses/WrongAccept' '411': $ref: '#/components/responses/NoParameter' '415': $ref: '#/components/responses/WrongBodyContentType' '422': $ref: '#/components/responses/InvalidInputs' /tcp-read/v1/tcp-sessions/_searchStatus: post: tags: - Tcp sessions security: - Bearer: [] summary: Preaggregated search for parsing status histogram description: |- ### Description Histogram aggregation analysis on TCP sessions ### Rules Size:0, no next, no sort. - Admin - User with admin monitoring right ### Access - Admin - Customer - Limited access to the whisperers it has access to requestBody: $ref: '#/components/requestBodies/DataSearches' responses: '200': $ref: '#/components/responses/Collection' '400': $ref: '#/components/responses/TooBig' '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '406': $ref: '#/components/responses/WrongAccept' '411': $ref: '#/components/responses/NoParameter' '415': $ref: '#/components/responses/WrongBodyContentType' '422': $ref: '#/components/responses/InvalidInputs' /tcp-update/v1/tcp-sessions/{id}: patch: tags: - Tcp sessions security: - Bearer: [] summary: Modify the parsing result of a TCP session description: |- ### Description Update the Tcp session once the parsing is done. ### Rules - Only /parsers block can be updated ### Access - Admin - Application - Whisperer - Limited access to the whisperer linked to the Tcp session parameters: - in: path name: id schema: type: string description: Internal id of TCP session required: true - in: header name: If-Match description: eTag of previous state of the TCP session schema: type: string required: true requestBody: description: Json patch with the changes required: true content: application/json-patch+json: schema: $ref: '#/components/schemas/JsonPatch' example: - op: replace path: /parsers/http/status value: PARSED - op: replace path: /parsers/http/lastParsing value: '2017-01-08T20:34:30.013Z' responses: '204': description: Tcp session has been updated headers: Location: schema: type: string description: '@id of the updated Tcp session' eTag: schema: type: string description: eTag of the new resource state '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '404': $ref: '#/components/responses/NotFound' '406': $ref: '#/components/responses/WrongAccept' '411': $ref: '#/components/responses/NoParameter' '412': $ref: '#/components/responses/ConcurrencyError' '415': $ref: '#/components/responses/WrongBodyContentType' '422': $ref: '#/components/responses/InvalidInputs' '428': $ref: '#/components/responses/MissingETag' /tcp-update/v1/tcp-sessions/: patch: tags: - Tcp sessions security: - Bearer: [] summary: Modify the parsing result of TCP sessions, in bulk description: |- ### Description Update Tcp sessions once the parsing is done. ### Rules - Only /parsers block can be updated ### Access - Admin - Application requestBody: description: Array of Patches to TcpSessions required: true content: application/json: schema: type: array items: type: object properties: '@id': description: System Id of the Tcp session to update type: string _eTag: description: eTag of the Tcp session to update type: string patch: $ref: '#/components/schemas/JsonPatch' example: - '@id': OOX8KrBXTaKpLHhy1is0ng==.1457182652.130872.1 _eTag: '"43-E6FGZXDbp5+5MDzDUKD0RNz6ApA"' patch: - op: replace path: /parsers/http/status value: PARSED - op: replace path: /parsers/http/lastParsing value: '2017-01-08T20:34:30.013Z' - op: replace path: /parsers/http/itemsCount value: 3 - op: replace path: /parsers/http/lastPacketLotComplete value: 6 - op: replace path: /parsers/http/lastPacketParsedIndex value: 24 responses: '200': description: Some failures happened content: application/json: schema: type: object properties: successes: description: List of Tcp sessions with successful patching type: array items: type: object properties: '@id': type: string description: System Id of the Tcp session in success failures: description: List of Tcp sessions for which Patch failed and why. type: array items: type: object properties: '@id': type: string description: System Id of the Tcp session in error code: type: string description: Code of the error enum: - ETAG_MISMATCH - PATCH_FAILED reason: type: string description: Details of the error '204': description: All TcpSessions have been updated '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '406': $ref: '#/components/responses/WrongAccept' '411': $ref: '#/components/responses/NoParameter' '415': $ref: '#/components/responses/WrongBodyContentType' '422': $ref: '#/components/responses/InvalidInputs' /tcp-update/v1/tcp-sessions/by/client_random: get: tags: - Tcp sessions security: - Bearer: [] summary: Get a set of TCP sessions with Tls by their client random description: |- ### Description Get a set of TCP session from a list of client ranndoms. ### Access - Admin - Tls keys linker application requestBody: description: Array of Whisperer+ClientRandom required: true content: application/json: schema: type: array items: type: object properties: whisperer: description: Whisperer Id of Whisperer having captured the TcpSession type: string clientRandom: description: Client random found in the TLS handshake type: string example: - whisperer: OOX8KrBXTaKpLHhy1is0ng clientRandom: >- 1fa2b42654f4deede463f394406ece35c2a3aee704a65f0d700d1bb3ca2be7a7 responses: '200': description: Array of requested TCP sessions content: application/json: schema: type: array items: type: object headers: eTag: schema: type: string description: eTag of the resource state '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '404': $ref: '#/components/responses/NotFound' '406': $ref: '#/components/responses/WrongAccept' /tcp-update/v1/parsing-jobs/{type}: post: tags: - Tcp sessions security: - Bearer: [] summary: Ask for Tcp sessions to parse by {type} parser description: |- ### Description Get Tcp sessions that needs parsing by {type} parser - Returns a collection of Tcp sessions to parse ### Access - Admin - Application parameters: - in: path name: type schema: type: string enum: - HTTP description: Type of parsing required: true requestBody: description: Polling parameters required: true content: application/json: schema: type: object properties: before: type: string format: date-time description: Date before which to get sessions to parse. Safety delay. responses: '201': $ref: '#/components/responses/ParsingJob' '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '404': $ref: '#/components/responses/NotFound' '406': $ref: '#/components/responses/WrongAccept' /tcp-update/v1/parsing-jobs/{type}/warnings/: post: tags: - Tcp sessions security: - Bearer: [] summary: Ask for Tcp sessions in Warning status to parse by {type} parser description: >- ### Description Get Tcp sessions that have already been parsed by {type} parser, but that ended in WARNING parsing status. Indeed, most often the WARNING status means that the TCP session could not be parsed do to missing packets. We retry a bit ater once packets should be there. - Returns a collection of Tcp sessions to parse ### Access - Admin - Application parameters: - in: path name: type schema: type: string enum: - HTTP description: Type of parsing required: true requestBody: description: Polling parameters required: true content: application/json: schema: type: object properties: before: type: string format: date-time description: Date before which to get sessions to parse. Safety delay. responses: '201': $ref: '#/components/responses/ParsingJob' '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '404': $ref: '#/components/responses/NotFound' '406': $ref: '#/components/responses/WrongAccept' /tcp-update/v1/tcp-sessions/purge/: post: tags: - Tcp sessions security: - Bearer: [] summary: Purge TCP sessions description: |- ### Description Create a asynchronous purging job of TCP sessions. ### Access - Admin - User asking to purge only on whisperers it owns of with purge right requestBody: description: List of TCP sessions identifiers required: true content: application/json: schema: description: Purge request type: object properties: whisperers: type: array items: description: System id of whisperers on which to purge type: string from: description: >- Start unix timestamp of purge window. Up to 6 decimals for microseconds. type: number format: double to: description: >- Stop unix timestamp of purge window. Up to 6 decimals for microseconds. type: number format: double responses: '202': description: Purge job created. headers: Location: schema: type: string description: '@id of the created job' '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '406': $ref: '#/components/responses/WrongAccept' '411': $ref: '#/components/responses/NoParameter' '415': $ref: '#/components/responses/WrongBodyContentType' '422': $ref: '#/components/responses/InvalidInputs' /tcp-update/v1/tcp-sessions/purge/{job}: get: tags: - Tcp sessions security: - Bearer: [] summary: Get purge TCP sessions progress description: |- ### Description Get purge progress ### Access - Admin - Customer parameters: - in: path name: job schema: type: string description: Internal id of job required: true responses: '200': $ref: '#/components/responses/PurgeProgress' '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '406': $ref: '#/components/responses/WrongAccept' '411': $ref: '#/components/responses/NoParameter' '415': $ref: '#/components/responses/WrongBodyContentType' '422': $ref: '#/components/responses/InvalidInputs' /tcp-update/v1/waiting-stats/: get: tags: - Tcp sessions security: - Bearer: [] summary: Get parsing queue status description: |- ### Description Get parsing queue status for TCP sessions in WAITING parsing status ### Access - Admin - Application - Customer responses: '200': description: The parsing queue stats content: application/json: schema: type: object properties: totalInWaiting: description: Number of sessions WAITING for parsing type: integer ageOfFirst: description: Age of first session in queue in ms type: integer ageOfLast: description: Age of last session in queue in ms type: integer example: totalInWaiting: 2 ageOfFirst: 2345 ageOfLast: 2456 /tls-keys/v1/tls-keys/: post: tags: - Tls keys security: - Bearer: [] summary: Push Tls keys description: |- ### Description Stores Tls keys and trigger parsing of to link them to TCP sessions. ### Access - Gociphers - Whisperers - Admin requestBody: description: Tls Keys to store. required: true content: application/json: schema: type: array description: Array of Tls Keys items: type: object example: - '@id': >- baf9dE5LRBC7Ok-wk9Fgfg-1fa2b42654f4deede463f394406ece35c2a3aee704a65f0d700d1bb3ca2be7a7 '@type': TlsKey gocipher: local-gocipher instanceId: local-gocipher-bwfbk captureDate: '2024-10-15T04:22:12.613262368Z' whisperer: baf9dE5LRBC7Ok-wk9Fgfg tlsVersion: TLS 1.3 clientRandom: >- 1fa2b42654f4deede463f394406ece35c2a3aee704a65f0d700d1bb3ca2be7a7 serverRandom: >- 088dc4da39e951395a7a006e62b7f2f9e9e205ba26ce23e09d4175a84064e963 masterKey: '' cipher: name: TLS_AES_256_GCM_SHA384 version: 3 handshakeSecret: >- 090e4cbb3fa81bd4bf613b45f785db6d5734153c045a077f45d082e4437156b7099441a493f45972802b09440d88fbce handshakeTrafficHash: >- 99d196bff2813190fe8779f638d490cec246b54eea5047e33e48a2dd33daafdf6cb38b82fb17b877da38677513bc1fc7 clientHandshakeTrafficSecret: >- 98c38930cb0a4276f22ccad85eb714f0a2260aabed3b58cfdc3bdda2d313dee54855fc650fbaf6711af61a70ecb1f63e serverHandshakeTrafficSecret: >- 20be12139df9ccbd5c4e61d9fee02e20d4a92276bfa56036086076752291cb13ef65946d78302a38303ae333781558a2 clientAppTrafficSecret: >- a19b4986e0b8acc6dee38363e77da9dce6f414a3d8280bc273df82939ed052a43f2f9d8ebc53287271565d1f2c5d2232 serverAppTrafficSecret: >- b1f7cbb2e73411993fe1a394cdcf00ab6a7da7d8319ce37aedf204480f344a76a8a8c5a0b3abec779d4a9302f585ef75 exporterMasterSecret: >- 340f2570571bbffb682f9977b0465fbfafbfe4a06be2ddfb8af7d6351ff183ae87fb16e0def4cba554c93297c2cffe29 responses: '201': description: Tls keys created '400': $ref: '#/components/responses/TooBig' '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '406': $ref: '#/components/responses/WrongAccept' '411': $ref: '#/components/responses/NoParameter' '415': $ref: '#/components/responses/WrongBodyContentType' '422': $ref: '#/components/responses/InvalidInputs' /tls-keys/v1/parsing-jobs/: post: tags: - Tls keys security: - Bearer: [] summary: Ask for new Tls keys to link to TCP sessions description: |- ### Description Get Tls keys that needs linking - Returns a collection of Tls keys to link ### Access - Admin - Application requestBody: description: Polling parameters required: true content: application/json: schema: type: object properties: before: type: string format: date-time description: Date before which to get tls keys to parse. Safety delay. responses: '201': $ref: '#/components/responses/ParsingJob' '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '404': $ref: '#/components/responses/NotFound' '406': $ref: '#/components/responses/WrongAccept' /tls-keys/v1/parsed/by/client_random: post: tags: - Tls keys security: - Bearer: [] summary: Ask for existing Tls keys to be removed from cache description: |- ### Description Remove Tls keys from cache ### Access - Admin - Application: tls-leys-linker responses: '204': description: Keys have been remove '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '404': $ref: '#/components/responses/NotFound' '406': $ref: '#/components/responses/WrongAccept' /web-write/v1/http-com/purge/: post: tags: - Http communications security: - Bearer: [] summary: Purge HTTP communications description: |- ### Description Create a asynchronous purging job of HTTP communications. ### Access - Admin - User asking to purge only on whisperers it owns of with purge right requestBody: description: List of HTTP communications identifiers required: true content: application/json: schema: description: Purge request type: object properties: whisperers: type: array items: description: System id of whisperers on which to purge type: string from: description: >- Start unix timestamp of purge window. Up to 6 decimals for microseconds. type: number format: double to: description: >- Stop unix timestamp of purge window. Up to 6 decimals for microseconds. type: number format: double responses: '202': description: Purge job created. headers: Location: schema: type: string description: '@id of the created job' '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '406': $ref: '#/components/responses/WrongAccept' '411': $ref: '#/components/responses/NoParameter' '415': $ref: '#/components/responses/WrongBodyContentType' '422': $ref: '#/components/responses/InvalidInputs' /web-write/v1/http-com/purge/{job}: get: tags: - Http communications security: - Bearer: [] summary: Get purge Http communications progress description: |- ### Description Get purge progress ### Access - Admin - Customer parameters: - in: path name: job schema: type: string description: Internal id of job required: true responses: '200': $ref: '#/components/responses/PurgeProgress' '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '406': $ref: '#/components/responses/WrongAccept' '411': $ref: '#/components/responses/NoParameter' '415': $ref: '#/components/responses/WrongBodyContentType' '422': $ref: '#/components/responses/InvalidInputs' /web-write/v1/http-pers/purge/: post: tags: - Http communications security: - Bearer: [] summary: Purge HTTP parsing logs description: |- ### Description Create a asynchronous purging job of HTTP parsing logs. ### Access - Admin - User asking to purge only on whisperers it owns of with purge right requestBody: description: List of HTTP parsing logs identifiers required: true content: application/json: schema: description: Purge request type: object properties: whisperers: type: array items: description: System id of whisperers on which to purge type: string from: description: >- Start unix timestamp of purge window. Up to 6 decimals for microseconds. type: number format: double to: description: >- Stop unix timestamp of purge window. Up to 6 decimals for microseconds. type: number format: double responses: '202': description: Purge job created. headers: Location: schema: type: string description: '@id of the created job' '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '406': $ref: '#/components/responses/WrongAccept' '411': $ref: '#/components/responses/NoParameter' '415': $ref: '#/components/responses/WrongBodyContentType' '422': $ref: '#/components/responses/InvalidInputs' /web-write/v1/http-pers/purge/{job}: get: tags: - Http communications security: - Bearer: [] summary: Get purge Http parsing logs progress description: |- ### Description Get purge progress ### Access - Admin - Customer parameters: - in: path name: job schema: type: string description: Internal id of job required: true responses: '200': $ref: '#/components/responses/PurgeProgress' '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '406': $ref: '#/components/responses/WrongAccept' '411': $ref: '#/components/responses/NoParameter' '415': $ref: '#/components/responses/WrongBodyContentType' '422': $ref: '#/components/responses/InvalidInputs' /web-parser/v1/playground: post: tags: - Http communications security: - Bearer: [] summary: Test HTTP tag & template rules against the parsing engine description: >- ### Description Runs draft tag and template rules through the actual Go (RE2) HTTP parsing engine against a set of already-parsed HTTP communications. Used by the Whisperer parsing-config "Playground". Because it uses the real engine, it reflects exactly what the parser would extract — and reports regexes the parser cannot use (RE2 rejects backreferences / lookaround). ### Access - Admin - User with whisperer-config rights on **every** whisperer referenced by the supplied communications - Not available to access-filter-restricted users requestBody: description: Communications to test against, plus the draft tag/template rules required: true content: application/json: schema: type: object properties: ids: description: '@ids of already-parsed HttpCom to test against (1–50)' type: array items: type: string tags: type: object properties: req: type: array items: type: object description: Request-scope tag rules res: type: array items: type: object description: Response-scope tag rules templates: type: array items: type: object description: Request template rules responses: '200': description: Per-com extraction results and per-rule errors. content: application/json: schema: type: object properties: results: type: object additionalProperties: type: object properties: tags: type: object properties: req: type: object res: type: object template: type: string summary: type: string error: type: string description: >- Set instead of tags/template/summary when extraction failed for this com ruleErrors: type: array items: type: object properties: scope: type: string description: '"req", "res" or "template"' name: type: string error: type: string '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '406': $ref: '#/components/responses/WrongAccept' '415': $ref: '#/components/responses/WrongBodyContentType' '422': $ref: '#/components/responses/InvalidInputs' /web-upload/v1/http-com/: post: tags: - Http communications security: - Bearer: [] summary: Import Http communications description: |- ### Description Import a collection of Http communications ### Access - Admin - Whisperer - Only Httpcoms on this Whisperer can be created requestBody: content: application/json: schema: type: object properties: '@type': type: string enum: - HttpComDownload user: type: string totalItems: description: Count of Http communications to import type: integer items: description: Array of Http communications type: array items: $ref: '#/components/schemas/HttpCommunication' responses: '201': description: Import succeeded '400': $ref: '#/components/responses/TooBig' '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '406': $ref: '#/components/responses/WrongAccept' '411': $ref: '#/components/responses/NoParameter' '415': $ref: '#/components/responses/WrongBodyContentType' '422': $ref: '#/components/responses/InvalidInputs' /web-read/v1/http-com/{id}: get: tags: - Http communications security: - Bearer: [] summary: Get an HTTP communication description: |- ### Description Get an Http communication by its Id ### Access - Admin - Customer - Limited access to the whisperers it has access to - Taking into account access filters - Taking into account time range for public links parameters: - in: path name: id schema: type: string description: System id of HTTP Communication required: true responses: '200': description: The requested HTTP Communication content: application/json: schema: $ref: '#/components/schemas/HttpCommunication' headers: eTag: schema: type: string description: eTag of the resource state '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '404': $ref: '#/components/responses/NotFound' '406': $ref: '#/components/responses/WrongAccept' /web-read/v1/http-com/{id}/{part}/headers/: get: tags: - Http communications security: - Bearer: [] summary: Get the headers part of an HTTP communication (request or response) description: |- ### Description Get the HTTP headers of an Http communication by its Id ### Access - Admin - Customer - Limited access to the whisperers it has access to - Taking into account access filters - Taking into account time range for public links parameters: - in: path name: id schema: type: string description: System id of HTTP Communication required: true - in: path name: part schema: type: string enum: - req - res description: Request or Response required: true responses: '200': description: The requested headers content: text/plain: schema: type: string '400': description: Wrong parameters content: application/json: schema: $ref: '#/components/schemas/Error' '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '404': $ref: '#/components/responses/NotFound' '406': $ref: '#/components/responses/WrongAccept' /web-read/v1/http-com/{id}/{part}/body/: get: tags: - Http communications security: - Bearer: [] summary: Get the body part of an HTTP communication (request or response) description: |- ### Description Get the payload body of an Http communication by its Id ### Output The output can be in 4 types, based on view parameter: - [not defined]: Get the payload inline with original content type and content encoding, and with filename {id}.{extension based on mime type} - raw: Get the payload as text with original encoding, inline, and with filename {id}.{extension based on mime type} - download: Get the payload with original content-type and encoding, in an attachement file {id}.{extension based on mime type} - octet-stream: Get the payload as captured on the network, in a binary attachement {id}.a ### Access - Admin - Customer - Limited access to the whisperers it has access to - Taking into account access filters - Taking into account time range for public links parameters: - in: path name: id schema: type: string description: System id of HTTP Communication required: true - in: path name: part schema: type: string enum: - req - res description: Request or Response required: true - in: query name: view schema: type: string enum: - raw - download - octet-stream description: Format of output required: true responses: '200': description: >- The requested body, or a message telling the body has been removed when parsing. content: binary/octet-stream: schema: {} text/plain: schema: {} '*/*': schema: {} headers: Content-Disposition: schema: type: string description: inline or attachement, with defined filename. '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '404': $ref: '#/components/responses/NotFound' '406': $ref: '#/components/responses/WrongAccept' /web-read/v1/http-pers/{id}: get: tags: - Http communications security: - Bearer: [] summary: Get an HTTP parsing log description: |- ### Description Get an Http parsing log by its Id ### Access - Admin - Customer - Limited access to the whisperers it has access to - Taking into account time range for public links parameters: - in: path name: id schema: type: string description: System id of HTTP Persistent Connection required: true responses: '200': description: The requested HTTP Persistent Connection content: application/json: schema: $ref: '#/components/schemas/HttpPers' headers: eTag: schema: type: string description: eTag of the resource state '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '404': $ref: '#/components/responses/NotFound' '406': $ref: '#/components/responses/WrongAccept' /web-read/v1/http-com/_search: post: tags: - Http communications security: - Bearer: [] summary: Search for Http communications description: |- ### Description Searches or aggregation analysis on Http communications _Also available by GET method on the collection_ ### Rules May ask for an aggregation, with size:0 and no whisperers defined: - Admin - User with admin monitoring right ### Access - Admin - Customer - Limited access to the whisperers it has access to - Taking into account access filters - Taking into account time range for public links requestBody: $ref: '#/components/requestBodies/DataSearches' responses: '200': $ref: '#/components/responses/Collection' '400': $ref: '#/components/responses/TooBig' '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '406': $ref: '#/components/responses/WrongAccept' '411': $ref: '#/components/responses/NoParameter' '415': $ref: '#/components/responses/WrongBodyContentType' '422': $ref: '#/components/responses/InvalidInputs' /web-read/v1/http-pers/_searchStatus: post: tags: - Http communications security: - Bearer: [] summary: Preaggregated search for parsing status histogram description: |- ### Description Histogram aggregation analysis on Http communications ### Access - Admin - Customer - Limited access to the whisperers it has access to requestBody: $ref: '#/components/requestBodies/DataSearches' responses: '200': $ref: '#/components/responses/Collection' '400': $ref: '#/components/responses/TooBig' '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '406': $ref: '#/components/responses/WrongAccept' '411': $ref: '#/components/responses/NoParameter' '415': $ref: '#/components/responses/WrongBodyContentType' '422': $ref: '#/components/responses/InvalidInputs' /pg-parser/v1/playground: post: tags: - PostgreSQL communications security: - Bearer: [] summary: Test PostgreSQL tag & template rules against the parsing engine description: >- ### Description Runs draft tag and template rules through the actual Go (RE2) PostgreSQL parsing engine against a set of already-parsed PostgreSQL communications. Used by the Whisperer parsing-config "Playground". Because it uses the real engine, it reflects exactly what the parser would extract — and reports regexes the parser cannot use (RE2 rejects backreferences / lookaround). ### Access - Admin - User with whisperer-config rights on **every** whisperer referenced by the supplied communications - Not available to access-filter-restricted users requestBody: description: Communications to test against, plus the draft tag/template rules required: true content: application/json: schema: type: object properties: ids: description: '@ids of already-parsed PgCom to test against (1–50)' type: array items: type: string tags: type: object properties: req: type: array items: type: object description: Request-scope tag rules res: type: array items: type: object description: Response-scope tag rules templates: type: array items: type: object description: Request template rules responses: '200': description: Per-com extraction results and per-rule errors. content: application/json: schema: type: object properties: results: type: object additionalProperties: type: object properties: tags: type: object properties: req: type: object res: type: object template: type: string summary: type: string error: type: string description: >- Set instead of tags/template/summary when extraction failed for this com ruleErrors: type: array items: type: object properties: scope: type: string description: '"req", "res" or "template"' name: type: string error: type: string '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '406': $ref: '#/components/responses/WrongAccept' '415': $ref: '#/components/responses/WrongBodyContentType' '422': $ref: '#/components/responses/InvalidInputs' /pg-read/v1/pg-com/{id}: get: tags: - PostgreSQL communications security: - Bearer: [] summary: Get an PostgreSQL communication description: |- ### Description Get an PostgreSQL communication by its Id ### Access - Admin - Customer - Limited access to the whisperers it has access to - Taking into account access filters - Taking into account time range for public links parameters: - in: path name: id schema: type: string description: System id of PostgreSQL Communication required: true responses: '200': description: The requested PostgreSQL Communication content: application/json: schema: $ref: '#/components/schemas/PgCommunication' headers: eTag: schema: type: string description: eTag of the resource state '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '404': $ref: '#/components/responses/NotFound' '406': $ref: '#/components/responses/WrongAccept' /pg-read/v1/pg-parsing/{id}: get: tags: - PostgreSQL communications security: - Bearer: [] summary: Get an PostgreSQL parsing log description: |- ### Description Get an PostgreSQL parsing log by its Id ### Access - Admin - Customer - Limited access to the whisperers it has access to - Taking into account time range for public links parameters: - in: path name: id schema: type: string description: System id of PostgreSQL parsing log required: true responses: '200': description: The requested PostgreSQL parsing log content: application/json: schema: $ref: '#/components/schemas/PgParsing' headers: eTag: schema: type: string description: eTag of the resource state '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '404': $ref: '#/components/responses/NotFound' '406': $ref: '#/components/responses/WrongAccept' /pg-read/v1/pg-com/_search: post: tags: - PostgreSQL communications security: - Bearer: [] summary: Search for PostgreSQL communications description: |- ### Description Searches or aggregation analysis on PostgreSQL communications _Also available by GET method on the collection_ ### Rules May ask for an aggregation, with size:0 and no whisperers defined: - Admin - User with admin monitoring right ### Access - Admin - Customer - Limited access to the whisperers it has access to - Taking into account access filters - Taking into account time range for public links requestBody: $ref: '#/components/requestBodies/DataSearches' responses: '200': $ref: '#/components/responses/Collection' '400': $ref: '#/components/responses/TooBig' '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '406': $ref: '#/components/responses/WrongAccept' '411': $ref: '#/components/responses/NoParameter' '415': $ref: '#/components/responses/WrongBodyContentType' '422': $ref: '#/components/responses/InvalidInputs' /pg-read/v1/pg-parsing/_searchStatus: post: tags: - PostgreSQL communications security: - Bearer: [] summary: Preaggregated search for parsing status histogram description: |- ### Description Histogram aggregation analysis on PostgreSQL communications ### Access - Admin - Customer - Limited access to the whisperers it has access to requestBody: $ref: '#/components/requestBodies/DataSearches' responses: '200': $ref: '#/components/responses/Collection' '400': $ref: '#/components/responses/TooBig' '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '406': $ref: '#/components/responses/WrongAccept' '411': $ref: '#/components/responses/NoParameter' '415': $ref: '#/components/responses/WrongBodyContentType' '422': $ref: '#/components/responses/InvalidInputs' /whisp/v1/whisperers/: post: tags: - Whisperers security: - Bearer: [] summary: Create a new Whisperer description: >- ### Description Create a new whisperer, and associate it to the owner customer. ### Access - Customer, with whisperers creation rights - User of the team owning the whisperer with whisperers right - User with userTrainer rights impersonating a trainee creating a whisperer for the trainee - Admin requestBody: description: Whisperer creation request content: application/json: schema: type: object properties: customer: type: string description: System Id of the customer. name: type: string description: Name of the whisperer to create. required: - customer - name example: customer: YOD66VZ54Jih name: Upload required: true responses: '201': description: Whisperer created headers: Location: schema: type: string description: '@id of the created Whisperer' '400': $ref: '#/components/responses/TooBig' '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '406': $ref: '#/components/responses/WrongAccept' '409': description: Whisperer with same name already exists for this customer content: application/json: schema: $ref: '#/components/schemas/Error' '411': $ref: '#/components/responses/NoParameter' '415': $ref: '#/components/responses/WrongBodyContentType' '422': $ref: '#/components/responses/InvalidInputs' /whisp/v1/defaultConfig: get: tags: - Whisperers summary: Get default Configuration for Whisperers description: |- ### Description Get default configuration. ### Access - User - Admin security: - Bearer: [] responses: '200': description: The default configuration headers: eTag: schema: type: string description: eTag of the resource state '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '404': $ref: '#/components/responses/NotFound' '406': $ref: '#/components/responses/WrongAccept' /whisp/v1/whisperers/_search: post: tags: - Whisperers security: - Bearer: [] summary: Search for Whisperers description: >- ### Description Search for whisperers _Also available by GET method on the collection_ ### Rules If client is not admin, the search will be limited to the Whisperers owned by this customer or shared with him. If client is using public link, - Search is done on the whisperers from its token, not based on whisperers date. - `users` and `teams` are omitted in response ### Access - Admin - Customer requestBody: $ref: '#/components/requestBodies/ConfSearches' responses: '200': $ref: '#/components/responses/Collection' '400': $ref: '#/components/responses/TooBig' '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '406': $ref: '#/components/responses/WrongAccept' '411': $ref: '#/components/responses/NoParameter' '415': $ref: '#/components/responses/WrongBodyContentType' '422': $ref: '#/components/responses/InvalidInputs' /whisp/v1/whisperers/{id}: parameters: - in: path name: id schema: type: string description: Internal id of Whisperer required: true get: tags: - Whisperers summary: Get a Whisperer description: |- ### Description Get a whisperer's details. ### Rules Only admins can see DELETED whisperers. ### Access - User owning the whisperer - The own whisperer - User being shared access to this whisperer - User of the team owning the whisperer with whisperers right - `customers` or `links` applications - A public link user having this Whisperer in the token - Admin ### Response Response varies depending on client: - Full resource for - User owning the whisperer - The own whisperer - User being shared access to this whisperer - User of the team owning the whisperer with whisperers right - User or Links applications - Full minus `users` and `teams` fields for public links users - Only id and name for all Customers security: - Bearer: [] responses: '200': description: The requested Whisperer content: application/json: schema: $ref: '#/components/schemas/Whisperer' headers: eTag: schema: type: string description: eTag of the resource state '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '404': $ref: '#/components/responses/NotFound' '406': $ref: '#/components/responses/WrongAccept' patch: tags: - Whisperers security: - Bearer: [] summary: Change whisperer's name or shared users description: >- ### Description Updates a customer. You can: - Update whisperer's name - Change sharing settings: users and users rights on this whisperer ### Rules - Can do any change: - Admin - User owning the whisperer - User of the team owning the whisperer with whisperers right - User being shared access to this whisperer with config rights can: - Change the name - User being shared access to this whisperer with share rights can: - Add or remove users from the sharing settings - User being shared access to this whisperer with rights change rights can: - Change rights of users in the sharing settings ### Side effects - When adding or removing customers of shared list, the service updates the whisperers list in the customer resource (by calling Customer service). - When adding or removing teams of shared list, the service updates the whisperers list in the team resource (by calling Customer service). ### Access - User owning the whisperer - User of the team owning the whisperer with whisperers right - User being shared access to this whisperer with config, share or rights modification rights - Admin parameters: - in: header name: If-Match description: eTag of previous state of the whisperer schema: type: string required: true requestBody: description: Json patch with the changes required: true content: application/json-patch+json: schema: $ref: '#/components/schemas/JsonPatch' responses: '204': description: Whisperer has been updated headers: eTag: schema: type: string description: eTag of the resource state '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '404': $ref: '#/components/responses/NotFound' '406': $ref: '#/components/responses/WrongAccept' '411': $ref: '#/components/responses/NoParameter' '412': $ref: '#/components/responses/ConcurrencyError' '415': $ref: '#/components/responses/WrongBodyContentType' '422': $ref: '#/components/responses/InvalidInputs' '428': $ref: '#/components/responses/MissingETag' delete: tags: - Whisperers summary: Delete a Whisperer description: |- ### Description Delete a whisperer. - Put it to technicalStatus DELETED. ### Access - User owning the whisperer - User being shared access to this whisperer and with delete right - User of the team owning the whisperer with whisperers right - Customer application - Admin security: - Bearer: [] responses: '204': $ref: '#/components/responses/NoContent' '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '404': $ref: '#/components/responses/NotFound' '406': $ref: '#/components/responses/WrongAccept' /whisp/v1/whisperers/{id}/apikey/: post: tags: - Whisperers security: - Bearer: [] summary: Create/Replace the whisperer API key description: >- ### Description Create/Replace the whisperer API key used for whisperer connection to Spider. The API key is a public/private key pair. - The public key is stored in whisperer's settings. - The private key is sent in response to this call. Any call to this API (if authorized) will overwrite the previous API key, and the whisperer will not be able to use the previous one. A connected whisperer will be disconnected when its current JWT token will expire. The API key is taken as a configuration AT START of whisperers, and need a restart to be changed. ### Output The API can supports two outputs: - application/json: - Provides a file with the private key, Spider's URL, and the whisperer's id This file is the only expected configuration file at whisperer start. - application/x-pem-file - Provides only the private key as a PEM file ### Access - User owning the whisperer - User of the team owning the whisperer with whisperers right - Admin parameters: - in: path name: id schema: type: string description: System id of Whisperer required: true responses: '201': description: Whisperer API key headers: Content-Disposition: schema: type: string description: Attachement and filename Cache-Control: schema: type: string description: Private content: application/json: schema: type: object properties: whisperer: type: string description: Id of Whisperer spiderConfigURI: type: string description: URI of Spider to get configuration privatePem: type: string description: Private PEM used for secured connection example: whisperer: abcdefghijklmnopqrstuv spiderConfigURI: >- https://spider.streetsmart.global/whisp/v1/whisperers/abcdefghijklmnopqrstuv/config?view=client privatePem: | -----BEGIN RSA PRIVATE KEY----- line1 line2 line3 ... -----END RSA PRIVATE KEY----- application/x-pem-file: schema: type: string example: | -----BEGIN RSA PRIVATE KEY----- line1 line2 line3 ... -----END RSA PRIVATE KEY----- '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '404': $ref: '#/components/responses/NotFound' '406': $ref: '#/components/responses/WrongAccept' /whisp/v1/whisperers/{id}/apikey/signature/: post: tags: - Whisperers security: - Bearer: [] summary: Generate an API key signature with the private key of the whisperer description: >- ### Description This endpoint is for testing purposes only: - It allows testing the API key or the whisperers API without a whisperer - It generates a valid signature for the whisperer to call the configuration endpoint - However, IT REQUIRES YOUR PRIVATE KEY IN INPUT - After testing, please, reset your API key ### Output - The signature for this whisperer, timestamp and private API key - A validation of this signature with the whisperer registered public API key ### Access - User owning the whisperer - Admin parameters: - in: path name: id schema: type: string description: System id of Whisperer required: true - in: query name: timeStamp description: Timestamp to use in signature schema: type: string format: date-time required: true - in: query name: instanceId description: InstanceId to use in signature schema: type: string required: true requestBody: content: application/x-pem-file: schema: description: The whisperer RSA private key, as a PEM type: string example: |- -----BEGIN RSA PRIVATE KEY----- ... -----END RSA PRIVATE KEY----- responses: '201': description: Whisperer signature and validation content: application/x-pem-file: schema: type: object properties: signature: type: string format: base64 description: Signature of the whisperer matchPubKey: type: boolean description: True if the signature - the private key - is valid '400': description: No API key in whisperer config content: application/json: schema: $ref: '#/components/schemas/Error' '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '404': $ref: '#/components/responses/NotFound' '406': $ref: '#/components/responses/WrongAccept' '411': $ref: '#/components/responses/NoParameter' '415': $ref: '#/components/responses/WrongBodyContentType' '422': $ref: '#/components/responses/InvalidInputs' /whisp/v1/whisperers/{id}/config/: parameters: - in: path name: id schema: type: string description: System id of Whisperer required: true post: tags: - Whisperers security: - Bearer: [] summary: Create the whisperer configuration description: >- ### Description Initialize the configuration of the whisperer ### Access - User owning the whisperer and whisperer creation rights - User of the team owning the whisperer with whisperers right - User with userTrainer rights impersonating a trainee creating a whisperer for the trainee - Admin requestBody: required: true content: application/json: schema: type: object properties: version: type: string client: type: object description: Settings used client side - the Sniffer properties: capture: type: object description: Global capture settings properties: mode: type: string description: Mode of capture enum: - FILE - INTERFACE - UPLOAD file: type: string description: Path to file to load interface: type: string description: Name of network interface to capture on filter: type: string description: Pcap filter slowItDown: type: integer minimum: 0 exclusiveMinimum: true description: Factor to use to slow down time when reading file captureBufferkB: type: integer minimum: 0 exclusiveMinimum: true description: Size for capture buffer (kB) packets: type: object description: Packets tracking settings properties: sendBufferSizekB: type: integer minimum: 0 exclusiveMinimum: true description: Size for buffer of packets before flushing to server sendBufferDelay: type: string format: ISO8601 duration description: >- Delay before the buffer is flushed to server (first one wins between size and delay) maxSendingInParallel: type: integer minimum: 0 exclusiveMinimum: true description: Pool size for sending maxSendingBufferLength: type: integer minimum: 0 exclusiveMinimum: true description: Size of buffer (nb of requests) when pool is full vxlan: type: object description: >- VXLAN is the standard encapsulation of packets for virtual host on a virtual network. Used by VMWare and Docker (at least). Over UDP. properties: decapsulate: type: boolean description: >- True if Whisperer should look for VXLAN packets and extract encapsulated packets from them keepOriginal: type: boolean description: >- True if Whisperer should capture both original and encapsulated packets (beware: size > x2) filterHosts: type: object description: Hostnames in these list are filtered out or in properties: hostsToTrack: description: Patterns to identify hosts to track type: array items: type: string hostsToIgnore: description: Patterns to identify hosts to ignore type: array items: type: string trackByDefault: description: If an host does not match, track or not? type: boolean waitForNameResolvingToTrack: description: Wait for having the host name to track type: boolean trackUnresolvedIp: description: When no resolution, track or not? type: boolean required: - trackByDefault dumpPackets: type: object description: Packets dump to local file properties: dumpToFile: type: boolean description: >- True if Whisperer is storing captured packets to disk fileBufferSizekB: type: integer description: Size for file buffer (kB) outputPath: type: string description: Folder where to store pcaps dnsCache: type: object description: DNS used to trach hostnames properties: customDnsServer: type: boolean description: Use standard DNS of the host or a custom one host: type: string description: IP of the DNS server port: type: integer description: DNS port to use trackIp: type: boolean description: True if Whisperer should try to get FQDN from DNS ttl: type: string format: ISO8601 duration description: Delay before refreshing an IP hostname that we track refreshRate: type: string format: ISO8601 duration description: >- Delay before refreshing an IP hostname that we do not track sendFullDelay: type: string format: ISO8601 duration description: Delay before sending full list of hosts sendUpdateDelay: type: string format: ISO8601 duration description: >- Delay before sending partial list of hosts (only updated ones) purgeDelay: type: string format: ISO8601 duration description: Delay before purging a host not seen from long required: - trackIp tcpSessions: type: object description: Tcp sessions capture settings properties: track: type: boolean description: True if Whisperer should try TCP sessions sendSessionDelay: type: string format: ISO8601 duration description: Delay before sending sessions to server sessionTimeOut: type: string format: ISO8601 duration description: >- Delay before removing a session not updated since long maxSendingInParallel: type: integer minimum: 0 exclusiveMinimum: true description: Pool size for sending maxSendingBufferLength: type: integer minimum: 0 exclusiveMinimum: true description: Size of buffer (nb of requests) when pool is full required: - track circuitBreakers: type: object description: Circuit breakers on CPU and RAM settings properties: breakOnHighCpu: description: Stop capture if CPU is above threshold type: boolean maxCpu: description: Maximum CPU allowed (10% = 0.1) type: number minimum: 0 exclusiveMinimum: true breakOnHighRam: description: Stop capture if RAM is above threshold type: boolean maxRam: description: Maximum RAM allowed in MB type: integer minimum: 0 exclusiveMinimum: true required: - capture - packets - dnsCache - tcpSessions server: type: object description: Settings used server side, for parsing properties: dns: type: object description: Settings for Hosts Lists management properties: ttl: type: string format: ISO8601 duration description: Delay before refreshing a hostname purgeDelay: type: string format: ISO8601 duration description: Delay before removing a host not seen since long maxDelta: type: string format: ISO8601 duration description: >- Max delta between two HostLists that triggers creation of a new list customNamePatterns: type: array description: Patterns used to determine shortened name of host items: type: string pack: type: object properties: savePackets: description: >- Should the packets be saved or trashed once parsing is done type: boolean tcpStreams: type: object description: Settings for Tcp sessions management properties: saveTcpSession: type: boolean description: Should the TcpSession resource be saved parseHTTP: type: boolean description: >- True if server should parse HTTP sessions for this Whisperer parseHTTPOptions: type: object description: Settings for HTTP parsing properties: portsToParse: type: array description: >- List of ports (80, 8080), or ports range ([3000-3100]) items: type: integer minimum: 0 exclusiveMinimum: true portsToIgnore: type: array description: >- List of ports (80, 8080), or ports range ([3000-3100]) items: type: integer minimum: 0 exclusiveMinimum: true parseByDefault: type: boolean description: True if server should parse by default required: - parseByDefault required: - parseHTTP webstreams: type: object properties: saveContent: description: >- Should the payload parsed by saved in the Http resource type: boolean headersToFilter: type: array description: Will remove these headers from the saved resource items: type: string urisToFilter: type: array description: Will not save resources with these URI patterns items: type: string urisToFilterReqContent: type: array description: Will not save request payload of these URI patterns items: type: string urisToFilterResContent: type: array description: Will not save response payload of these URI patterns items: type: string saveRawHeaders: description: >- Should the headers by saved as is in the Http resource type: boolean urisToFilterReqRawHeaders: type: array description: >- Will not save request raw headers of these URI patterns items: type: string urisToFilterResRawHeaders: type: array description: >- Will not save response raw headers of these URI patterns items: type: string reqTemplates: type: array description: >- Patterns to determine template of the HTTP communication items: type: object properties: name: type: string description: >- Name of the template, may include capture group pattern: type: string description: Regular expression to match toParse: type: object description: Components of the string to match properties: verb: type: boolean uri: type: boolean headers: type: boolean body: type: boolean reqTags: type: array description: Patterns to extract tags of the request items: type: object properties: name: type: string description: >- Name of the template, may include capture group pattern: type: string description: Regular expression to match toParse: type: object description: Components of the string to match properties: verb: type: boolean uri: type: boolean headers: type: boolean body: type: boolean resTags: type: array description: Patterns to extract tags of the response items: type: object properties: name: type: string description: >- Name of the template, may include capture group pattern: type: string description: Regular expression to match toParse: type: object description: Components of the string to match properties: status: type: boolean headers: type: boolean body: type: boolean saveHttpParsingLogs: type: boolean description: >- Should the httppers parsing tracking resource be saved required: - tcpStreams required: - version - client - server example: version: '0.1' client: capture: mode: INTERFACE interface: any filter: (udp port 4789) and not host spider.streetsmart.global packets: vxlan: decapsulate: true keepOriginal: false filterHosts: hostsToTrack: [] hostsToIgnore: - .*_poller[\._] - ^itproduction_context_repository - ^security_authorization_service trackByDefault: true dnsCache: trackIp: true tcpSessions: track: true server: tcpStreams: parseHTTP: true parseHTTPOptions: portsToParse: - 80 - 9200 - - 3000 - 3200 - - 5000 - 5100 portsToIgnore: - 5432 parseByDefault: true responses: '201': description: Config created headers: Location: schema: type: string description: Location of the config '400': $ref: '#/components/responses/TooBig' '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '404': $ref: '#/components/responses/NotFound' '406': $ref: '#/components/responses/WrongAccept' '409': description: Config has already been created, use PATCH to edit content: application/json: schema: $ref: '#/components/schemas/Error' '411': $ref: '#/components/responses/NoParameter' '415': $ref: '#/components/responses/WrongBodyContentType' '422': $ref: '#/components/responses/InvalidInputs' get: tags: - Whisperers security: - Bearer: [] summary: Get Whisperer configuration description: > ### Description Get the configuration of this Whisperer Usages: - Called by whisperer at start (or before token expiration) with their API key - Called by ephemeral whisperer at start with the token given by the Controller - Called by whisperers regularly to check for configuration change - Called by UI to get a whisperer token to perform upload - Called by UI to get configuration - Called by services to get whisperer configuration when parsing ### API key The first call from whisperers is made by: - building a json payload with current time and whisperer id, - then signing it with SHA256 algorithm using whisperer's private key ``` const timeStamp = moment().toISOString(); const info = { timeStamp, whispererId, instanceId }; const privKey = new NodeRSA(privatePem); const signature = privKey.sign(Buffer.from(JSON.stringify(info)), 'base64'); ``` - and then calling this API with specific headers: ``` Spider-TimeStamp: timeStamp Spider-InstanceId: instanceId, Spider-Signature: signature //base 64 encoded ``` ### Parameters The `view` parameter allows to modulate the output: #### server - Only server part (parsing) is sent - Current configuration is merged with default configuration #### client - Only client part (parsing) is sent - Current configuration is merged with default configuration #### full - Both client & server are sent - Current configuration is merged with default configuration #### null - The raw configuration is sent (without merging with defaults) - This view is used by UI to know what settings are specific, and what settings are defaults ### Output - The configuration - A JWT token to use on further calls in Spider-Token header - If no token provided, or if called from a Customer - A Customer may call to get the configuration of one of its whisperers and use the generated token to upload data (as on the UI) - If the Whisperer has a TTL associated to attachments, the token will have this as expiration instead of the default duration ### Access - The own whisperer - User owning the whisperer - User being shared access to this whisperer - User of the team owning the whisperer with whisperers right - Controller asking for a token for a Whisperer to spawn on an Pod (InstanceId) - Admin - Application parameters: - in: query name: view schema: type: string enum: - server - client - full description: Type of configuration to get required: false - in: header name: Spider-Signature schema: type: string format: base64 description: Signature of the call by the Whisperer, with its API key required: false - in: header name: Spider-Timestamp schema: type: string format: date-time description: >- Provided with API key in first Whisperer call to get its JWT token with the config required: false - in: header name: Spider-InstanceId schema: type: string description: >- Provided with API key in first Whisperer call to get its JWT token with the config required: false responses: '200': description: Whisperer configuration content: application/json: schema: $ref: '#/components/schemas/WhispererConfig' headers: eTag: schema: type: string description: eTag of the resource state (the generated config) Spider-Token: schema: type: string description: JWT token for the whisperer '400': description: Request timestamp is too old - when connecting with API key content: application/json: schema: $ref: '#/components/schemas/Error' '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '404': $ref: '#/components/responses/NotFound' '406': $ref: '#/components/responses/WrongAccept' '500': description: Whisperer does not have an API key - when connecting with API key content: application/json: schema: $ref: '#/components/schemas/Error' patch: tags: - Whisperers security: - Bearer: [] summary: Change whisperer's config description: >- ### Description Updates a Whisperer configuration. ### Rules - Can do any change: - Admin - User owning the whisperer - User being shared access to this whisperer with config rights can: - Change configuration settings - User being shared access to this whisperer with share rights can: - Change sharing settings - User being shared access to this whisperer with record rights can: - Start or stop capture ### Access - User owning the whisperer - User being shared access to this whisperer with config, share, or record rights - User of the team owning the whisperer with whisperers right - Admin parameters: - in: header name: If-Match description: eTag of previous state of the whisperer's config schema: type: string required: true requestBody: description: Json patch with the changes required: true content: application/json-patch+json: schema: $ref: '#/components/schemas/JsonPatch' responses: '204': description: Whisperer's configuration has been updated headers: eTag: schema: type: string description: eTag of the resource state '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '404': $ref: '#/components/responses/NotFound' '406': $ref: '#/components/responses/WrongAccept' '411': $ref: '#/components/responses/NoParameter' '412': $ref: '#/components/responses/ConcurrencyError' '415': $ref: '#/components/responses/WrongBodyContentType' '422': $ref: '#/components/responses/InvalidInputs' '428': $ref: '#/components/responses/MissingETag' /whisp/v1/whisperers/_tlsConfigs/: post: tags: - Whisperers security: - Bearer: [] summary: Get TLS configurations from a list of Whisperers description: |- ### Description From a list of Whisperers, provide their TLS configuration ### Access - Gocipher - Admin requestBody: content: 'application/json': schema: description: 'The whisperer RSA private key, as a PEM' type: array items: type: string description: Internal Id of Whisperers minItems: 1 responses: '200': description: Tls configurations, sorted by id headers: eTag: schema: type: string description: Hash of the response Cache-Control: schema: type: string description: Private content: application/json: schema: type: object properties: '@id': type: string '@type': type: string version: type: string whisperer: type: string description: Id of the Whisperer requiredState: type: string tlsKeys: type: object description: Subpart of Whisperer capture configuration '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '404': $ref: '#/components/responses/NotFound' '406': $ref: '#/components/responses/WrongAccept' /whisp/v1/whisperers/{id}/token/: parameters: - in: path name: id schema: type: string description: System id of Whisperer required: true get: tags: - Whisperers security: - Bearer: [] summary: Renew an ephemeral Whisperer token description: > ### Description Called by ephemeral Whisperers (only) to renew their token (they do not have an API key). If the Whisperer has a Time to Live associated to attachment, it won't renew. ### Output - A JWT token to use on further calls in Spider-Token header - If no token provided, or if called from a Customer - A Customer may call to get the configuration of one of its whisperers and use the generated token to upload data (as on the UI) ### Access - The own whisperer (only ephemerals can) - Admin responses: '200': description: Ok headers: Spider-Token: schema: type: string description: JWT token for the whisperer '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '404': $ref: '#/components/responses/NotFound' '406': $ref: '#/components/responses/WrongAccept' /ciphers/v1/gociphers/: post: tags: - Ciphers security: - Bearer: [] summary: Create a new Gocipher description: |- ### Description Create a new Gocipher, and associate it to the owner customer. ### Access - Customer, with Gociphers creation rights - Admin requestBody: description: Gocipher creation request content: application/json: schema: type: object properties: customer: type: string description: System Id of the customer. name: type: string description: Name of the Gocipher to create. required: - customer - name example: customer: YOD66VZ54Jih name: Dev cluster required: true responses: '201': description: Gocipher created headers: Location: schema: type: string description: '@id of the created Gocipher' '400': $ref: '#/components/responses/TooBig' '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '406': $ref: '#/components/responses/WrongAccept' '409': description: Gocipher with same name already exists for this customer content: application/json: schema: $ref: '#/components/schemas/Error' '411': $ref: '#/components/responses/NoParameter' '415': $ref: '#/components/responses/WrongBodyContentType' '422': $ref: '#/components/responses/InvalidInputs' /ciphers/v1/defaultConfig: get: tags: - Ciphers summary: Get default Configuration for Gociphers description: |- ### Description Get default configuration. ### Access - User - Admin security: - Bearer: [] responses: '200': description: The default configuration headers: eTag: schema: type: string description: eTag of the resource state '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '404': $ref: '#/components/responses/NotFound' '406': $ref: '#/components/responses/WrongAccept' /ciphers/v1/gociphers/_search: post: tags: - Ciphers security: - Bearer: [] summary: Search for Gociphers description: |- ### Description Search for Gociphers _Also available by GET method on the collection_ ### Rules If client is not admin, the search will be limited to the Gociphers owned by this customer or shared with him. ### Access - Admin - Customer requestBody: $ref: '#/components/requestBodies/ConfSearches' responses: '200': $ref: '#/components/responses/Collection' '400': $ref: '#/components/responses/TooBig' '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '406': $ref: '#/components/responses/WrongAccept' '411': $ref: '#/components/responses/NoParameter' '415': $ref: '#/components/responses/WrongBodyContentType' '422': $ref: '#/components/responses/InvalidInputs' /ciphers/v1/gociphers/{id}: parameters: - in: path name: id schema: type: string description: Internal id of Gocipher required: true get: tags: - Ciphers summary: Get a Gocipher description: |- ### Description Get a Gocipher's details. ### Rules Only admins can see DELETED Gociphers. ### Access - User owning the Gocipher - The own Gocipher - User being shared access to this Gocipher - User of a Team having access to this Gocipher - Admin security: - Bearer: [] responses: '200': description: The requested Gocipher content: application/json: schema: $ref: '#/components/schemas/Gocipher' headers: eTag: schema: type: string description: eTag of the resource state '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '404': $ref: '#/components/responses/NotFound' '406': $ref: '#/components/responses/WrongAccept' patch: tags: - Ciphers security: - Bearer: [] summary: Change Gocipher's name or shared users/teams description: >- ### Description Updates a customer. You can: - Update Gocipher's name - Change sharing settings: users and users rights on this Gocipher ### Rules - Can do any change: - Admin - User owning the Gocipher - User of the team owning the Gocipher with Gociphers right - User being shared access to this Gocipher with config rights can: - Change the name - User being shared access to this Gocipher with share rights can: - Add or remove users from the sharing settings - User being shared access to this Gocipher with rights change rights can: - Change rights of users in the sharing settings ### Access - User owning the Gocipher - User being shared access to this Gocipher with config, share or rights modification rights - Admin parameters: - in: header name: If-Match description: eTag of previous state of the Gocipher schema: type: string required: true requestBody: description: Json patch with the changes required: true content: application/json-patch+json: schema: $ref: '#/components/schemas/JsonPatch' responses: '204': description: Gocipher has been updated headers: eTag: schema: type: string description: eTag of the resource state '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '404': $ref: '#/components/responses/NotFound' '406': $ref: '#/components/responses/WrongAccept' '411': $ref: '#/components/responses/NoParameter' '412': $ref: '#/components/responses/ConcurrencyError' '415': $ref: '#/components/responses/WrongBodyContentType' '422': $ref: '#/components/responses/InvalidInputs' '428': $ref: '#/components/responses/MissingETag' delete: tags: - Ciphers summary: Delete a Gocipher description: |- ### Description Delete a Gocipher. - Put it to technicalStatus DELETED. ### Access - User owning the Gocipher - User being shared access to this Gocipher and with delete right - Customer application - Admin security: - Bearer: [] responses: '204': $ref: '#/components/responses/NoContent' '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '404': $ref: '#/components/responses/NotFound' '406': $ref: '#/components/responses/WrongAccept' /ciphers/v1/gociphers/{id}/apikey/: post: tags: - Ciphers security: - Bearer: [] summary: Create/Replace the Gocipher API key description: >- ### Description Create/Replace the Gocipher API key used for Gocipher connection to Spider. The API key is a public/private key pair. - The public key is stored in Gocipher's settings. - The private key is sent in response to this call. Any call to this API (if authorized) will overwrite the previous API key, and the Gocipher will not be able to use the previous one. A connected Gocipher will be disconnected when its current JWT token will expire. The API key is taken as a configuration AT START of Gociphers, and need a restart to be changed. ### Output The API can supports two outputs: - application/json: - Provides a file with the private key, Spider's URL, and the Gocipher's id This file is the only expected configuration file at Gocipher start. - application/x-pem-file - Provides only the private key as a PEM file ### Access - User owning the Gocipher - User of the team owning the Gocipher with Gociphers right - Admin parameters: - in: path name: id schema: type: string description: System id of Gocipher required: true responses: '201': description: Gocipher API key headers: Content-Disposition: schema: type: string description: Attachement and filename Cache-Control: schema: type: string description: Private content: application/json: schema: type: object properties: gocipher: type: string description: Id of Gocipher spiderConfigURI: type: string description: URI of Spider to get configuration privatePem: type: string description: Private PEM used for secured connection example: Gocipher: abcdefghijklmnopqrstuv spiderConfigURI: >- https://spider.streetsmart.global/ciphers/v1/gociphers/abcdefghijklmnopqrstuv/config?view=client privatePem: | -----BEGIN RSA PRIVATE KEY----- line1 line2 line3 ... -----END RSA PRIVATE KEY----- application/x-pem-file: schema: type: string example: | -----BEGIN RSA PRIVATE KEY----- line1 line2 line3 ... -----END RSA PRIVATE KEY----- '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '404': $ref: '#/components/responses/NotFound' '406': $ref: '#/components/responses/WrongAccept' /ciphers/v1/gociphers/{id}/apikey/signature/: post: tags: - Ciphers security: - Bearer: [] summary: Generate an API key signature with the private key of the Gocipher description: >- ### Description This endpoint is for testing purposes only: - It allows testing the API key or the Gociphers API without a Gocipher - It generates a valid signature for the Gocipher to call the configuration endpoint - However, IT REQUIRES YOUR PRIVATE KEY IN INPUT - After testing, please, reset your API key ### Output - The signature for this Gocipher, timestamp and private API key - A validation of this signature with the Gocipher registered public API key ### Access - User owning the Gocipher - Admin parameters: - in: path name: id schema: type: string description: System id of Gocipher required: true - in: query name: timeStamp description: Timestamp to use in signature schema: type: string format: date-time required: true - in: query name: instanceId description: InstanceId to use in signature schema: type: string required: true requestBody: content: application/x-pem-file: schema: description: The Gocipher RSA private key, as a PEM type: string example: |- -----BEGIN RSA PRIVATE KEY----- ... -----END RSA PRIVATE KEY----- responses: '201': description: Gocipher signature and validation content: application/x-pem-file: schema: type: object properties: signature: type: string format: base64 description: Signature of the Gocipher matchPubKey: type: boolean description: True if the signature - the private key - is valid '400': description: No API key in Gocipher config content: application/json: schema: $ref: '#/components/schemas/Error' '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '404': $ref: '#/components/responses/NotFound' '406': $ref: '#/components/responses/WrongAccept' '411': $ref: '#/components/responses/NoParameter' '415': $ref: '#/components/responses/WrongBodyContentType' '422': $ref: '#/components/responses/InvalidInputs' /ciphers/v1/gociphers/{id}/config/: parameters: - in: path name: id schema: type: string description: System id of Gocipher required: true get: tags: - Ciphers security: - Bearer: [] summary: Get Gocipher configuration description: > ### Description Get the configuration of this Gocipher Usages: - Called by Gocipher at start (or before token expiration) with their API key - Called by ephemeral Gocipher at start with the token given by the Controller - Called by Gociphers regularly to check for configuration change - Called by UI to get a Gocipher token to perform upload - Called by UI to get configuration - Called by services to get Gocipher configuration when parsing ### API key The first call from Gociphers is made by: - building a json payload with current time and Gocipher id, - then signing it with SHA256 algorithm using Gocipher's private key ``` const timeStamp = moment().toISOString(); const info = { timeStamp, GocipherId, instanceId }; const privKey = new NodeRSA(privatePem); const signature = privKey.sign(Buffer.from(JSON.stringify(info)), 'base64'); ``` - and then calling this API with specific headers: ``` Spider-TimeStamp: timeStamp Spider-InstanceId: instanceId, Spider-Signature: signature //base 64 encoded ``` ### Parameters The `view` parameter allows to modulate the output: #### server - Only server part (parsing) is sent - Current configuration is merged with default configuration #### client - Only client part (parsing) is sent - Current configuration is merged with default configuration #### full - Both client & server are sent - Current configuration is merged with default configuration #### null - The raw configuration is sent (without merging with defaults) - This view is used by UI to know what settings are specific, and what settings are defaults ### Output - The configuration - A JWT token to use on further calls in Spider-Token header - If no token provided ### Access - The own Gocipher - User owning the Gocipher - User being shared access to this Gocipher - Admin parameters: - in: query name: view schema: type: string enum: - server - client - full description: Type of configuration to get required: false - in: header name: Spider-Signature schema: type: string format: base64 description: Signature of the call by the Gocipher, with its API key required: false - in: header name: Spider-Timestamp schema: type: string format: date-time description: >- Provided with API key in first Gocipher call to get its JWT token with the config required: false - in: header name: Spider-InstanceId schema: type: string description: >- Provided with API key in first Gocipher call to get its JWT token with the config required: false responses: '200': description: Gocipher configuration content: application/json: type: object headers: eTag: schema: type: string description: eTag of the resource state (the generated config) Spider-Token: schema: type: string description: JWT token for the Gocipher '400': description: Request timestamp is too old - when connecting with API key content: application/json: schema: $ref: '#/components/schemas/Error' '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '404': $ref: '#/components/responses/NotFound' '406': $ref: '#/components/responses/WrongAccept' '500': description: Gocipher does not have an API key - when connecting with API key content: application/json: schema: $ref: '#/components/schemas/Error' patch: tags: - Ciphers security: - Bearer: [] summary: Change Gocipher's config description: |- ### Description Updates a Gocipher configuration. ### Rules - Can do any change: - Admin - User owning the Gocipher - User being shared access to this Gocipher with config rights can: - Change configuration settings - User being shared access to this Gocipher with share rights can: - Change sharing settings ### Access - User owning the Gocipher - User being shared access to this Gocipher with config or share rights - Admin parameters: - in: header name: If-Match description: eTag of previous state of the Gocipher's config schema: type: string required: true requestBody: description: Json patch with the changes required: true content: application/json-patch+json: schema: $ref: '#/components/schemas/JsonPatch' responses: '204': description: Gocipher's configuration has been updated headers: eTag: schema: type: string description: eTag of the resource state '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '404': $ref: '#/components/responses/NotFound' '406': $ref: '#/components/responses/WrongAccept' '411': $ref: '#/components/responses/NoParameter' '412': $ref: '#/components/responses/ConcurrencyError' '415': $ref: '#/components/responses/WrongBodyContentType' '422': $ref: '#/components/responses/InvalidInputs' '428': $ref: '#/components/responses/MissingETag' /ciphers-status/v1/status/: post: tags: - Ciphers status security: - Bearer: [] summary: Send status information from a Gocipher description: |- ### Description Gociphers are sending status every x seconds when started ### Access - Own whisperer - Admin requestBody: content: application/json: schema: $ref: '#/components/schemas/RawStatus' responses: '201': description: Status accepted '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '404': $ref: '#/components/responses/NotFound' '406': $ref: '#/components/responses/WrongAccept' '411': $ref: '#/components/responses/NoParameter' '415': $ref: '#/components/responses/WrongBodyContentType' '422': $ref: '#/components/responses/InvalidInputs' /ciphers-status/v1/status/by/list/: post: tags: - Ciphers status security: - Bearer: [] summary: Get a collection of current Gociphers status description: |- ### Description Get the statuses listed in input (by id). _Also available by GET method_ ### Access - Admin - Customer - Limited access to the Gociphers it has access to - Taking into account time range for public links requestBody: description: List of Gociphers identifiers required: true content: application/json: schema: type: array items: description: System id of Gocipher type: string responses: '200': description: Collection of Gociphers status content: application/json: schema: type: array items: type: object '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '406': $ref: '#/components/responses/WrongAccept' '411': $ref: '#/components/responses/NoParameter' '415': $ref: '#/components/responses/WrongBodyContentType' '422': $ref: '#/components/responses/InvalidInputs' /ciphers-status/v1/status/history/_search: post: tags: - Ciphers status security: - Bearer: [] summary: Search for raw status sent by a Gocipher description: |- ### Description Search for raw status ### Rules May ask searching without specifying a Gocipher: - Admin - User with admin monitoring rights ### Access - Admin - Customer, limited on search on its Gociphers requestBody: $ref: '#/components/requestBodies/DataSearches' responses: '200': $ref: '#/components/responses/Collection' '400': $ref: '#/components/responses/TooBig' '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '406': $ref: '#/components/responses/WrongAccept' '411': $ref: '#/components/responses/NoParameter' '415': $ref: '#/components/responses/WrongBodyContentType' '422': $ref: '#/components/responses/InvalidInputs' /ciphers-status/v1/status/_search: post: tags: - Ciphers status security: - Bearer: [] summary: Search for current status of Gociphers description: |- ### Description Search for current status of Gociphers ### Rules May ask searching without specifying a Gocipher: - Admin - User with admin monitoring rights ### Access - Admin - Customer, limited on search on its Gociphers requestBody: $ref: '#/components/requestBodies/CurrentSearches' responses: '200': $ref: '#/components/responses/Collection' '400': $ref: '#/components/responses/TooBig' '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '406': $ref: '#/components/responses/WrongAccept' '411': $ref: '#/components/responses/NoParameter' '415': $ref: '#/components/responses/WrongBodyContentType' '422': $ref: '#/components/responses/InvalidInputs' /ciphers-status/v1/targets/for/whisperer/{whisperer}: parameters: - in: path name: whisperer schema: type: string description: Internal id of Whisperer required: true get: tags: - Ciphers status security: - Bearer: [] summary: Get current targets for this Whisperer across all Gociphers description: |- ### Description Get the targets being watched for this Whisperer. _Also available by GET method_ ### Access - Admin - Customer - Limited access to the Whisperers it has access to responses: '200': description: Collection of Gociphers target content: application/json: schema: type: array items: type: object '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '406': $ref: '#/components/responses/WrongAccept' '411': $ref: '#/components/responses/NoParameter' '415': $ref: '#/components/responses/WrongBodyContentType' '422': $ref: '#/components/responses/InvalidInputs' /hosts/v1/hosts/forWhisperer/{id}/: post: tags: - Hosts security: - Bearer: [] summary: Send an hosts update for a whisperer description: >- ### Description An hosts list contains all hosts the whisperer has seen during its capture. When processing the list, Spider can create a new entry of hosts for this Whisperer or update the nearest one. Hosts lists can be sent in any order (to support upload). ### Access - Own whisperer - Admin parameters: - in: path name: id schema: type: string description: System id of Whisperer required: true requestBody: content: application/json: schema: type: array description: List of hosts items: type: object properties: ip: type: string description: IP address of host format: ipv4 name: type: string description: FQDN of host as given by DNS type: type: string description: Type of host enum: - SERVER - CLIENT - null firstSeen: type: string format: date-time description: Date of first packet seen for this host lastSeen: type: string format: date-time description: Date of last packet seen for this host lastUpdate: type: string format: date-time description: Last time the DNS was queried to update the host name required: - ip - name - type - firstSeen - lastSeen - lastUpdate minItems: 1 example: - ip: 10.0.0.116 name: terminal_businessappsconfigs_service type: SERVER lastSeen: '2019-01-27T13:43:44.260Z' lastUpdate: '2019-01-27T13:43:16.092Z' firstSeen: '2019-01-25T00:41:51.087Z' responses: '201': description: Hosts update accepted '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '404': $ref: '#/components/responses/NotFound' '406': $ref: '#/components/responses/WrongAccept' '411': $ref: '#/components/responses/NoParameter' '415': $ref: '#/components/responses/WrongBodyContentType' '422': $ref: '#/components/responses/InvalidInputs' /hosts/v1/hosts/: patch: tags: - Hosts security: - Bearer: [] summary: Change the custom name of a Host. description: |- ### Description Update an Host: - identified by an IP - between two dates - on one to many whisperers The update: - May change the customName of the host - May update several hosts lists of the same whisperer - May update several hosts lists of many whisperers ### Access - Admin - User asking to update only on its whisperers (owned or shared) requestBody: required: true content: application/json: schema: type: object properties: whisperers: description: List of whisperers on which to do the update type: array items: type: string description: System id of the whisperers minItems: 1 ip: type: string format: ipv4 description: IP of the host for which to set custom name startTime: type: number format: double description: Timestamp of period start for which to change stopTime: type: number format: double description: Timestamp of period end for which to change customName: type: string description: The new name / optional - can be null required: - whisperers - ip - startTime - stopTime example: whisperers: - YVWyUFNPRyCouvPBNmV9aw - Y2GaZbDXRLq3cj-m8Zjviw - GAf8TMCVRA-p8clfrxuduw ip: 10.0.0.236 startTime: 1548595820.545 stopTime: 1548595882.677 customName: test-service responses: '201': description: Hosts updated '400': $ref: '#/components/responses/TooBig' '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '404': $ref: '#/components/responses/NotFound' '406': $ref: '#/components/responses/WrongAccept' '411': $ref: '#/components/responses/NoParameter' '415': $ref: '#/components/responses/WrongBodyContentType' '422': $ref: '#/components/responses/InvalidInputs' /hosts/v1/hosts/_search: post: tags: - Hosts security: - Bearer: [] summary: Search for Hosts detected by a whisperer description: |- ### Description Search for hosts Returns a collection of HostsList ### Rules May ask searching without specifying a whisperer: - Admin - User with admin monitoring rights ### Access - Admin - Customer - Limited access to the whisperers it has access to - Taking into account time range for public links requestBody: $ref: '#/components/requestBodies/DataSearches' responses: '200': $ref: '#/components/responses/Collection' '400': $ref: '#/components/responses/TooBig' '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '406': $ref: '#/components/responses/WrongAccept' '411': $ref: '#/components/responses/NoParameter' '415': $ref: '#/components/responses/WrongBodyContentType' '422': $ref: '#/components/responses/InvalidInputs' /whisps-status/v1/status/: post: tags: - Whisp status security: - Bearer: [] summary: Send status information from a whisperer description: |- ### Description Whisperers are sending status every x seconds when started ### Access - Own whisperer - Admin requestBody: content: application/json: schema: $ref: '#/components/schemas/RawStatus' responses: '201': description: Status accepted '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '404': $ref: '#/components/responses/NotFound' '406': $ref: '#/components/responses/WrongAccept' '411': $ref: '#/components/responses/NoParameter' '415': $ref: '#/components/responses/WrongBodyContentType' '422': $ref: '#/components/responses/InvalidInputs' /whisps-status/v1/status/by/list/: post: tags: - Whisp status security: - Bearer: [] summary: Get a collection of current whisps status description: |- ### Description Get the statuses listed in input (by id). _Also available by GET method_ ### Access - Admin - Customer - Limited access to the whisperers it has access to - Taking into account time range for public links requestBody: description: List of whisperers identifiers required: true content: application/json: schema: type: array items: description: System id of whisperer type: string responses: '200': description: Collection of whisps status content: application/json: schema: type: array items: type: object '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '406': $ref: '#/components/responses/WrongAccept' '411': $ref: '#/components/responses/NoParameter' '415': $ref: '#/components/responses/WrongBodyContentType' '422': $ref: '#/components/responses/InvalidInputs' /whisps-status/v1/status/history/_search: post: tags: - Whisp status security: - Bearer: [] summary: Search for raw status sent by a whisperer description: |- ### Description Search for raw status ### Rules May ask searching without specifying a whisperer: - Admin - User with admin monitoring rights ### Access - Admin - Customer, limited on search on its whisperers requestBody: $ref: '#/components/requestBodies/DataSearches' responses: '200': $ref: '#/components/responses/Collection' '400': $ref: '#/components/responses/TooBig' '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '406': $ref: '#/components/responses/WrongAccept' '411': $ref: '#/components/responses/NoParameter' '415': $ref: '#/components/responses/WrongBodyContentType' '422': $ref: '#/components/responses/InvalidInputs' /whisps-status/v1/status/_searchCaptureStatus: post: tags: - Whisp status security: - Bearer: [] summary: Preaggregated search for parsing status histogram description: |- ### Description Histogram aggregation analysis on Whisperers status ### Access - Admin - Customer - Limited access to the whisperers it has access to - Taking into account time range for public links requestBody: $ref: '#/components/requestBodies/DataSearches' responses: '200': $ref: '#/components/responses/Collection' '400': $ref: '#/components/responses/TooBig' '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '406': $ref: '#/components/responses/WrongAccept' '411': $ref: '#/components/responses/NoParameter' '415': $ref: '#/components/responses/WrongBodyContentType' '422': $ref: '#/components/responses/InvalidInputs' /whisps-status/v1/status/_search: post: tags: - Whisp status security: - Bearer: [] summary: Search for current status of whisperers description: |- ### Description Search for current status of whisperers ### Rules May ask searching without specifying a whisperer: - Admin - User with admin monitoring rights ### Access - Admin - Customer, limited on search on its whisperers requestBody: $ref: '#/components/requestBodies/CurrentSearches' responses: '200': $ref: '#/components/responses/Collection' '400': $ref: '#/components/responses/TooBig' '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '406': $ref: '#/components/responses/WrongAccept' '411': $ref: '#/components/responses/NoParameter' '415': $ref: '#/components/responses/WrongBodyContentType' '422': $ref: '#/components/responses/InvalidInputs' /link/v1/links/: post: tags: - Links security: - Bearer: [] summary: Create a new Link description: |- ### Description Stores a UI state and gives a link in returns. ### Access - Customer - Admin requestBody: description: The state to link content: application/json: schema: type: object description: Extract of UI state properties: userInfo: type: object properties: whisp: type: object properties: selected: type: array items: type: string minItems: 1 impersonate: type: object properties: selected: type: string time: type: object properties: timeSpan: type: object properties: start: type: string format: date-time stop: type: string format: date-time timeAggregation: type: string main: type: object networkMap: type: object properties: origin: type: object zoom: type: object required: - origin - zoom search: type: object required: - userInfo - main - networkMap - search required: true responses: '201': description: Link created headers: Location: schema: type: string description: '@id of the created link' '400': $ref: '#/components/responses/TooBig' '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '406': $ref: '#/components/responses/WrongAccept' '411': $ref: '#/components/responses/NoParameter' '415': $ref: '#/components/responses/WrongBodyContentType' '422': $ref: '#/components/responses/InvalidInputs' /link/v1/links/_search: post: tags: - Links security: - Bearer: [] summary: Search for Links description: |- ### Description Search for links. _Also available by GET method on the collection_ ### Rules May ask searching without specifying a whisperer: - Admin - User with admin monitoring rights ### Access - Admin - Customer requestBody: $ref: '#/components/requestBodies/ConfSearches' responses: '200': $ref: '#/components/responses/Collection' '400': $ref: '#/components/responses/TooBig' '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '406': $ref: '#/components/responses/WrongAccept' '411': $ref: '#/components/responses/NoParameter' '415': $ref: '#/components/responses/WrongBodyContentType' '422': $ref: '#/components/responses/InvalidInputs' /link/v1/links/{id}: parameters: - in: path name: id schema: type: string description: Links system Id required: true get: tags: - Links security: - Bearer: [] summary: Get a link description: |- ### Description Get a link The link may be public or not ### Access for private link - Customer - Admin ### Rules - If Public link, it is not DELETED or deprecated ### Access for public link - User who created it - Admin - When using public link token - FreeAccess is true - User email is in recipients list - User email domain is in domains list responses: '200': description: The requested link. content: application/json: schema: $ref: '#/components/schemas/Link' '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '404': $ref: '#/components/responses/NotFound' '406': $ref: '#/components/responses/WrongAccept' /link/v1/publiclinks/: post: tags: - Links security: - Bearer: [] summary: Create a new Public Link description: >- ### Description Stores a UI state and gives a public link in return. ### Access - User with publishing permission on the whisperers present in the state, or on the team - Admin requestBody: required: true content: application/json: schema: type: object description: The public link to create. properties: accessFilters: type: array description: A collection of filters defining access control. items: type: object properties: view: type: string description: Type of view (e.g., HTTP, TCP) the filter applies to. query: type: string description: Optional query string for further filtering. accessForbidden: type: boolean description: >- Flag indicating if access is forbidden under this filter. required: true freeAccess: type: boolean description: >- Flag indicating if the link provides free access without restrictions. sendEmail: type: boolean description: Indicates whether an email notification should be sent. dateDeprecated: type: string format: date description: The ISO date when the public link becomes deprecated. domains: type: array description: >- List of domains whose emails have access to the public link. Starting with '@'. items: type: string required: true recipients: type: array description: Email addresses of the recipients with access. items: type: string required: true stateToShare: type: object description: Object representing the UI state to be shared. required: true required: - accessFilters - domains - recipients - stateToShare responses: '201': description: Public link created headers: Location: schema: type: string description: '@id of the created public link' content: application/json: schema: type: object properties: urlToShare: type: string description: The URL to share as a result of the API call. required: - urlToShare '400': $ref: '#/components/responses/TooBig' '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '406': $ref: '#/components/responses/WrongAccept' '411': $ref: '#/components/responses/NoParameter' '415': $ref: '#/components/responses/WrongBodyContentType' '422': $ref: '#/components/responses/InvalidInputs' /link/v1/publicLinks/_search: post: tags: - Links security: - Bearer: [] summary: Search for Public Links description: |- ### Description Search for public links. _Also available by GET method on the collection_ ### Rules - `content` is removed - `urlToShare` field is added with the Url of the link ### Access - Admin - User to the public links he created requestBody: $ref: '#/components/requestBodies/ConfSearches' responses: '200': $ref: '#/components/responses/Collection' '400': $ref: '#/components/responses/TooBig' '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '406': $ref: '#/components/responses/WrongAccept' '411': $ref: '#/components/responses/NoParameter' '415': $ref: '#/components/responses/WrongBodyContentType' '422': $ref: '#/components/responses/InvalidInputs' /link/v1/publicLinks/{id}: parameters: - in: path name: id schema: type: string description: Public link system Id required: true delete: tags: - Links security: - Bearer: [] summary: Delete a public link description: >- ### Description Set a public link to DELETED status, making it invisible to searches and get. ### Access - User that has created the public link - Admin responses: '204': description: Public link deleted (or already deleted) '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '404': $ref: '#/components/responses/NotFound' '406': $ref: '#/components/responses/WrongAccept' /link/v1/publicLinks/{id}/otp: parameters: - in: path name: id schema: type: string description: Public link system Id required: true post: tags: - Links summary: >- Create and send a One Time Password for a user trying to connect to a Public link description: >- ### Description Create a One Time Password for a user trying to connect to a Public link. Send this OTP to the email of the user. ### Rules - Link is not DELETED or deprecated ### Access - FreeAccess is true - User email is in recipients list - User email domain is in domains list requestBody: required: true content: application/json: schema: type: object description: The public link to create. properties: email: description: Email address of the user trying to access. items: type: string required: true responses: '201': description: OTP sent to the provided email content: application/json: schema: type: object properties: status: type: string description: Result status. required: - status '400': $ref: '#/components/responses/TooBig' '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '406': $ref: '#/components/responses/WrongAccept' '411': $ref: '#/components/responses/NoParameter' '415': $ref: '#/components/responses/WrongBodyContentType' '422': $ref: '#/components/responses/InvalidInputs' /link/v1/publicLinks/{id}/sessions: parameters: - in: path name: id schema: type: string description: Public link system Id required: true post: tags: - Links summary: Connects a public user and generates a public link JWT description: >- ### Description Connects a public user by generating a JWT specific to this public link, with dedicated access filters. Issues also a cookie with refresh token scoped to the token renewal api. ### Rules - Link is not DELETED or deprecated - OTP is not too old - OTP is associated to this email and link ### Processing - Email and connection date are added to public link tracked sessions ### Access - FreeAccess is true - User email is in recipients list - User email domain is in domains list requestBody: required: true content: application/json: schema: type: object description: The public link to create. properties: email: description: Email address of the user trying to access. items: type: string required: true otp: description: One Time Password associated to this email. items: type: string required: true responses: '201': description: JWT token for service calls content: application/json: schema: type: object properties: token: type: string description: Generated JWT token required: - status '400': $ref: '#/components/responses/TooBig' '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '406': $ref: '#/components/responses/WrongAccept' '411': $ref: '#/components/responses/NoParameter' '415': $ref: '#/components/responses/WrongBodyContentType' '422': $ref: '#/components/responses/InvalidInputs' delete: tags: - Links summary: Logs out a user from its public link sessions description: |- ### Description Logs out a user and clears the refresh token cookie ### Access - The cookie with the refresh token is expected in input responses: '204': description: Disconnected '401': $ref: '#/components/responses/Unauthenticated' '404': $ref: '#/components/responses/NotFound' '422': $ref: '#/components/responses/InvalidInputs' /link/v1/publicLinks/sessions/token: get: tags: - Links summary: Generate a new access token and refreshes the refresh token description: |- ### Description Rotate the access token and issue a new refresh token for a user responses: '200': description: Session created / User connected content: application/json: schema: type: object properties: token: description: JWT token to use for services calls type: string '401': $ref: '#/components/responses/Unauthenticated' '404': description: Link deleted or draft '406': $ref: '#/components/responses/WrongAccept' '409': description: Token already refreshed or invalid content: application/json: schema: $ref: '#/components/schemas/Error' '422': $ref: '#/components/responses/InvalidInputs' /session/v1/sessions/_search: post: tags: - Sessions security: - Bearer: [] summary: Search for Sessions description: |- ### Description Search for sessions. Sessions are confidential, searching them is much restricted. _Also available by GET method on the collection_ ### Access - Admin - User with admin monitoring right asking for an aggregation with size:0 requestBody: $ref: '#/components/requestBodies/DataSearches' responses: '200': $ref: '#/components/responses/Collection' '400': $ref: '#/components/responses/TooBig' '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '406': $ref: '#/components/responses/WrongAccept' '411': $ref: '#/components/responses/NoParameter' '415': $ref: '#/components/responses/WrongBodyContentType' '422': $ref: '#/components/responses/InvalidInputs' /session/v1/sessions/{id}: parameters: - in: path name: id schema: type: string description: Sessions system Id required: true get: tags: - Sessions security: - Bearer: [] summary: Get a session description: |- ### Description Get a session ### Access - User whose session it is - Admin responses: '200': description: The requested session. content: application/json: schema: $ref: '#/components/schemas/Session' '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '404': $ref: '#/components/responses/NotFound' '406': $ref: '#/components/responses/WrongAccept' put: tags: - Sessions security: - Bearer: [] summary: Save a session description: |- ### Description Save a session ### Access - User whose session it is - Admin requestBody: description: The session required: true content: application/json: schema: $ref: '#/components/schemas/Session' responses: '201': description: Session created / updated headers: Location: schema: type: string description: '@id of the created session' '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '404': $ref: '#/components/responses/NotFound' '406': $ref: '#/components/responses/WrongAccept' /mail-sender/v1/emails/: post: tags: - MailSender summary: Send an email description: |- ### Description Sends an email from the configured account. Can be configured to be connected by OAuth (gmail for instance) or login/password. Can send to one or several recipients, a text message or html. Txt message being mandatory. As it is not intended for mass mailing, it makes a connection to the SMTP server for each email. ### Access - Can only be call by applications or admin requestBody: description: The email to send required: true content: application/json: schema: type: object properties: to: type: array description: List of recipients items: description: Email of recipients type: string subject: description: Subject of the mail. Accepts unicodes characters (for icons) type: string text: description: The plain message type: string html: description: The message in html type: string required: - to - subject - text responses: '204': description: Request accepted, password updated, mail sent. '400': $ref: '#/components/responses/TooBig' '404': description: User deleted or token not found content: application/json: schema: $ref: '#/components/schemas/Error' '406': $ref: '#/components/responses/WrongAccept' '411': $ref: '#/components/responses/NoParameter' '415': $ref: '#/components/responses/WrongBodyContentType' '422': $ref: '#/components/responses/InvalidInputs' /job/v1/jobs/: post: tags: - Jobs security: - Bearer: [] summary: Create a new Job description: |- ### Description Create a job ### Process - If type is PurgeJob - Launches the purge on all selected resources - Check progress and update the job regularly until success or failure ### Access - Admin - User creating a Job on its whisperers requestBody: description: The Job details content: application/json: schema: type: object description: Based on https://schema.org/action properties: jobType: type: string description: Type of job enum: - PurgeJob - DownloadJob - UploadJob - ... jobParameters: description: Input of the job type: object progress: description: Progress in % type: integer whisperer: description: List of whisperers type: array items: description: System id of whisperers type: string startTime: description: Start of the job type: string format: date-time endTime: description: End of the job type: string format: date-time updateTime: description: Last update of the job type: string format: date-time actionStatus: description: Status of the job type: string enum: - ActiveActionStatus - CompletedActionStatus - FailedActionStatus - PotentialActionStatus result: description: Result of the job type: object error: description: Error of the job, if any type: object required: - jobType - jobParameters - progress - whisperer - actionStatus - creator - creationTime required: true responses: '201': description: Job created headers: Location: schema: type: string description: '@id of the created job' '400': $ref: '#/components/responses/TooBig' '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '406': $ref: '#/components/responses/WrongAccept' '411': $ref: '#/components/responses/NoParameter' '415': $ref: '#/components/responses/WrongBodyContentType' '422': $ref: '#/components/responses/InvalidInputs' /job/v1/jobs/_search: post: tags: - Jobs security: - Bearer: [] summary: Search for Jobs description: |- ### Description Search for jobs. _Also available by GET method on the collection_ ### Rules Admin and monitoring admin may search without specifying a whisperer ### Access - Admin - Customer, limited on its whisperers requestBody: $ref: '#/components/requestBodies/DataSearches' responses: '200': $ref: '#/components/responses/Collection' '400': $ref: '#/components/responses/TooBig' '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '406': $ref: '#/components/responses/WrongAccept' '411': $ref: '#/components/responses/NoParameter' '415': $ref: '#/components/responses/WrongBodyContentType' '422': $ref: '#/components/responses/InvalidInputs' /job/v1/jobs/{id}: parameters: - in: path name: id schema: type: string description: Customer internal @id required: true get: tags: - Jobs security: - Bearer: [] summary: Get a jobs details description: |- ### Description Get a jobs details. ### Access - User linked to the whisperers in this job - Admin responses: '200': description: The requested job. content: application/json: schema: $ref: '#/components/schemas/Job' headers: eTag: schema: type: string description: eTag of the resource state '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '404': $ref: '#/components/responses/NotFound' '406': $ref: '#/components/responses/WrongAccept' patch: tags: - Jobs security: - Bearer: [] summary: Update job details description: |- ### Description Update a Job ### Rules Impossible to update the following fields: - jobType - creationTime - creator ### Access - Admin - User that created the job and linked to all whisperers of the job parameters: - in: header name: If-Match description: eTag of previous state of the job schema: type: string required: true requestBody: description: Json patch with the changes required: true content: application/json-patch+json: schema: $ref: '#/components/schemas/JsonPatch' responses: '204': description: Job has been updated headers: eTag: schema: type: string description: eTag of the resource state '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '404': $ref: '#/components/responses/NotFound' '406': $ref: '#/components/responses/WrongAccept' '411': $ref: '#/components/responses/NoParameter' '412': $ref: '#/components/responses/ConcurrencyError' '415': $ref: '#/components/responses/WrongBodyContentType' '422': $ref: '#/components/responses/InvalidInputs' '428': $ref: '#/components/responses/MissingETag' /gui-logs/v1/logs/: post: tags: - GuiLogs security: - Bearer: [] summary: Create a new Gui Log description: |- ### Description Create a Gui Log to trace an error happened on GUI ### Access - No identification required, but if identified, it will be saved requestBody: description: The Gui Log content: application/json: schema: type: object properties: app: description: Name of application type: string enum: - Login - NetworkView - SelfMonitoring time: type: string format: date-time description: Time of log type: description: Type of error type: string enum: - BAD REQUEST - SERVER ERROR - UNEXPECTED - TIMEOUT - UNKNOWN - XXX RENDER name: description: Name of the error type: string enum: - Saga Error - React Error level: description: Level of Log type: string enum: - ERROR - WARNING message: description: Message of the error (from JS error.message) type: string stack: description: Stack of the error type: string details: description: Details of the error. Component stack in React. type: string whisperer: description: List of whisperers type: array items: description: System id of whisperers type: string customer: description: System id of the customer type: string timeout: description: Timeout value in case of timeouts type: integer req: type: object description: Request that failed properties: method: description: HTTP method type: string uri: description: URI of request type: string body: description: Body in case of POST, PATCH, PUT type: string headers: type: object description: Headers of the request required: - method - uri res: type: object description: Response that failed properties: status: description: Status of response type: string statusText: description: Status text of response type: string body: description: Body of response type: string headers: type: object description: Headers of the response required: - status - statusText required: - app - time - name - type - level - message - stack required: true responses: '201': description: Job created headers: Location: schema: type: string description: '@id of the created job' '400': $ref: '#/components/responses/TooBig' '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '406': $ref: '#/components/responses/WrongAccept' '411': $ref: '#/components/responses/NoParameter' '415': $ref: '#/components/responses/WrongBodyContentType' '422': $ref: '#/components/responses/InvalidInputs' /gui-logs/v1/logs/_search: post: tags: - GuiLogs security: - Bearer: [] summary: Search for Gui Logs description: |- ### Description Search for Gui Logs. _Also available by GET method on the collection_ ### Rules Admin and monitoring admin may search without specifying a whisperer ### Access - Admin - User with admin monitoring right asking for an aggregation with size:0 requestBody: $ref: '#/components/requestBodies/DataSearches' responses: '200': $ref: '#/components/responses/Collection' '400': $ref: '#/components/responses/TooBig' '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '406': $ref: '#/components/responses/WrongAccept' '411': $ref: '#/components/responses/NoParameter' '415': $ref: '#/components/responses/WrongBodyContentType' '422': $ref: '#/components/responses/InvalidInputs' /gui-settings/v1/settings/: post: tags: - GuiSettings security: - Bearer: [] summary: Create / update GUI settings description: |- ### Description Stores the user settings. Mostly used by the UI itself ### Access - Customer - Admin requestBody: description: The settings to save content: application/json: schema: type: object description: Settings properties: '@id': type: string description: Id of the settings, and of the customer settings: type: object description: GUI settings properties: global: type: object description: >- Global settings, may be overriden by team ones, like saved settings... user: type: object description: >- User specific settings, mostly layout and behavior options required: - settings required: true responses: '201': description: Settings created headers: Location: schema: type: string description: '@id of the created settings' '400': $ref: '#/components/responses/TooBig' '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '406': $ref: '#/components/responses/WrongAccept' '411': $ref: '#/components/responses/NoParameter' '415': $ref: '#/components/responses/WrongBodyContentType' '422': $ref: '#/components/responses/InvalidInputs' /gui-settings/v1/settings/_search: post: tags: - GuiSettings security: - Bearer: [] summary: Search for Settings description: |- ### Description Search for settings. _Also available by GET method on the collection_ ### Access - Admin requestBody: $ref: '#/components/requestBodies/ConfSearches' responses: '200': $ref: '#/components/responses/Collection' '400': $ref: '#/components/responses/TooBig' '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '406': $ref: '#/components/responses/WrongAccept' '411': $ref: '#/components/responses/NoParameter' '415': $ref: '#/components/responses/WrongBodyContentType' '422': $ref: '#/components/responses/InvalidInputs' /gui-settings/v1/settings/{id}: parameters: - in: path name: id schema: type: string description: Settings Id == Id of user required: true get: tags: - GuiSettings security: - Bearer: [] summary: Get GUI settings for a user description: |- ### Description Get settings ### Access - Customer - Admin responses: '200': description: The requested settings. content: application/json: schema: $ref: '#/components/schemas/GuiSettings' '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '404': $ref: '#/components/responses/NotFound' '406': $ref: '#/components/responses/WrongAccept' /plugins/v1/plugins/: post: tags: - Plugins security: - Bearer: [] summary: Create / update a plugin description: |- ### Description Uploads a plugin. Depending on `global` setting, the plugin is stored locally in ES or remotely in Floocus managed S3 service. ### Access - User with `plugins.upload` permission - Admin requestBody: description: The plugin to store content: application/json: schema: type: object description: Plugin properties: '@id': type: string description: Id of the plugin, unique for the organisation '@type': type: string description: Type of the plugin '@version': type: string description: Version of the plugin name: type: string description: Human name of the plugin description: type: string description: Explains what the plugins does source: type: string description: Base64 encoded javascript source code of the plugin required: - '@id' - '@type' - '@version' - name - source required: true responses: '201': description: Plugin created headers: Location: schema: type: string description: '@id of the created plugin' '400': $ref: '#/components/responses/TooBig' '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '406': $ref: '#/components/responses/WrongAccept' '409': description: Conflict. A plugin with same @id and same @version exist. content: application/json: schema: $ref: '#/components/schemas/Error' '411': $ref: '#/components/responses/NoParameter' '415': $ref: '#/components/responses/WrongBodyContentType' '422': $ref: '#/components/responses/InvalidInputs' /plugins/v1/plugins/_search: post: tags: - Plugins security: - Bearer: [] summary: Search for Plugins description: >- ### Description When `global`, searches for both own organisation plugins and 'Floocus' ones. When local, searches for any plugin. _Also available by GET method on the collection_ ### Access - Customer requestBody: description: The plugin to store content: application/json: schema: type: object description: Plugin search query properties: type: type: string description: Type of the plugin sort: description: Sorting option. type: array items: type: object properties: key: description: Property on which to sort. type: string order: description: Sort order. type: string enum: - asc - desc default: asc next: description: Ids of the last plugin fetched. type: array items: type: string size: description: Page size. type: integer required: - size required: true responses: '200': $ref: '#/components/responses/Collection' '400': $ref: '#/components/responses/TooBig' '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '406': $ref: '#/components/responses/WrongAccept' '411': $ref: '#/components/responses/NoParameter' '415': $ref: '#/components/responses/WrongBodyContentType' '422': $ref: '#/components/responses/InvalidInputs' /plugins/v1/plugins/{id}: parameters: - in: path name: id schema: type: string description: Plugin @id required: true get: tags: - Plugins security: - Bearer: [] summary: Get the manifest of a specific plugin description: |- ### Description Get plugin manifest ### Access - Customer - Admin responses: '200': description: The requested settings. content: application/json: schema: $ref: '#/components/schemas/GuiSettings' '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '404': $ref: '#/components/responses/NotFound' '406': $ref: '#/components/responses/WrongAccept' /plugins/v1/plugins/{id}.js: parameters: - in: path name: id schema: type: string description: Plugin @id required: true get: tags: - Plugins summary: Get the code of a specific plugin description: |- ### Description Get the code ### Access - Open to everybody responses: '200': description: The requested plugin code. content: application/json: schema: $ref: '#/components/schemas/GuiSettings' '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '404': $ref: '#/components/responses/NotFound' /{service}/v1/queueStatus/: get: tags: - Pollers security: - Bearer: [] summary: Get poller queue status description: |- ### Description Give the queue status of this poller ### Access - Admin - Application - Customer parameters: - in: path name: service schema: $ref: '#/components/schemas/PollerNames' required: true responses: '200': content: application/json: schema: $ref: '#/components/schemas/PollerStats' description: Poller stats /monitor-read/v1/{collection}/_search: post: tags: - Monitor security: - Bearer: [] summary: Search for monitoring information description: |- ### Description Searches or aggregation analysis on monitored metrics: - Processes stats - Circuit breakers stats - Pollers stats - Parsing queues stats - Elasticsearch indices stats - Elasticsearch nodes stats - Redis stats - Applicative logs stats _Also available by GET method on the collections_ ### Access - Admin - User with admin monitoring right parameters: - in: path name: collection schema: type: string enum: - pollers - parsers - parsingqueues - circuitbreakers - redis - elasticsearch - elasticsearchNodes - processes - api - logs description: Collection of metrics required: true requestBody: $ref: '#/components/requestBodies/DataSearches' responses: '200': $ref: '#/components/responses/Collection' '400': $ref: '#/components/responses/TooBig' '401': $ref: '#/components/responses/Unauthenticated' '403': $ref: '#/components/responses/Unauthorized' '406': $ref: '#/components/responses/WrongAccept' '411': $ref: '#/components/responses/NoParameter' '415': $ref: '#/components/responses/WrongBodyContentType' '422': $ref: '#/components/responses/InvalidInputs' components: requestBodies: ConfSearches: description: Search parameters required: true content: application/json: schema: $ref: '#/components/schemas/SearchParametersForConf' DataSearches: description: Search parameters required: true content: application/json: schema: $ref: '#/components/schemas/SearchParameters' examples: searchByTime: summary: Search by timestamp value: size: 20 whisperers: - KxZOoCLxSM6cVee7xq6iPw - Km77Vs-SS4yRDXwivW8nlA startTime: 1547789298.676 stopTime: 1547805717.362 query: >- !req.uri:contexts AND !req.uri:version AND !req.query:afterUpdate* sort: - key: req.start order: asc searchByDate: summary: Search by timestamp value: size: 20 whisperers: - KxZOoCLxSM6cVee7xq6iPw - Km77Vs-SS4yRDXwivW8nlA startDate: '2019-01-18T05:28:18.676123' stopDate: '2019-01-18T10:01:57.362456' query: >- !req.uri:contexts AND !req.uri:version AND !req.query:afterUpdate* sort: - key: req.start order: asc aggregation: summary: Aggregation value: size: 0 whisperers: - KxZOoCLxSM6cVee7xq6iPw - Km77Vs-SS4yRDXwivW8nlA startTime: 1547636400 stopTime: 1548068400 query: >- !req.uri:contexts AND !req.uri:version AND !req.query:afterUpdate* aggs: items: histogram: field: req.start interval: 1440 aggs: sumIn: sum: field: res.size sumOut: sum: field: req.size CurrentSearches: description: Search parameters required: true content: application/json: schema: $ref: '#/components/schemas/SearchParametersNoTime' examples: search: summary: Classic search value: size: 20 whisperers: - KxZOoCLxSM6cVee7xq6iPw - Km77Vs-SS4yRDXwivW8nlA query: '' sort: - key: time order: asc responses: Collection: description: Collection of items content: application/json: schema: $ref: '#/components/schemas/Collection' ParsingJob: description: Purge progress content: application/json: schema: $ref: '#/components/schemas/ParsingJob' PurgeProgress: description: Purge progress content: application/json: schema: $ref: '#/components/schemas/PurgeProgress' NoContent: description: Success, no content sent TooBig: description: Request body too big content: application/json: schema: $ref: '#/components/schemas/Error' NotFound: description: Not found content: application/json: schema: $ref: '#/components/schemas/Error' Unauthenticated: description: Not authenticated content: application/json: schema: $ref: '#/components/schemas/Error' Unauthorized: description: Not authorized content: application/json: schema: $ref: '#/components/schemas/Error' ConcurrencyError: description: Resource has been modified by someone else content: application/json: schema: $ref: '#/components/schemas/Error' MissingETag: description: Action must be done with previous ETag content: application/json: schema: $ref: '#/components/schemas/Error' WrongAccept: description: Cannot produce requested content type in output content: application/json: schema: $ref: '#/components/schemas/Error' NoParameter: description: No parameter content: application/json: schema: $ref: '#/components/schemas/Error' WrongBodyContentType: description: Wrong body content type content: application/json: schema: $ref: '#/components/schemas/Error' InvalidInputs: description: Invalid structure of input parameters content: application/json: schema: $ref: '#/components/schemas/Error' schemas: ServiceNames: type: string description: Micro service name enum: - alert - capture-status-poller - ciphers - ciphers-status - ciphers-status-poller - ciphers-raw-status-poller - ciphers-status-agg - controls - customer - gui-logs - gui-settings - hosts - hosts-agg - hosts-poller - job - link - mail-sender - maintenance - pack-poller - pack-read - pack-write - pack-update - parsing-status-tcpsession-poller - parsing-status-httppers-poller - parsing-status-pgparsing-poller - pg-com-poller - pg-com-content-poller - pg-parser - pg-parsing-poller - pg-read - pg-upload - plugins - session - stats-collector - status-poller - tcp-poller - tcp-read - tcp-write - tcp-update - teams - tls-keys - tls-keys-linker - web-httpcom-poller - web-httpcom-content-poller - web-httppers-poller - web-read - web-upload - web-write - whisp - whisp-status-poller - whisps-status - whisps-status-agg ParserNames: type: string description: Micro service name enum: - web-write - tls-keys-linker PollerNames: type: string description: Pollers service name enum: - pack-poller - tcp-poller - pg-com-poller - pg-com-content-poller - pg-parsing-poller - web-httpcom-poller - web-httpcom-content-poller - web-httppers-poller - hosts-poller - hosts-agg - whisp-status-poller - whisps-status-agg - status-poller - capture-status-poller - ciphers-status-poller - ciphers-raw-status-poller - ciphers-status-agg - parsing-status-tcpsession-poller - parsing-status-httppers-poller - parsing-status-pgparsing-poller JsonPatch: type: array items: type: object properties: op: type: string enum: - test - remove - add - replace - move - copy path: type: string value: type: string from: type: string required: - op - path SearchParameters: type: object properties: whisperers: description: List of whisperers to search on. type: array items: type: string description: System id of whisperer startTime: description: >- Start unix timestamp of search window. Up to 6 decimals for microseconds. type: number format: double stopTime: description: >- Stop unix timestamp of search window. Up to 6 decimals for microseconds. type: number format: double startDate: description: Start date of search window (can replace startTime). type: string format: date-time stopDate: description: Stop date of search window (can replace stopTime). type: string format: date-time query: description: Free query, using Elasticsearch query string DSL. type: string aggs: description: Aggregation request, using Elasticsearch aggregation DSL. type: object sort: description: Sorting option. Sorting by @id is automaticaly added. type: array items: type: object properties: key: description: Property on which to sort. type: string order: description: Sort order. type: string enum: - asc - desc default: asc next: description: Ids of the last resource already fetched to get next ones. type: array items: type: string size: description: Page size. type: integer withContent: description: >- True if you want to embed content in the result items (only for HTTP coms) type: boolean avoidTotalHits: description: Tells if response should include total hits count. type: boolean async: description: Ask for an async search (when supported). type: boolean asyncDelayMs: description: >- How long (in milliseconds) server should wait for an answer before answering with a partial answer (when async). type: number format: integer asyncId: description: >- Id of previous async answer from server (to get follow up). Included in hypermedia answer from server when async answer. type: string asyncKeepAliveS: description: >- How long (in seconds) the async answer is allowed to search before being killed. type: number format: integer example: size: 20 whisperers: - KxZOoCLxSM6cVee7xq6iPw - Km77Vs-SS4yRDXwivW8nlA startDate: '2019-01-18T05:28:18.676123' stopDate: '2019-01-18T10:01:57.362456' query: '' sort: - key: req.start order: asc avoidTotalHits: true SearchParametersNoTime: type: object properties: whisperers: description: List of whisperers to search on. type: array items: type: string description: System id of whisperer query: description: Free query, using Elasticsearch query string DSL. type: string aggs: description: Aggregation request, using Elasticsearch aggregation DSL. type: object sort: description: Sorting option. Sorting by @id is automaticaly added. type: array items: type: object properties: key: description: Property on which to sort. type: string order: description: Sort order. type: string enum: - asc - desc default: asc next: description: Ids of the last resources fetched to get next ones. type: array items: type: string size: description: Page size. type: integer avoidTotalHits: description: Tells if response should include total hits count. type: boolean required: - size SearchParametersForConf: type: object properties: query: description: Free query, using Elasticsearch query string DSL. type: string aggs: description: Aggregation request, using Elasticsearch aggregation DSL. type: object sort: description: Sorting option. Sorting by @id is automaticaly added. type: array items: type: object properties: key: description: Property on which to sort. type: string order: description: Sort order. type: string enum: - asc - desc default: asc next: description: Aggregation request, using Elasticsearch aggregation DSL. type: array items: type: string size: description: Page size. type: integer avoidTotalHits: description: Tells if response should include total hits count. type: boolean includeETags: description: Tells if response should include _eTags fields for update. type: boolean required: - size Collection: type: object properties: total: description: Total number of records corresponding to search. type: integer items: description: List of items type: array items: oneOf: - $ref: '#/components/schemas/Packet' - $ref: '#/components/schemas/TcpSession' - $ref: '#/components/schemas/HttpCommunication' - $ref: '#/components/schemas/Customer' - $ref: '#/components/schemas/HttpCommunication' - $ref: '#/components/schemas/Whisperer' - $ref: '#/components/schemas/HostsList' - $ref: '#/components/schemas/RawStatus' - $ref: '#/components/schemas/CurrentStatus' - $ref: '#/components/schemas/Link' - $ref: '#/components/schemas/Session' - $ref: '#/components/schemas/Job' aggs: description: Result of ElasticSearch aggregation (as is) type: object nextPage: description: Hypermedia link for next page type: object properties: '@type': type: string enum: - SearchAction query: type: object description: JSON to use in the body of the request for next page asyncResults: description: Hypermedia link for async follow-up type: object properties: '@type': type: string enum: - SearchAction query: type: object description: JSON to use in the body of the request for async answer followup ParsingJob: type: object properties: total: description: Total number of items corresponding to parse. type: integer items: description: List of items type: array items: type: object Packet: type: object externalDocs: description: Reference url: https://en.wikipedia.org/wiki/Network_packet properties: '@id': type: string description: Unique id of the packet in the system. '@type': type: string enum: - Packet version: type: string description: Version of the schema. 0.1 is deprecated. 2.0 is latest. name: type: string description: Name of the packet (for display). whisperer: type: string description: Whisperer that captured the packet tcpSession: type: string description: Tcp session Id the packet is in instanceId: type: string description: Instance id of the whisperer timestamp: type: number format: double description: Unix timestamp of capture, with microseconds date: description: Date of packet type: string format: date minute: description: Date of packet, truncated to the minute type: string format: date length: type: integer description: Size of the packet (size of rawPacket.buf buffer) protocols: type: object description: List of protocols and details used by this packet additionalProperties: type: object properties: src: type: string description: Source identifier in this protocol (ip address, tcp port, ...) dst: type: string description: Destination identifier direction: type: string description: Direction of the packet (only for TCP protocol) enum: - in - out index: type: integer description: Index of the packet in the Tcp session (only for TCP protocol) packetLot: type: integer description: >- Index of the packetLot - consecutive packets with data in same direction - in the Tcp session (only for TCP protocol) relativeSeq: type: integer description: >- Relative seq of the packet in this Tcp session (only for TCP protocol) relativeAck: type: integer description: >- Relative ack of the packet in this Tcp session (only for TCP protocol) hasData: type: boolean description: If the packet has Tcp data (only for TCP protocol) protocolsList: description: List of protocols (for searching) type: array items: type: string rawPacket: type: object properties: '@type': type: string buf: description: Buffer with packet content type: array items: type: integer header: description: Buffer with packet pcap header type: array items: type: integer link_type: description: Type of network physical link type: string enum: - LINKTYPE_ETHERNET - LINKTYPE_NULL - LINKTYPE_RAW - LINKTYPE_IEEE802_11_RADIO - LINKTYPE_LINUX_SLL example: '@id': >- ROlrqlFhTY2ayXIxTV2uZA.rd-srv508-bes.456285.172.16.102.72-43118-172.16.102.125-8080.1682747931-5 '@type': Packet version: '2.0' commonId: >- ROlrqlFhTY2ayXIxTV2uZA.456285.172.16.102.72-43118-172.16.102.125-8080.1682747931-5 name: 456285.172.16.102.72-43118-172.16.102.125-8080#5 whisperer: ROlrqlFhTY2ayXIxTV2uZA instanceId: rd-srv508-bes tcpSession: >- ROlrqlFhTY2ayXIxTV2uZA.rd-srv508-bes.456285.172.16.102.72-43118-172.16.102.125-8080.1682747931 timestamp: 1642625100.188968 length: 229 rawPacket: link_type: LINKTYPE_LINUX_SLL buf: [] header: [] protocols: Linux SLL: src: 50:6b:8d:3a:97:undefined dst: null IPv4: src: 172.16.102.125 dst: 172.16.102.72 TCP: src: '8080' dst: '43118' direction: in packetLot: 1 index: 5 relativeSeq: 1 relativeAck: 0 hasData: true date: '2022-01-19T20:45:00.188Z' minute: '2022-01-19T20:45:00.000Z' protocolsList: - Linux SLL - IPv4 - TCP PurgeProgress: type: object properties: completed: description: Is purge job completed. type: boolean total: description: Count of total items processed (deleted + failures). type: integer deleted: description: Count of total items deleted. type: integer failures: description: Items that failed to be deleted (got modified after purge start). type: array items: type: object durationMs: description: Duration of purge process (in Ms). type: integer TcpSession: externalDocs: description: Reference url: https://en.wikipedia.org/wiki/Transmission_Control_Protocol type: object properties: '@id': type: string description: System id of TCP session '@type': type: string enum: - Tcp session version: type: string description: Version of the schema. name: type: string description: Name of the tcTCP session (for display). whisperer: type: string description: Whisperer that captured the TCP session instanceId: type: string description: Instance id of the whisperer src: description: Client host type: object properties: ip: description: IP address type: string format: ipv4 port: description: TCP port type: integer socket: description: 'TCP socket: ''IP:Port''' type: string name: description: Hostname type: string dst: description: Server host type: object properties: ip: description: IP address type: string format: ipv4 port: description: TCP port type: integer socket: description: 'TCP socket: ''IP:Port''' type: string name: description: Hostname type: string state: description: State of TCP session lifecycle type: string enum: - SYN_SENT - SYN_RECEIVED - ESTABLISHED - CLOSE_WAIT - LAST_ACK - CLOSED packetsCount: description: Count of packets in the sessions type: integer syn: description: Timestamp of SYN packet type: number format: double missedSyn: description: If whisperer missed SYN type: boolean connect: description: Timestamp when connection was established type: number format: double first: description: Timestamp of first packet (different from SYN when missedSyn) type: number format: double firstDate: description: Date of first packet type: string format: date last: description: Timestamp of last packet type: number format: double lastDate: description: Date of last packet type: string format: date duration: description: Duration of session (first - last), in seconds type: number format: float timespan: description: Time range of TCP sessions type: object properties: gte: description: Start date type: string format: date lte: description: End date type: integer format: date latency: description: Latency of connection (SYN packet -> ACK SYN packet) type: number format: float out: description: Out packets (responses from server) type: object properties: ip: description: IP headers total overhead type: integer tcp: description: TCP headers total overhead type: integer payload: description: TCP payload total size type: integer initialSeq: description: Initial seq of this side of the communication type: integer in: description: In packets (responses from server) type: object properties: ip: description: IP headers total overhead type: integer tcp: description: TCP headers total overhead type: integer payload: description: TCP payload total size type: integer initialSeq: description: Initial seq of this side of the communication type: integer parsers: type: object properties: http: description: If session was parsed by HTTP parser type: object properties: status: description: Result of parsing type: string enum: - PARSING_COMPLETED - PARSING_NO_DATA - PARSING_PARSED - PARSING_PENDING - PARSING_WAITING - PARSING_WARNING - PARSING_ERROR lastParsing: description: Date of last parsing type: string format: date httpPers: description: Link to Http parsing log type: string itemsCount: description: Count of HTTP coms found inside type: integer lastPacketLotComplete: description: Last packetlot that has been completely parsed type: integer lastPacketParsedIndex: description: Last packet index from last parsed packetLot type: integer dateModified: description: Date of last modification (update of parsing) type: string format: date HttpCommunication: externalDocs: description: Reference url: https://en.wikipedia.org/wiki/Hypertext_Transfer_Protocol type: object properties: '@id': type: string '@type': type: string version: type: string name: type: string description: Name of the resource (for display) tcpSession: type: string description: TCP session containing the HTTP communication httpPers: type: string description: Http parsing log containing the HTTP communication whisperer: type: string description: Whisperer that captured instanceId: type: string description: Instance id of the whisperer uploaded: type: boolean description: True if Http communication has been imported as is stats: type: object properties: statusCode: type: string description: Response code (200, 404...) statusText: type: integer description: Response status text (OK, NOT FOUND...) duration: type: number format: float description: Duration of HTTP communication timespan: description: Time range of HTTP communication type: object properties: gte: description: Start date type: string format: date lte: description: End date type: integer format: date src: description: Client host type: object properties: ip: description: IP address type: string format: ipv4 port: description: TCP port type: integer socket: description: 'TCP socket: ''IP:Port''' type: string name: description: Hostname type: string origin: description: Origin IP address (from x-forwarded-for) type: string format: ipv4 identification: description: >- Identification of the client. The login from a Basic Auth, the 'sub' from a JWT... type: string dst: description: Server host type: object properties: ip: description: IP address type: string format: ipv4 port: description: TCP port type: integer socket: description: 'TCP socket: ''IP:Port''' type: string name: description: Hostname type: string prox: description: List of proxies in between (not implemented yet) type: array items: type: string req: description: Request part type: object properties: status: description: Parsing status in Spider type: string enum: - COMPLETE - INCOMPLETE - BODY_OVERSIZED - UNKNOWN method: description: HTTP method used (GET, POST...) type: string httpVersion: description: HTTP protocol version type: string uri: description: URI of request (without Host, until ?) type: string query: description: Query string part (after the ?) type: string hash: description: 'Hash part (after the #)' type: string template: description: Request template matched on parsing type: string start: description: Timestamp of request first packet type: number format: double startDate: description: Start date type: string format: date end: description: Timestamp of request last packet type: number format: double size: description: Size of request, including headers type: number format: float packets: description: List of packets that compose the request type: array items: type: string headers: description: All request headers type: object rawHeaders: description: HTTP headers, as found on the wire type: object properties: size: description: Effective size type: integer filtered: description: True if headers were filtered out when parsing type: boolean content: description: Headers type: string body: description: Request body metadata (if a body is present) type: object properties: contentType: description: Content type header type: string embedded: description: True if content is embedded in resource type: boolean filtered: description: True if content was filtered out when parsing type: boolean length: description: Declared size type: integer size: description: Effective size type: integer content: description: Base 64 encoding of the body type: string format: base64 tags: description: Collection of tags extracted on parsing type: object res: description: Response part type: object properties: status: description: Parsing status in Spider type: string enum: - COMPLETE - INCOMPLETE - BODY_OVERSIZED - UNKNOWN httpVersion: description: HTTP protocol version type: string start: description: Timestamp of first response packet type: number format: double end: description: Timestamp of response last packet type: number format: double endDate: description: End date type: string format: date size: description: Size of response, including headers type: number format: float packets: description: List of packets that compose the response type: array items: type: string headers: description: All response headers type: object rawHeaders: description: HTTP headers, as found on the wire type: object properties: size: description: Effective size type: integer filtered: description: True if headers were filtered out when parsing type: boolean content: description: Headers type: string body: description: Response body metadata (if a body is present) type: object properties: contentType: description: Content type header type: string embedded: description: True if content is embedded in resource filtered: description: True if content was filtered out when parsing type: boolean length: description: Declared size type: integer size: description: Effective size type: integer content: description: Base 64 encoding of the body type: string format: base64 tags: description: Collection of tags extracted on parsing type: object PgCommunication: type: object properties: '@id': type: string '@type': type: string version: type: string name: type: string description: Name of the resource (for display) tcpSession: type: string description: TCP session containing the PostgreSQL communication pgParsing: type: string description: PostgreSQL parsing log containing the PostgreSQL communication whisperer: type: string description: Whisperer that captured instanceId: type: string description: Instance id of the whisperer uploaded: type: boolean description: True if PostgreSQL communication has been imported as is stats: type: object properties: statusCode: type: string description: Response code (200, 500...) statusText: type: integer description: Response status text (OK, NOT FOUND...) duration: type: number format: float description: Duration of PostgreSQL communication timespan: description: Time range of PostgreSQL communication type: object properties: gte: description: Start date type: string format: date lte: description: End date type: integer format: date src: description: Client host type: object properties: ip: description: IP address type: string format: ipv4 port: description: TCP port type: integer socket: description: 'TCP socket: ''IP:Port''' type: string name: description: Hostname type: string dst: description: Server host type: object properties: ip: description: IP address type: string format: ipv4 port: description: TCP port type: integer socket: description: 'TCP socket: ''IP:Port''' type: string name: description: Hostname type: string req: description: Request part type: object properties: status: description: Parsing status in Spider type: string enum: - COMPLETE - INCOMPLETE command: description: Main SQL command (INSERT, UPDATE, DELETE, SELECT...) type: string template: description: Request template matched on parsing type: string tables: description: Tables used in the request type: array items: type: string start: description: Timestamp of request first packet type: number format: double startDate: description: Start date type: string format: date end: description: Timestamp of request last packet type: number format: double size: description: Size of request, including headers type: number format: float packets: description: List of packets that compose the request type: array items: type: string tags: description: Collection of tags extracted on parsing type: object subCommands: description: List of subcommands issues in the request type: array items: type: object properties: messageType: description: Type of message (query, startup, prepare, exec, sync...) type: string command: type: string content: description: Content of the message, vary by messageType type: object properties: query: description: SQL query type: string _analyzedQuery: description: SQL query as analyzed by Spider with extracted fields type: object commandsCount: description: Number of sub-commands in the request type: integer res: description: Response part type: object properties: status: description: Parsing status in Spider type: string enum: - COMPLETE - INCOMPLETE start: description: Timestamp of first response packet type: number format: double end: description: Timestamp of response last packet type: number format: double endDate: description: End date type: string format: date size: description: Size of response, including headers type: number format: float packets: description: List of packets that compose the response type: array items: type: string tags: description: Collection of tags extracted on parsing type: object rowCount: description: Number of rows returned by the request type: integer resultsCount: description: Number of sub-results returned by the request type: integer subResults: description: List of sub-results returned by the request type: array items: type: object properties: messageType: description: Type of message (dataRow, error, notice...) type: string content: description: Content of the message, vary by messageType type: object properties: fields: description: >- List of fields returned by the request in a RowDescription message type: array items: type: object properties: name: type: string tableOID: type: integer colAttr: type: integer dataTypeOID: type: integer dataTypeSize: type: integer typeModifier: type: integer format: type: integer values: description: >- List of values returned by the request in a DataRow message type: array items: type: string nullable: true rowCount: description: Number of rows returned by the request type: integer statusCode: description: Response code (200, 500...) type: integer connectionMetadata: description: Metadata extracted from the startup phase type: object properties: user: description: User name type: string database: description: Database name type: string protocolVersion: description: PostgreSQL protocol version type: string server_version: description: PostgreSQL version type: string session_authorization: description: User used (in case of sudo) type: string is_superuser: description: Tells if the user is a superuser type: string client_encoding: description: Client encoding (UTF8...) type: string DateStyle: description: PostgreSQL date style type: string TimeZone: description: PostgreSQL timezone type: string PgParsing: description: Parsing log of PostgreSQL parsing. type: object properties: '@id': type: string '@type': type: string version: type: string name: type: string tcpSession: type: string description: TCP session containing the HTTP communication whisperer: type: string description: Whisperer that captured the TCP session instanceId: type: string description: Instance id of the whisperer first: type: number format: double description: Timestamp of first packet of associated TCP session last: type: number format: double description: Timestamp of last packet of associated TCP session lastPacketLotComplete: description: Last packetlot that has been completely parsed type: integer lastPacketParsedIndex: description: Last packet index from last parsed packetLot type: integer parsingCount: description: Number of parsing done already type: integer itemsFound: description: How many items (HTTP coms) were found type: integer packetLots: type: array description: Group of consecutive packets with data in the same direction items: type: object properties: '@id': type: string '@type': type: string index: type: integer description: Index of this packetLot in the Tcp session dir: type: string description: Direction of packetLot enum: - in - out fetched: type: array items: type: object properties: date: type: string description: Date this packetLot was fetched packets: type: array description: List of packets composing the packetLot items: type: object description: Packet fetched status: type: string description: Fetching status of packetLot enum: - PENDING - DISCARDED - FETCHED - FETCHING - FETCHING_WARNING - FETCHING_ERROR - PARSED - COMPLETE error: type: object description: Error description if got an error fetching the packets properties: code: type: string description: HTTP status code returned by pack message: type: string description: HTTP status code returned by pack HttpPers: description: Parsing log of Http parsing. V2.0, v0.1 is deprecated. type: object properties: '@id': type: string '@type': type: string version: type: string name: type: string tcpSession: type: string description: TCP session containing the HTTP communication whisperer: type: string description: Whisperer that captured the TCP session instanceId: type: string description: Instance id of the whisperer first: type: number format: double description: Timestamp of first packet of associated TCP session last: type: number format: double description: Timestamp of last packet of associated TCP session lastPacketLotComplete: description: Last packetlot that has been completely parsed type: integer lastPacketParsedIndex: description: Last packet index from last parsed packetLot type: integer parsingCount: description: Number of parsing done already type: integer itemsFound: description: How many items (HTTP coms) were found type: integer packetLots: type: array description: Group of consecutive packets with data in the same direction items: type: object properties: '@id': type: string '@type': type: string index: type: integer description: Index of this packetLot in the Tcp session dir: type: string description: Direction of packetLot enum: - in - out fetched: type: array items: type: object properties: date: type: string description: Date this packetLot was fetched packets: type: array description: List of packets composing the packetLot items: type: object description: Packet fetched status: type: string description: Fetching status of packetLot enum: - PENDING - DISCARDED - FETCHED - FETCHING - FETCHING_WARNING - FETCHING_ERROR - PARSED - COMPLETE error: type: object description: Error description if got an error fetching the packets properties: code: type: string description: HTTP status code returned by pack message: type: string description: HTTP status code returned by pack Controller: type: object description: Controller properties: '@id': description: System id type: string '@type': type: string enum: - Controller version: description: Version of schema type: string name: description: Name of Controller type: string customer: type: string description: Customer whose Controller it is apikey: type: string format: base64 description: Public PEM of the Controller API Key config: type: object properties: '@id': description: System id type: string '@type': type: string enum: - ControllerConfig version: description: Version of schema type: string creator: type: string description: Creator of the Controller Config dateCreated: type: string description: Creation date format: date editor: type: string description: Last editor of the Controller Config dateModified: type: string description: Modification date format: date attachments: type: array items: type: object properties: '@id': description: System id type: string '@type': type: string enum: - Attachment controller: type: string description: Id of the controller used whisperer: type: string description: Id of the whisperer used item: type: string description: Name of the workload namespace: type: string description: Namespace of the workload collection: type: string description: Collection of the workload status: type: string description: Status of the attachment enum: - ATTACHMENT_REQUESTED - ATTACHED - DETACHMENT_REQUESTED - DETACHED - EXPIRED expires: type: string description: >- Expiry date of the attachment if a TTL is applied on the Whisperer format: date creator: type: string description: Creator of the Attachment dateCreated: type: string description: Creation date format: date editor: type: string description: Last editor of the Attachment dateModified: type: string description: Modification date format: date users: description: Customers to whom the Controller is shared type: array items: type: object properties: '@id': description: System id of the customer type: string email: description: Email of the customer (on last update of rights) type: string rights: description: Rights of the customer on this Controller type: object properties: share: description: Can add/remove shared users type: boolean config: description: Can change configuration and name type: boolean install: description: Can install the Controller type: boolean delete: description: Can delete the Controller type: boolean namespaces: description: List of namespaces the customer can access type: object properties: '*': description: All namespaces type: boolean namespace-name: description: Any namespace type: boolean teams: description: Teams to whom the Controller is shared type: array items: type: object properties: '@id': description: System id of the team type: string name: description: Name of the team (on last update of rights) type: string namespaces: description: List of namespaces the team can access type: object properties: '*': description: All namespaces type: boolean namespace-name: description: Any namespace type: boolean status: type: object description: Status of the Controller properties: linked: type: boolean description: If the Controller is connected connected: type: string description: Date of connection format: date subscriptions: type: number format: integer description: Count of requests in progress over the websocket last: type: object description: Metrics sent by the Controller, subject to changes ;) creator: type: string description: Creator of the Controller dateCreated: type: string description: Creation date format: date editor: type: string description: Last editor of the Controller dateModified: type: string description: Modification date format: date Gocipher: type: object description: Gocipher properties: '@id': description: System id type: string '@type': type: string enum: - Gocipher version: description: Version of schema type: string name: description: Name of Gocipher type: string customer: type: string description: Customer whose Gocipher it is apikey: type: string format: base64 description: Public PEM of the Gocipher API Key config: type: object properties: '@id': description: System id type: string '@type': type: string enum: - ControllerConfig version: description: Version of schema type: string creator: type: string description: Creator of the Controller Config dateCreated: type: string description: Creation date format: date editor: type: string description: Last editor of the Controller Config dateModified: type: string description: Modification date format: date users: description: Customers to whom the Gocipher is shared type: array items: type: object properties: '@id': description: System id of the customer type: string email: description: Email of the customer (on last update of rights) type: string rights: description: Rights of the customer on this Gocipher type: object properties: share: description: Can add/remove shared users type: boolean config: description: Can change configuration and name type: boolean install: description: Can install the Gocipher type: boolean delete: description: Can delete the Gocipher type: boolean teams: description: Teams to whom the Gocipher is shared type: array items: type: object properties: '@id': description: System id of the team type: string name: description: Name of the team (on last update of rights) type: string creator: type: string description: Creator of the Gocipher dateCreated: type: string description: Creation date format: date editor: type: string description: Last editor of the Gocipher dateModified: type: string description: Modification date format: date Customer: type: object description: Based on https://schema.org/person properties: '@id': type: string description: Unique id of the customer in the system. '@type': type: string enum: - Customer _password: type: string description: True if user is an administrator. minLength: 6 _admin: type: boolean description: True if the user is admin. version: type: string description: Version of the schema. dateCreated: type: string format: date description: Date of creation. dateModified: type: string format: date description: Date of last modification. editor: type: string description: User that did last modification. email: type: string description: Customer's email. address: type: object properties: addressCountry: description: The country. For example, USA. type: string addressLocality: description: The locality. For example, Mountain View. type: string addressRegion: description: The region. For example, CA. type: string postalCode: description: The postal code. For example, 94043. type: string streetAddress: description: The street address. For example, 1600 Ampitheatre Pkwy. type: string birthDate: description: Date of birth. type: string format: date honorificPrefix: description: An honorific prefix preceding a name such as Dr/Mrs/Mr. type: string givenName: description: The given name, the first name. type: string familyName: description: The family name, the last name. type: string nationality: description: Nationality. type: string jobTitle: description: The job title (for example, Financial Manager). type: string worksFor: description: Organizations'name that the person works for. type: string whisperers: type: array description: List of Whisperers the users can access to. items: type: string description: System id of the whisperer. rights: type: object description: List of access rights for the user. properties: users: description: Rights associated to customer/user management. type: object properties: create: description: Can create users, and initialize user's details. type: boolean impersonate: description: Can impersonate users and act with their resources. type: boolean delete: description: Can delete users. type: boolean password: description: Can change users passwords (but can't see existing one). type: boolean rights: description: Can change users rights. type: boolean whisperers: description: Rights associated to whisperers management. type: object properties: create: description: Can create whisperers. type: boolean monitor: description: Can access whisperers monitoring dashboard. type: boolean teams: description: Rights associated to teams management. type: object properties: create: description: Can create teams. type: boolean training: description: Rights associated to trainings. type: object properties: userTrainer: description: >- Can create trainees account, impersonate, and delete them. Can create training teams. type: boolean trainee: description: User created for a training, no specific rights (yet). type: boolean admin: description: Rights associated to global administration. type: object properties: monitoring: description: Can access full monitoring dashboard. type: boolean example: '@id': Jz6lxOiuQYaIZNNouiyt6w '@type': Person version: '0.1' technicalStatus: ACTIVE creator: AWUb00luIXCLtCIFlzoO dateCreated: '2018-12-21T10:00:00.837Z' editor: Jz6lxOiuQYaIZNNouiyt6w dateModified: '2018-12-21T16:15:18.978Z' givenName: John familyName: Doe nationality: American email: example@gmail.com address: addressCountry: France rights: whisperers: monitor: true whisperers: - '@id': Gu0ManDYSXGr8Sxi3s-sxg name: WhispTest - '@id': rSSa3P5gQ-2S037OBIp6NA name: Private Whisp _eTag: '"d7-+6EM6pRmKNKDSTtAgeIK5g"' Team: type: object description: A team gather common settings and whisperers for a set of users properties: '@id': type: string description: Unique id of the team in the system. '@type': type: string enum: - Team version: type: string description: Version of the schema. name: type: string description: Team name. description: type: string description: Team description. dateCreated: type: string format: date description: Date of creation. creator: type: string description: User that created the team. dateModified: type: string format: date description: Date of last modification. editor: type: string description: User that did last modification. token: type: string description: Share token used to join the team. isTraining: description: true if the team is a training team. type: boolean whisperers: type: array description: List of Whisperers the team users can access to. items: type: string description: System id of the whisperer. customers: type: object description: List of customers in the team. properties: rights: description: Rights associated to customer. type: object properties: whisperers: description: >- Can manage own team whisperers (add, remove, configuration, start/stop). type: boolean share: description: Can add/remove customers from the team. type: boolean rights: description: Can change customers rights in the team. type: boolean settings: description: Can edit teams common settings (UI). type: boolean update: description: Can edit name, description and share token. type: boolean publish: description: Can create public links on Team's own whisperers. type: boolean settings: type: object description: Common UI settings for customers. accessFilters: type: object properties: '@id': type: string description: Unique identifier for the access filter. name: type: string description: Name of the access filter. description: type: string description: Description of the access filter. whisperers: type: array items: type: string description: List of whisperer identifiers. view: type: string description: Type of view, e.g., HTTP. headersForbidden: type: array items: type: string description: List of forbidden HTTP headers. customers: type: array items: type: string description: List of customer identifiers. query: type: string description: Query to define the filter criteria. defaultFilter: type: boolean description: Indicates if this is the default filter. accessForbidden: type: boolean description: Indicates if access is forbidden. payloadsForbidden: type: boolean description: Indicates if HTTP payloads are forbidden access. required: - '@id' - name - whisperers - view example: '@id': AnaRheQISOmLu8bKiQv11w '@type': Team version: '0.1' name: First team description: Short team description. dateCreated: '2021-03-07T14:43:35.152Z' creator: hfQ1rsfcRKWslHwWAPJ9bg customers: - '@id': hfQ1rsfcRKWslHwWAPJ9bg email: owner@gmail.com rights: share: true rights: true whisperers: true settings: true update: true - '@id': FExgyFYtQdmskfNGpnKVBQ email: test@gmail.com rights: rights: true whisperers: true share: false whisperers: - dR6Gujz4RC2QyoAOyfDnWA - H_rczKsfRXSJbkgQMk-ZYQ - x7jxYx5PQb2wPEYa7D4Kgw - M_OhNqT8TWGEEdVWVBOmiQ - xGFWIX7zQMungzHcGz0YOw - qCC4TpvyThic6FVAYf2VTw settings: mergePattern: ^spiderdev_([^.]+).? clientsIdsCompactingPattern: ^http://spider.io/((?:apps|whisperers|customers)/.*)$ technicalStatus: ACTIVE dateModified: '2021-03-18T22:15:39.345Z' token: 7p8i66dCQsynmFNULKdnjA editor: hfQ1rsfcRKWslHwWAPJ9bg Whisperer: type: object properties: '@id': description: System id type: string '@type': type: string enum: - Whisperer version: description: Version of schema type: string name: description: Name of whisperer type: string customer: type: string description: Customer whose Whisperer it is team: type: string description: Team whose Whisperer it is dns: type: string description: Link to current Hosts List apikey: type: string format: base64 description: Public PEM of the Whisperer API Key config: $ref: '#/components/schemas/WhispererConfig' users: description: Customers to whom the whisperer is shared type: array items: type: object properties: '@id': description: System id of the customer type: string email: description: Email of the customer (on last update of rights) type: string rights: description: Rights of the customer on this whisperer type: object properties: record: description: Can start/stop capture type: boolean share: description: Can add/remove shared users type: boolean publish: description: Can create public links on this Whisperer type: boolean rights: description: Can change sharing rights type: boolean config: description: Can change configuration and name type: boolean delete: description: Can delete the whisperer type: boolean creator: type: string description: Creator of the Whisperer dateCreated: type: string description: Creation date format: date editor: type: string description: Last editor of the Whisperer dateModified: type: string description: Modification date format: date WhispererConfig: type: object description: Whisperer configuration properties: '@id': description: System id type: string '@type': type: string version: description: Version of schema type: string dateCreated: description: Creation date type: string format: date dateModified: description: Modification date type: string format: date editor: description: Last editor of the config type: string creator: description: Creator of the config type: string client: type: object description: Settings used client side - the Sniffer properties: capture: type: object description: Global capture settings properties: mode: type: string description: Mode of capture enum: - FILE - INTERFACE - UPLOAD file: type: string description: Path to file to load interface: type: string description: Name of network interface to capture on filter: type: string description: Pcap filter slowItDown: type: integer minimum: 0 exclusiveMinimum: true description: Factor to use to slow down time when reading file captureBufferkB: type: integer minimum: 0 exclusiveMinimum: true description: Size for capture buffer (kB) packets: type: object description: Packets tracking settings properties: sendBufferSizekB: type: integer minimum: 0 exclusiveMinimum: true description: Size for buffer of packets before flushing to server sendBufferDelay: type: string format: ISO8601 duration description: >- Delay before the buffer is flushed to server (first one wins between size and delay) maxSendingInParallel: type: integer minimum: 0 exclusiveMinimum: true description: Pool size for sending maxSendingBufferLength: type: integer minimum: 0 exclusiveMinimum: true description: Size of buffer (nb of requests) when pool is full vxlan: type: object description: >- VXLAN is the standard encapsulation of packets for virtual host on a virtual network. Used by VMWare and Docker (at least). Over UDP. properties: decapsulate: type: boolean description: >- True if Whisperer should look for VXLAN packets and extract encapsulated packets from them keepOriginal: type: boolean description: >- True if Whisperer should capture both original and encapsulated packets (beware: size > x2) filterHosts: type: object description: Hostnames in these list are filtered out or in properties: hostsToTrack: description: Patterns to identify hosts to track type: array items: type: string hostsToIgnore: description: Patterns to identify hosts to ignore type: array items: type: string trackByDefault: description: If an host does not match, track or not? type: boolean waitForNameResolvingToTrack: description: Wait for having the host name to track type: boolean trackUnresolvedIp: description: When no resolution, track or not? type: boolean required: - trackByDefault dumpPackets: type: object description: Packets dump to local file properties: dumpToFile: type: boolean description: True if Whisperer is storing captured packets to disk fileBufferSizekB: type: integer description: Size for file buffer (kB) outputPath: type: string description: Folder where to store pcaps dnsCache: type: object description: DNS used to trach hostnames properties: customDnsServer: type: boolean description: Use standard DNS of the host or a custom one host: type: string description: IP of the DNS server port: type: integer description: DNS port to use trackIp: type: boolean description: True if Whisperer should try to get FQDN from DNS ttl: type: string format: ISO8601 duration description: Delay before refreshing an IP hostname that we track refreshRate: type: string format: ISO8601 duration description: Delay before refreshing an IP hostname that we do not track sendFullDelay: type: string format: ISO8601 duration description: Delay before sending full list of hosts sendUpdateDelay: type: string format: ISO8601 duration description: >- Delay before sending partial list of hosts (only updated ones) purgeDelay: type: string format: ISO8601 duration description: Delay before purging a host not seen from long required: - trackIp tcpSessions: type: object description: Tcp sessions capture settings properties: track: type: boolean description: True if Whisperer should try TCP sessions sendSessionDelay: type: string format: ISO8601 duration description: Delay before sending sessions to server sessionTimeOut: type: string format: ISO8601 duration description: Delay before removing a session not updated since long maxSendingInParallel: type: integer minimum: 0 exclusiveMinimum: true description: Pool size for sending maxSendingBufferLength: type: integer minimum: 0 exclusiveMinimum: true description: Size of buffer (nb of requests) when pool is full required: - track circuitBreakers: type: object description: Circuit breakers on CPU and RAM settings properties: breakOnHighCpu: description: Stop capture if CPU is above threshold type: boolean maxCpu: description: Maximum CPU allowed (10% = 0.1) type: number minimum: 0 exclusiveMinimum: true breakOnHighRam: description: Stop capture if RAM is above threshold type: boolean maxRam: description: Maximum RAM allowed in MB type: integer minimum: 0 exclusiveMinimum: true required: - capture - packets - dnsCache - tcpSessions server: type: object description: Settings used server side, for parsing properties: dns: type: object description: Settings for Hosts Lists management properties: ttl: type: string format: ISO8601 duration description: Delay before refreshing a hostname purgeDelay: type: string format: ISO8601 duration description: Delay before removing a host not seen since long maxDelta: type: string format: ISO8601 duration description: >- Max delta between two HostLists that triggers creation of a new list customNamePatterns: type: array description: Patterns used to determine shortened name of host items: type: string pack: type: object properties: savePackets: description: Should the packets be saved or trashed once parsing is done type: boolean tcpStreams: type: object description: Settings for Tcp sessions management properties: saveTcpSession: type: boolean description: Should the TcpSession resource be saved parseHTTP: type: boolean description: True if server should parse HTTP sessions for this Whisperer parseHTTPOptions: type: object description: Settings for HTTP parsing properties: portsToParse: type: array description: List of ports (80, 8080), or ports range ([3000-3100]) items: type: integer minimum: 0 exclusiveMinimum: true portsToIgnore: type: array description: List of ports (80, 8080), or ports range ([3000-3100]) items: type: integer minimum: 0 exclusiveMinimum: true parseByDefault: type: boolean description: True if server should parse by default required: - parseByDefault required: - parseHTTP webstreams: type: object properties: saveContent: description: Should the payload parsed by saved in the Http resource type: boolean headersToFilter: type: array description: Will remove these headers from the saved resource items: type: string urisToFilter: type: array description: Will not save resources with these URI patterns items: type: string urisToFilterReqContent: type: array description: Will not save request payload of these URI patterns items: type: string urisToFilterResContent: type: array description: Will not save response payload of these URI patterns items: type: string saveRawHeaders: description: Should the headers by saved as is in the Http resource type: boolean urisToFilterReqRawHeaders: type: array description: Will not save request raw headers of these URI patterns items: type: string urisToFilterResRawHeaders: type: array description: Will not save response raw headers of these URI patterns items: type: string reqTemplates: type: array description: Patterns to determine template of the HTTP communication items: type: object properties: name: type: string description: Name of the template, may include capture group pattern: type: string description: Regular expression to match toParse: type: object description: Components of the string to match properties: verb: type: boolean uri: type: boolean headers: type: boolean body: type: boolean reqTags: type: array description: Patterns to extract tags of the request items: type: object properties: name: type: string description: Name of the template, may include capture group pattern: type: string description: Regular expression to match toParse: type: object description: Components of the string to match properties: verb: type: boolean uri: type: boolean headers: type: boolean body: type: boolean resTags: type: array description: Patterns to extract tags of the response items: type: object properties: name: type: string description: Name of the template, may include capture group pattern: type: string description: Regular expression to match toParse: type: object description: Components of the string to match properties: status: type: boolean headers: type: boolean body: type: boolean saveHttpParsingLogs: type: boolean description: Should the httppers parsing tracking resource be saved required: - tcpStreams HostsList: type: object properties: '@id': description: System id type: string '@type': type: string version: type: string description: Version of the schema whisperer: type: string description: System id of the whisperer owning this Hosts list aggregated: type: boolean description: True if resulting of the aggregation of all instance ids object instanceId: type: string description: Instance id of the whisperer dateStart: type: string description: Start of the period for which the hosts list is active format: date-time timestamp: type: number description: Start of the period for which the hosts list is active dateEnd: type: string description: End of the period for which the hosts list is active format: date-time dateModified: type: string description: Date of last modification format: date-time editor: type: string description: Client that last modified it hosts: type: object description: A hashmap with ip adresses as key and host model as value additionalProperties: type: object properties: ip: type: string description: IP address of host format: ipv4 name: type: string description: FQDN of host as given by DNS customName: type: string description: Custom name set on GUI or computed when parsing type: type: string description: Type of host enum: - SERVER - CLIENT - null firstSeen: type: string format: date-time description: Date of first packet seen for this host lastSeen: type: string format: date-time description: Date of last packet seen for this host lastUpdate: type: string format: date-time description: Last time the DNS was queried to update the host name example: '@id': YVWyUFNPRyCouvPBNmV9aw.dns '@type': HostsList version: '0.1' whisperer: YVWyUFNPRyCouvPBNmV9aw hosts: - - 10.0.0.5 - ip: 10.0.0.5 name: >- itproduction_gateway.lv1vnc7d26hhhhgjqlbqafvxp.x3pofn98yn117pbn17bdvj3qc type: CLIENT lastSeen: '2019-01-27T14:38:35.730Z' lastUpdate: '2019-01-22T11:23:36.430Z' firstSeen: '2019-01-22T11:23:36.430Z' customName: itproduction_gateway.lv - - 10.0.0.236 - ip: 10.0.0.236 name: itproduction_context_repository type: SERVER lastSeen: '2019-01-27T14:38:33.819Z' lastUpdate: '2019-01-27T14:38:25.775Z' firstSeen: '2019-01-22T11:23:36.428Z' dateStart: '2019-01-22T11:23:36.428Z' dateEnd: '2019-01-27T14:38:35.730Z' dateModified: '2019-01-27T14:38:35.730Z' timeStamp: 1548599915730 lastPurge: '2019-01-22T11:24:16.436Z' editor: VBqPbgjYRsK00O76DVeroQ== RawStatus: type: object description: Raw status sent from Whisperer properties: time: description: Time of status type: string format: date-time whisperer: description: System Id of whisperer type: string instanceId: type: string description: Instance id of the whisperer whispererName: description: Name of whisperer - enriched by service type: string hostname: description: FQDN of the host type: string startTime: description: Start time of the whisperer type: string format: date-time sessionStartTime: description: Recording session start time type: string upTime: description: Uptime of whisperer type: number state: description: State of the whisperer type: string enum: - STARTING - RECORDING - STOPPED - SERVER_DOWN - BREAK - INTERNAL_ERROR - INVALID_CONFIG circuitBreakers: description: Statistics of REST API calls to the server, by API type: object properties: summary: description: Summary of all API calls type: object properties: totalCount: description: Total requests count type: integer totalSuccessful: description: Total successful type: integer totalErrors: description: Total in error type: integer additionalProperties: type: object properties: key: description: Name of API type: string open: description: Is circuit breaker open type: boolean max90: description: 90 percentiles of latency type: number latencyMean: description: Average of latency type: number successful: description: Count of successful requests type: integer errors: description: Count of erroneous requests type: integer shortCircuited: description: Count of short circuited requests type: integer failed: description: Count of failed requests type: integer timedOut: description: Count of timed out requests type: integer totalCount: description: Total count of requests type: integer required: - summary total: description: Total of metrics values since start of session type: object properties: cpuUsage: description: CPU usage type: object properties: overall: description: Total CPU usage on the host type: number process: description: CPU usage of the whisperer type: number memoryUsage: description: RAM usage type: object properties: free: description: Free RAM on the host type: integer process: description: RAM usage of the whisperer type: integer pcapSession: description: Metrics on Pcap capture session type: object properties: received: description: Count of packets received type: integer dropped: description: Count of packets dropped (processing not fast enough) type: integer ifDropped: description: >- Count of packets dropped by interface (libpcap not fast enough) type: integer tcpSessions: description: Count of tcp sessions tracked type: integer hosts: description: Count of hosts currently tracked type: integer packets: type: object properties: count: description: Count of packets recorded (without the filtered) type: integer size: description: Size of captured data type: integer queues: description: Status of sending queues type: object properties: packets: description: Packets sending queue type: object properties: length: description: Count of items in queue type: integer overflow: description: >- Count of overflows - items that were dropped because the queue was full type: integer packetsVxlan: description: VX Lan packets sending queue type: object properties: length: description: Count of items in queue type: integer overflow: description: >- Count of overflows - items that were dropped because the queue was full type: integer tcpSessions: description: Tcp sessions sending queue type: object properties: length: description: Count of items in queue type: integer overflow: description: >- Count of overflows - items that were dropped because the queue was full type: integer new: description: Metrics values since last send of status type: object properties: cpuUsage: description: CPU usage type: object properties: overall: description: Total CPU usage on the host type: number process: description: CPU usage of the whisperer type: number packets: type: object properties: count: description: Count of packets recorded (without the filtered) type: integer size: description: Size of captured data type: integer queues: description: Status of sending queues type: object properties: packets: description: Packets sending queue type: object properties: overflow: description: >- Count of overflows - items that were dropped because the queue was full type: integer packetsVxlan: description: VX Lan packets sending queue type: object properties: overflow: description: >- Count of overflows - items that were dropped because the queue was full type: integer tcpSessions: description: Tcp sessions sending queue type: object properties: overflow: description: >- Count of overflows - items that were dropped because the queue was full type: integer pcapSession: description: Metrics on Pcap capture session type: object properties: received: description: Count of packets received type: integer dropped: description: Count of packets dropped (processing not fast enough) type: integer ifDropped: description: >- Count of packets dropped by interface (libpcap not fast enough) type: integer tcpSessions: description: Count of tcp sessions tracked type: integer interfaces: description: List of network interfaces of the host type: array items: description: A network interface type: object properties: interface: description: Interface name type: string address: description: IPv4 address type: string format: ipv4 netmask: description: Network mask type: string format: ipv4 family: description: Nework family type: string enum: - IPv4 mac: description: Mac address type: string format: mac address internal: description: >- True if the network interface is a loopback or similar interface that is not remotely accessible type: boolean cidr: description: CIDR address block type: string required: - time - state - whisperer - hostname - instanceId - startTime - upTime - total example: '@id': Y2GaZbDXRLq3cj-m8Zjviw.1548595914874 time: '2019-01-27T13:31:54.874Z' whisperer: Y2GaZbDXRLq3cj-m8Zjviw hostname: node-3.streetsmart.sit3 instanceId: 1a2aebcf734f startTime: '2019-01-16T14:40:19.843Z' sessionStartTime: '2019-01-25T11:17:40.234Z' upTime: 946295.88 state: RECORDING circuitBreakers: summary: totalCount: 20 totalSuccessful: 20 totalErrors: 0 Get whisperer config with API key: key: Get whisperer config with API key open: false max90: 0 latencyMean: 0 successful: 0 errors: 0 shortCircuited: 0 failed: 0 timedOut: 0 totalCount: 0 Post status: key: Post status open: false max90: 27 latencyMean: 27 successful: 1 errors: 0 shortCircuited: 0 failed: 0 timedOut: 0 totalCount: 1 Post packets: key: Post packets open: false max90: 44 latencyMean: 26 successful: 9 errors: 0 shortCircuited: 0 failed: 0 timedOut: 0 totalCount: 9 Post sessions: key: Post sessions open: false max90: 17 latencyMean: 12 successful: 7 errors: 0 shortCircuited: 0 failed: 0 timedOut: 0 totalCount: 7 Get whisperer config with token: key: Get whisperer config with token open: false max90: 14 latencyMean: 12 successful: 2 errors: 0 shortCircuited: 0 failed: 0 timedOut: 0 totalCount: 2 Post hosts: key: Post hosts open: false max90: 18 latencyMean: 18 successful: 1 errors: 0 shortCircuited: 0 failed: 0 timedOut: 0 totalCount: 1 total: cpuUsage: overall: 6080110.3 process: 19450.72 memoryUsage: free: 162 process: 132 pcapSession: received: 90041092 dropped: 208 ifDropped: 0 tcpSessions: 399033 hosts: 43 packets: count: 4067089 size: 1677558858 queues: packets: length: 0 overflow: 0 packetsVxlan: {} tcpSessions: length: 0 overflow: 0 new: cpuUsage: overall: 2.33 process: 0.02 packets: count: 925 size: 391289 queues: packets: overflow: 0 packetsVxlan: {} tcpSessions: overflow: 0 pcapSession: received: 10516 dropped: 0 ifDropped: 0 tcpSessions: 89 interfaces: - interface: lo address: 127.0.0.1 netmask: 255.0.0.0 family: IPv4 mac: '00:00:00:00:00:00' internal: true cidr: 127.0.0.1/8 - interface: eth0 address: 10.255.0.8 netmask: 255.255.0.0 family: IPv4 mac: 02:42:0a:ff:00:08 internal: false cidr: 10.255.0.8/16 - interface: eth2 address: 172.18.0.3 netmask: 255.255.0.0 family: IPv4 mac: 02:42:ac:12:00:03 internal: false cidr: 172.18.0.3/16 - interface: eth1 address: 10.0.0.7 netmask: 255.255.255.0 family: IPv4 mac: 02:42:0a:00:00:07 internal: false cidr: 10.0.0.7/24 CurrentStatus: type: object description: Current status of Whisperer compiled from raw statuses properties: '@id': description: System Id of the whisperer type: string '@type': type: string enum: - WhispStatus version: description: '' type: string time: description: Time of status type: string format: date-time timestamp: description: Time of status type: string format: number whisperer: description: System Id of whisperer type: string instanceId: type: string description: Instance id of the whisperer aggregated: type: boolean description: True of resulting of the aggregation of all instances data hostname: description: FQDN of the host type: string startTime: description: Start time of the whisperer type: string format: date-time upTime: description: Uptime of whisperer type: number state: description: State of the whisperer type: string enum: - STARTING - RECORDING - STOPPED - SERVER_DOWN - BREAK - INTERNAL_ERROR - INVALID_CONFIG sessionMetrics: type: object properties: startTime: description: Recording session start time type: string duration: description: Duration of session type: integer apiCalls: description: Statistics of REST API calls to the server, by API type: array items: type: object properties: key: description: Name of API type: string open: description: Is circuit breaker open type: boolean last: type: object properties: max90: description: 90 percentiles of latency type: number average: description: Average of latency type: number success: description: Count of successful requests type: integer errors: description: Count of erroneous requests type: integer shortCircuited: description: Count of short circuited requests type: integer failed: description: Count of failed requests type: integer timedOut: description: Count of timed out requests type: integer count: description: Total count of requests type: integer total: type: object properties: time: description: Total time spent calling this API type: number average: description: Average of latency type: number success: description: Count of successful requests type: integer errors: description: Count of erroneous requests type: integer shortCircuited: description: Count of short circuited requests type: integer failed: description: Count of failed requests type: integer timedOut: description: Count of timed out requests type: integer count: description: Total count of requests type: integer apiCallsSummary: description: Statistics for all API type: object properties: time: description: Total time spent calling all API type: number average: description: Average of latency type: number success: description: Count of successful requests type: integer errors: description: Count of erroneous requests type: integer shortCircuited: description: Count of short circuited requests type: integer failed: description: Count of failed requests type: integer timedOut: description: Count of timed out requests type: integer count: description: Total count of requests type: integer cpu: description: CPU usage type: number memory: description: RAM usage (MB) type: integer size: type: object properties: total: description: Total size of data uploaded type: integer lastSpeedPerMin: description: Last speed per min of data uploaded type: integer packets: type: object properties: totalCaptured: description: Total packets captured type: integer totalFiltered: description: Total packets filtered type: integer totalDropped: description: Total packets dropped type: integer totalOverflow: description: Total packets overflow type: integer lastSpeedPerMin: description: Last packets captured by min type: integer tcpSessions: type: object properties: total: description: Total tcp sessions tracked type: integer totalOverflow: description: Total Tcp sessions overflow type: integer lastSpeedPerMin: description: Last Tcp sessions tracked per min type: integer overallMetrics: type: object properties: size: description: Total size of data captured type: integer packets: description: Total count of packets captured type: integer tcpSessions: description: Total count of Tcp sessions tracked type: integer interfaces: description: List of network interfaces of the host type: array items: description: A network interface type: object properties: interface: description: Interface name type: string address: description: IPv4 address type: string format: ipv4 netmask: description: Network mask type: string format: ipv4 family: description: Nework family type: string enum: - IPv4 mac: description: Mac address type: string format: mac address internal: description: >- True if the network interface is a loopback or similar interface that is not remotely accessible type: boolean cidr: description: CIDR address block type: string example: '@id': UOZkEG7NRzuAkGMkVX29eg '@type': WhispStatus version: '0.1' whisperer: UOZkEG7NRzuAkGMkVX29eg time: '2019-01-27T15:08:19.217Z' timeStamp: 1548601699217 state: RECORDING hostname: node-1.streetsmart.sit1 instanceId: 845909b7bd7e startTime: '2019-01-27T05:28:08.379Z' upTime: 34813563 sessionMetrics: startTime: '2019-01-27T05:28:08.550Z' duration: 34810667 apiCalls: - key: Get whisperer config with API key open: false last: max90: 0 average: 0 count: 0 success: 0 failed: 0 timedOut: 0 shortCircuited: 0 errors: 0 total: time: 139 average: 139 count: 1 success: 1 failed: 0 timedOut: 0 errors: 0 shortCircuited: 0 - key: Post status open: false last: max90: 15 average: 15 count: 1 success: 1 failed: 0 timedOut: 0 shortCircuited: 0 errors: 0 total: time: 31246 average: 17 count: 1890 success: 1890 failed: 0 timedOut: 0 errors: 0 shortCircuited: 0 - key: Post packets open: false last: max90: 57 average: 26 count: 15 success: 15 failed: 0 timedOut: 0 shortCircuited: 0 errors: 0 total: time: 1272465 average: 38 count: 33747 success: 33747 failed: 0 timedOut: 0 errors: 0 shortCircuited: 0 - key: Post sessions open: false last: max90: 26 average: 15 count: 8 success: 8 failed: 0 timedOut: 0 shortCircuited: 0 errors: 0 total: time: 318661 average: 23 count: 14069 success: 14069 failed: 0 timedOut: 0 errors: 0 shortCircuited: 0 - key: Get whisperer config with token open: false last: max90: 9 average: 9 count: 1 success: 1 failed: 0 timedOut: 0 shortCircuited: 0 errors: 0 total: time: 33202 average: 12 count: 2805 success: 2805 failed: 0 timedOut: 0 errors: 0 shortCircuited: 0 - key: Post hosts open: false last: max90: 20 average: 17 count: 2 success: 2 failed: 0 timedOut: 0 shortCircuited: 0 errors: 0 total: time: 44145 average: 23 count: 1935 success: 1935 failed: 0 timedOut: 0 errors: 0 shortCircuited: 0 cpu: 0.03 memory: 77 size: total: 1503111464 lastSpeedPerMin: 2403834 packets: totalCaptured: 9420722 totalFiltered: 3322866 totalDropped: 0 totalOverflow: 0 lastSpeedPerMin: 5728 tcpSessions: total: 318127 totalOverflow: 0 lastSpeedPerMin: 555 apiCallsSummary: time: 1699858 average: 31 count: 54447 success: 54447 failed: 0 timedOut: 0 shortCircuited: 0 errors: 0 overallMetrics: size: 421508757347 packets: 987010023 tcpSessions: 92123551 interfaces: - interface: lo address: 127.0.0.1 netmask: 255.0.0.0 family: IPv4 mac: '00:00:00:00:00:00' internal: true cidr: 127.0.0.1/8 - interface: eth0 address: 10.255.0.9 netmask: 255.255.0.0 family: IPv4 mac: 02:42:0a:ff:00:09 internal: false cidr: 10.255.0.9/16 - interface: eth2 address: 172.19.0.4 netmask: 255.255.0.0 family: IPv4 mac: 02:42:ac:13:00:04 internal: false cidr: 172.19.0.4/16 - interface: eth1 address: 10.0.0.97 netmask: 255.255.255.0 family: IPv4 mac: 02:42:0a:00:00:61 internal: false cidr: 10.0.0.97/24 Link: description: A UI state shared between customers type: object example: '@id': fH5R3ZjhR7WPHBn2GDg1cA '@type': sp:link dateCreated: '2019-01-27T20:58:30.421Z' creator: VBqPbgjYRsK00O76DVeroQ== version: '0.1' content: {} PublicLink: type: object description: >- Represents a public link with various properties including access rights, time constraints, and associated metadata. properties: '@id': type: string description: A unique identifier for the link. '@type': type: string description: The type of the object, here it is a PublicLink. freeAccess: type: boolean description: Indicates if the link provides free access. recipients: type: array items: type: string description: A list of email addresses that are recipients of the link. domains: type: array items: type: string description: A list of domains whose email addresses may use the link. sendEmail: type: boolean description: Indicates if an email should be sent for the link. accessFilters: type: array items: type: object properties: view: type: string description: The type of view access provided by the filter. accessForbidden: type: boolean description: Indicates if access is forbidden under this filter. query: type: string description: A query associated with the filter. whisperers: type: array items: type: string description: A list of identifiers for whisperers access. required: - view - accessForbidden - whisperers description: Filters defining access control for different views. technicalStatus: type: string description: The technical status of the link (e.g., ACTIVE, DELETED). whisperers: type: array items: type: string description: A list of whisperers identifiers used in the link. timeRange: type: object properties: min: type: string format: date-time description: The start time for the link's validity. max: type: string format: date-time description: The end time for the link's validity. required: - min - max description: Specifies the time range during which the link shows data. sessions: type: array items: type: object properties: email: type: string description: The email address associated with the session. connectionDate: type: string format: date-time description: The date and time when the connection was made. required: - email - connectionDate description: >- A session represents a single use or connection instance to the link. content: type: object description: An object containing the UI state associated with the link. dateCreated: type: string format: date-time description: The date and time when the link was created. dateDeprecated: type: string format: date-time description: The date and time when the link was deprecated. customer: type: string description: The identifier of the owner of the link. creator: type: string description: The identifier of the creator of the link. version: type: string description: The version of the link. _eTag: type: string description: >- A version tag for the link, used for managing updates and concurrency. dateModified: type: string format: date-time description: The date and time when the link was last modified. editor: type: string description: The identifier of the last editor of the link. required: - accessFilters - sessions - content - dateDeprecated Session: type: object properties: '@id': type: string description: System id user: type: object description: Connected user properties: '@id': description: User system id type: string email: description: User email type: string format: email main: type: object properties: start: description: Start of session (launching the UI) type: string format: date-time stop: description: End of session (last user action) type: string format: date-time duration: description: Session duration type: integer reloads: description: Number of reload of page - reload of UI soon after last action type: integer views: description: Statistics on the view used type: object subViews: description: Statistics on the subview used type: object options: description: Options selected type: object whisperers: description: List of selected whisperers type: object actions: description: Statistics on the actions triggered by the user type: array items: description: Action type: object properties: name: description: Name of action type: string count: description: Count of action execution type: string required: - '@id' - views - user - main example: '@id': 06cb72b6-5a87-4baa-a3ca-620806a6aa7b dateCreated: '2019-01-27T21:03:38.750Z' creator: VBqPbgjYRsK00dhzdVeroQ version: '0.1' dateModified: '2019-01-27T21:18:41.746Z' updater: VBqPbgjYRsK00dhzdVeroQ user: '@id': VBqPbgjYRsK00dhzdVeroQ email: user@nomail.com main: app: network-view start: '2019-01-27T20:57:47.468Z' stop: '2019-01-27T20:58:38.320Z' duration: 50 reloads: 2 activeHours: - '2019-01-27T20:00:00.000Z' views: HTTP: duration: 44 active: true start: '2019-01-27T20:58:38.320Z' TCP: {} PACKET: {} subViews: TABLE: duration: 44 active: true start: '2019-01-27T20:58:38.320Z' SEQ_DIAG: {} STATS: {} options: hideGateways: active: false duration: 0 mergedReplicas: active: false duration: 0 showCircle: active: false duration: 0 menuOpened: active: false duration: 0 drawerOpened: active: true start: '2019-01-27T20:58:38.320Z' duration: 44 detailsPinned: active: false duration: 0 whisperers: selected: - SIT1 - W1 - SIT1 - W2 - SIT1 - W3 - SIT1 - W4 - SIT1 - W5 actions: - name: menu.whisperers.search count: 2 - name: menu.whisperers.select count: 4 - name: map.zoom count: 2 - name: timeLine.drag count: 1 - name: links.create count: 1 actionsTotalCount: 11 Job: type: object description: Based on https://schema.org/action properties: '@id': type: string '@type': type: string enum: - Job creator: type: string jobType: type: string description: Type of job enum: - PurgeJob - DownloadJob - UploadJob - ... jobParameters: description: Input of the job type: object progress: description: Progress in % type: integer whisperer: description: List of whisperers type: array items: description: System id of whisperers type: string startTime: description: Start of the job type: string format: date-time endTime: description: End of the job type: string format: date-time updateTime: description: Last update of the job type: string format: date-time actionStatus: description: Status of the job type: string enum: - ActiveActionStatus - CompletedActionStatus - FailedActionStatus - PotentialActionStatus result: description: Result of the job type: object error: description: Error of the job, if any type: object example: '@id': Ju_4O7N9QvWLC8x7PswfnQ '@type': Job jobType: DownloadJsonJob creator: wB0wdZgkTJqxcSZYv8yZ8g creationTime: '2019-01-18T15:20:09.158Z' updateTime: '2019-01-18T15:20:09.159Z' endTime: '2019-01-18T15:20:09.106Z' actionStatus: CompletedActionStatus progress: 100 jobParameters: user: user@nomail.com resourceType: HTTP totalItems: 25 result: totalItems: 25 totalHosts: 6 base64Size: 198684 whisperer: - 3k_C6E1STvy6VWGRdPC4Qg - YN1W5EgyRBKw1qqghVINyA GuiLog: type: object properties: '@id': description: System Id of the gui log type: string '@type': type: string enum: - GuiLog creator: type: string description: Creator of the Whisperer dateCreated: type: string description: Creation date format: date version: description: Version of schema type: string app: description: Name of application type: string enum: - Login - NetworkView - SelfMonitoring time: type: string format: date-time description: Time of log type: description: Type of error type: string enum: - BAD REQUEST - SERVER ERROR - UNEXPECTED - TIMEOUT - UNKNOWN - XXX RENDER name: description: Name of the error type: string enum: - Saga Error - React Error level: description: Level of Log type: string enum: - ERROR - WARNING message: description: Message of the error (from JS error.message) type: string stack: description: Stack of the error type: string details: description: Details of the error. Component stack in React. type: string whisperer: description: List of whisperers type: array items: description: System id of whisperers type: string customer: description: System id of the customer type: string timeout: description: Timeout value in case of timeouts type: integer origin: description: Origin IP of the client posting the log type: string format: ipv4 req: type: object description: Request that failed properties: method: description: HTTP method type: string uri: description: URI of request type: string body: description: Body in case of POST, PATCH, PUT type: string headers: type: object description: Headers of the request res: type: object description: Response that failed properties: status: description: Status of response type: string statusText: description: Status text of response type: string body: description: Body of response type: string headers: type: object description: Headers of the response example: app: NetworkView time: '2019-05-07T08:30:55.943Z' name: Saga error type: TIMEOUT level: WARNING message: >- Timeout in calling POST https://.../spider/whisp/v1/whisperers/_search?searchConfig, after 5s. stack: '' whisperer: [] customer: A8dj-1IGS_2KNg4jzLDs3Q timeout: 5 req: method: POST uri: https://.../spider/whisp/v1/whisperers/_search?searchConfig headers: Authorization: Bearer ... Accept: application/json Content-Type: application/json body: '' origin: 185.149.63.251 '@id': Zfo7LPiNSE6UvYlL_xIIeQ '@type': GUI-Log dateCreated: '2019-05-07T08:31:01.515Z' creator: A8dj-1IGS_2KNg4jzLDs3Q version: '0.1' GuiSettings: type: object properties: '@id': description: System Id of the customer type: string '@type': type: string enum: - GuiSettings creator: type: string description: Creator of the Whisperer dateCreated: type: string description: Creation date format: date version: description: Version of schema type: string settings: type: object description: GUI settings properties: global: type: object description: >- Global settings, may be overriden by team ones, like saved settings... user: type: object description: User specific settings, mostly layout and behavior options ApiStats: type: object properties: application: description: Name of service type: string hostname: description: Hostname of cluster node type: string instanceId: description: Docker instance type: string requests: description: Total requests count over all APIs type: integer successes: description: Total requests in success over all APIs type: integer errors: description: Total requests in error over all APIs type: integer errors4xx: description: Total requests in error with return code 4xx over all APIs type: integer errors5xx: description: Total requests in error with return code 5xx over all APIs type: integer duration: description: Total duration of all requests over all APIs type: integer api: type: object description: >- Object having keys as: "[Method] [Endpoint]" and Api metrics as nested objects. additionalProperties: type: object properties: method: description: HTTP method type: string endpoint: description: Endpoint (path with variables) type: string requests: description: Total requests count type: integer successes: description: Total requests in success type: integer errors: description: Total requests in error type: integer errors4xx: description: Total requests in error with return code 4xx type: integer errors5xx: description: Total requests in error with return code 5xx type: integer duration: description: Total duration of all requests type: integer example: application: tcp-streams-update hostname: traballand-Latitude-E7240 instanceId: c1033254c68f requests: 886 successes: 886 errors: 0 errors4xx: 0 errors5xx: 0 duration: 2311 api: POST /v1/parsing-jobs/:type: method: POST endpoint: /v1/parsing-jobs/:type requests: 213 successes: 213 errors: 0 errors4xx: 0 errors5xx: 0 duration: 603 percentiles: '0': 1 '25': 1 '50': 1 '75': 2 '90': 4 '95': 4 '99': 4 GET /v1/waiting-stats: method: GET endpoint: /v1/waiting-stats requests: 180 successes: 180 errors: 0 errors4xx: 0 errors5xx: 0 duration: 355 percentiles: '0': 1 '25': 1 '50': 1 '75': 2 '90': 2 '95': 2 '99': 2 PATCH /v1/tcp-sessions/: method: PATCH endpoint: /v1/tcp-sessions/ requests: 70 successes: 70 errors: 0 errors4xx: 0 errors5xx: 0 duration: 697 percentiles: '0': 4 '25': 4 '50': 5 '75': 5 '90': 5 '95': 5 '99': 5 CbStats: type: object properties: application: description: Name of service type: string hostname: description: Hostname of cluster node type: string instanceId: description: Docker instance type: string circuitBreakers: type: object description: 'List downstreams: services or datastores' additionalProperties: type: object example: application: tcp-streams-write hostname: spider4 instanceId: 9300ef2cec06 circuitBreakers: Redis: Save sessions: name: Save sessions group: Redis time: 1548884598363 open: false circuitDuration: 15000 threshold: 1 waitThreshold: 100 stats: failed: 0 timedOut: 0 total: 14 shortCircuited: 0 latencyMean: 6 successful: 14 percentiles: '0': 2 '1': 15 '0.25': 2 '0.5': 3 '0.75': 8 '0.9': 12 '0.95': 15 '0.99': 15 '0.995': 15 Get sessions: name: Get sessions group: Redis time: 1548884598363 open: false circuitDuration: 15000 threshold: 1 waitThreshold: 100 stats: failed: 0 timedOut: 0 total: 14 shortCircuited: 0 latencyMean: 0 successful: 14 percentiles: '0': 0 '1': 1 '0.25': 0 '0.5': 0 '0.75': 1 '0.9': 1 '0.95': 1 '0.99': 1 '0.995': 1 ES: Get sessions: name: Get sessions group: ES time: 1548884598363 open: false circuitDuration: 30000 threshold: 1 waitThreshold: 100 stats: failed: 0 timedOut: 0 total: 0 shortCircuited: 0 latencyMean: 0 successful: 0 percentiles: '0': 0 '1': 0 '0.25': 0 '0.5': 0 '0.75': 0 '0.9': 0 '0.95': 0 '0.99': 0 '0.995': 0 PackUpdate: POST /pack-update/packets/parsed: name: POST /pack-update/packets/parsed group: PackUpdate time: 1548884598359 open: false circuitDuration: 10000 threshold: 1 waitThreshold: 100 stats: failed: 0 timedOut: 0 total: 1 shortCircuited: 0 latencyMean: 3 successful: 1 percentiles: '0': 3 '1': 3 '0.25': 3 '0.5': 3 '0.75': 3 '0.9': 3 '0.95': 3 '0.99': 3 '0.995': 3 ProcessStats: type: object properties: application: description: Name of service type: string hostname: description: Hostname of cluster node type: string instanceId: description: Docker instance type: string startTime: description: Service start time type: string format: date-time upTime: description: Time in s since service start time type: number cpu: type: object properties: overall: type: object properties: cores: type: integer description: Count of CPU cores idle: type: number description: CPU idle time since start (s) usage: type: number description: CPU usage time since start (s) total: type: number description: CPU total time since start (s) process: type: object properties: user: description: CPU user time since start (s) type: number system: description: CPU system time since start (s) type: number memory: type: object properties: total: description: System total memory in MB type: integer free: description: Free memory in MB type: integer process: description: Process memory usage in MB type: integer example: application: tcp-streams-write hostname: spider7 instanceId: dd8ff8ac5565 startTime: '2019-01-16T21:56:26.109Z' upTime: 1209512.309 cpu: overall: cores: 2 idle: 62050469.2 usage: 27683911.2 total: 89734380.4 process: user: 14021.34 system: 1117.168 memory: total: 8061386752 free: 2321321984 process: 113864704 ParsingStats: type: object properties: application: description: Name of service type: string hostname: description: Hostname of cluster node type: string instanceId: description: Docker instance type: string parsed: description: Count of parsed Tcp sessions type: integer created: description: Count of created communications from Tcp type: integer errors: description: Count of errors when parsing type: integer completed: description: Count of successful parsing iteration type: integer started: description: Count of parsing iteration type: integer duration: description: Total of duration spent parsing type: number delay: description: >- Total of delay between time of parsing and first tcp session synchro time type: integer durationPercentiles: description: Percentiles of duration distribution type: object delayPercentiles: description: Percentiles of duration distribution type: object example: application: web-streams-write hostname: spider4 instanceId: cce9207215ae parsed: 13259 created: 5005 errors: 0 completed: 856 duration: 131623.869581 started: 856 delay: 8747139 durationPercentiles: '0': 51.69567 '25': 67.413543 '50': 83.969753 '75': 94.689424 '90': 145.061248 '95': 1009.291372 '99': 1050.284597 '100': 1059.290069 delayPercentiles: '0': 10012 '25': 10187 '50': 10297 '75': 10395 '90': 10425 '95': 10446 '99': 10458 '100': 10479 PollerStats: type: object properties: count: description: Items count in queue type: integer first: description: Date of first item in queue type: string format: date-time last: description: Date of last item in queue type: string format: date-time example: count: 182 first: '2019-01-30T22:35:51.254Z' last: '2019-01-30T22:36:01.641Z' Error: type: object properties: title: type: string description: type: string example: '@type': Error title: Error title message: Error details securitySchemes: Bearer: description: > To access the API a valid JWT token must be passed in almost all the queries in the 'Authorization' header. The JWT token is generated by the API and returned as answer to: - POST /customer/v1/sessions - for users, providing valid email & password - POST /whisp/v1/whisperers/{id}/config/ - for whisperers, signing the call with a valid API key - Private services init call to get their configuration The following syntax must be used in the 'Authorization' header : 'Bearer {token}' Spider fields in the JWT: - isWhisperer: boolean, true for Whisperers - isController: boolean, true for Controllers - isGocipher: boolean, true for Gociphers - isAdmin: boolean, true for admins - isCustomer: boolean, true for customers - isApplication: boolean, true for application - isUsingTeam: if the token is a team token - isImpersonating: if the token is of another user - customer: system id of customer, if a customer - team: system id of team, when customer is using a team - whisperer: system id of whisperer, if a whisperer - impersonated: system id of impersonated customer - instanceId: instance id given by the whisperer to identify its specific instance, or '_' for uploaders - application: system id of application, if an application - whisperers: [] array of system Ids of attached whisperers type: http scheme: bearer bearerFormat: JWT externalDocs: description: Spider Analyzer Website url: https://spider-analyzer.io/