Sending Logs

Use this page to send a first log event to Sauron OpenSearch and verify it in OpenSearch Dashboards or Kibana before configuring a log shipper.

Component or client Role in this page
OpenSearch Receives and stores log documents.
OpenSearch Dashboards or Kibana Searches and verifies indexed logs in a browser.
Filebeat, FluentBit, Fluentd, Logstash, or another shipper Sends production logs after the first bulk test succeeds.
Send your first log event

Use this quick test to confirm your OpenSearch endpoint accepts log data before you configure Filebeat, FluentBit, Logstash, or another log shipper.

You need:

  • Your Sauron OpenSearch endpoint.
  • A user that can write to OpenSearch, such as the reporter user or another approved credential for your Sauron.
  • An index name for the test event.
Basic OpenSearch bulk test

If you have a brand new OpenSearch endpoint and want to make sure you can send logs to it, follow the steps below.

  • Create a small sauron-log-test.json file with two test log events:
cat > sauron-log-test.json <<'EOF'
{"index":{"_id":"1"}}
{"message":"Sauron log ingestion test event 1","service":"sauron-doc-test","level":"INFO"}
{"index":{"_id":"2"}}
{"message":"Sauron log ingestion test event 2","service":"sauron-doc-test","level":"INFO"}
EOF
cat sauron-log-test.json
  • Bulk load sauron-log-test.json into OpenSearch:
curl --fail --show-error -u sauron:mypassword \
  -X POST \
  -H 'Content-Type: application/x-ndjson' \
  --data-binary @sauron-log-test.json \
  'https://elasticsearch.stage.apatil.developers.oracledx.com/sauron-log-test/_bulk?pretty'
  • Verify both test events are available for search:
curl --fail --show-error -u sauron:mypassword \
  'https://elasticsearch.stage.apatil.developers.oracledx.com/sauron-log-test/_search?pretty'

In the response, confirm the test events are returned. For example, look for:

"service" : "sauron-doc-test"
  • If the response returns the test events, your OpenSearch endpoint accepted the bulk request. You can then configure your normal log shipper.
Prove it works before production

Before relying on a log shipper in production:

  1. Send the small bulk test event above.
  2. Confirm OpenSearch returns the expected test documents.
  3. Open OpenSearch Dashboards or Kibana and widen the time range.
  4. Confirm the index pattern that matches your index name shows records.
  5. Configure your normal shipper only after the endpoint, auth method, index name, and time range are known-good.
Using FluentBit to send logs to OpenSearch

Please refer to FluentBit documentation for more details

Find logs in OpenSearch Dashboards or Kibana

After data is indexed, open https://kibana.stage.apatil.developers.oracledx.com and search for the index pattern that matches your index name. If you do not see your logs, widen the time range and confirm the index name, timestamp field, and shipper output configuration.

Self-check before escalation

Most questions for this topic can be resolved with the steps above. For missing logs or OpenSearch ingestion errors, check and collect:

  • Sauron OpenSearch endpoint.
  • Component or endpoint.
  • Client or shipper name and version.
  • Output plugin name.
  • Index name or index pattern.
  • Authentication method used.
  • Exact error/status from bulk request, proxy, or dashboard.
  • Start time.
  • Recent credential, client, index, proxy, or deployment changes.
  • Checks completed.
  • Production impact, if any.
  • Whether any documents are visible in a wider time range.

Do not paste passwords, bearer tokens, private keys, or other secrets in Slack.