Troubleshoot

Use this page when you know the symptom but are not sure which Sauron page to read first. If your question is about how to use a Sauron feature, start with the linked help page for that feature. Escalate only after you have completed the relevant checks below or when the request requires a Sauron-team action.

Symptom area First component or page to check
Access or login User Access, OIM, SSO, and the failed endpoint.
Missing metrics Prometheus target status, scrape config, Prometheus query, and Thanos query.
Missing logs OpenSearch ingest/search, OpenSearch Dashboards or Kibana time range, and index pattern.
Alert not delivered Prometheus alert state, Alertmanager route, receiver, silences, and delivery error.
Dashboard has no data Grafana panel query, datasource, time range, and the same query in Prometheus or Thanos.
Can I do this myself?
Task or symptom Start here Self-service?
Request endpoint access after the OIM entitlement exists User Access and OIM Yes. Request the correct entitlement in OIM and retry after approval.
SSO login fails or shows access denied SSO Usually yes. Check the URL, entitlement, role, and retry in an incognito browser.
Configure Basic Auth or Bearer token access for clients User Access Yes when the endpoint supports that method and you have the needed credential or token.
Test first metric or first log event Sending Metrics and Sending Logs Yes. Use the documented first-test flows before production setup.
Prometheus target is missing or down Sending Metrics Yes. Check scrape config, target reachability, target status, and scrape error.
Grafana dashboard has no data Create Grafana Dashboards Yes. Compare the same query in Grafana, Prometheus, and Thanos for the same time range.
Alert is not delivered Defining Alerts Yes. First confirm the alert is firing, then check routes, receivers, silences, and delivery errors.
Need a new Sauron provisioned FAQs No. This requires a provisioning request in #sauron-support.
Restore many indices, change retention, unlock, or re-enable FAQs No. These require Sauron-team action.
Enable SMTP, mTLS, Grafana plugins, or OpenSearch plugins FAQs and Defining Alerts No. These require Sauron-team enablement, review, or approval.
Common Symptoms
I am not sure which page to read

Use the Help home page as the starting point:

  • Access, roles, or credentials: User Access, OIM, and SSO.
  • Metrics, scrape jobs, PushProx, or OpenTelemetry: Sending Metrics.
  • Dashboards or PromQL panels: Grafana Dashboards.
  • Logs, index patterns, or OpenSearch ingestion: Sending Logs.
  • Alert rules, routes, receivers, or notifications: Alerts.
  • Restricted-network scraping: Shuttleproxy.
  • Restore, retention, unlock, re-enable, or other privileged action: FAQ escalation guidance.

If you are still unsure, include the page you tried and the specific task you are trying to complete when escalating.

I cannot log in, or I get Access denied

First, confirm which endpoint failed. Grafana, Prometheus, Thanos, Alertmanager, API, Help, Console, and OpenSearch Dashboards can have different access paths depending on how your Sauron is configured.

Check these items:

  • Confirm you are using the correct Sauron URL.
  • Confirm the expected OIM entitlement has been requested and approved.
  • Sign out of Oracle SSO and sign back in, or test in an incognito browser.
  • If only Grafana fails, confirm whether the expected role is Admin, Editor, or Viewer.
  • If a non-Grafana endpoint fails, confirm whether your role is allowed to use that endpoint.

More details:

My Grafana dashboard shows no data

Check the time range first. A dashboard can look empty when the selected time range does not overlap with available metrics.

Then check:

  • The dashboard datasource.
  • The PromQL query in the panel.
  • Dashboard variables and selected labels.
  • Whether the same query returns data in Thanos or Prometheus.
  • Whether the expected metric exists and is still being scraped.

More details:

My Prometheus target is missing or down

A successful configuration update means Sauron accepted the request. It does not prove the target is reachable or valid.

Check these items:

  • Fetch the current Prometheus config and confirm your scrape job is present.
  • Check the Prometheus targets page for the job and target health.
  • Confirm the target host, port, scheme, path, TLS settings, and credentials.
  • Confirm the network allowlist permits Sauron Prometheus to reach the target.
  • If using Kubernetes discovery, confirm labels, namespaces, and service port names match the scrape configuration.

More details:

Shuttleproxy returns 403, 504, or does not register

Shuttleproxy problems usually come from egress/DNS, authentication or domain configuration, client registration, target reachability, or changed endpoint allowlists.

Check these items:

  • The Shuttleproxy client can resolve and reach the Shuttleproxy endpoint.
  • The configured domain ID and client ID match the Sauron setup.
  • The client is using real credentials, not placeholder values.
  • The expected target appears in the Prometheus targets page.
  • Recent endpoint IP or firewall changes have been allowlisted.

More details:

Logs are missing from OpenSearch Dashboards or Kibana

First widen the time range in OpenSearch Dashboards or Kibana and confirm the index pattern you are searching. Then check the client sending logs.

Check these items:

  • Endpoint URL and credentials.
  • Client type and version.
  • Output plugin and index name.
  • Bulk request errors or proxy errors.
  • Whether any documents are visible in a wider time range.

More details:

Alert notifications are not delivered

First confirm whether the alert is firing. If an alert is not firing, debug the rule or query before debugging notification delivery.

Check these items:

  • Alert rule state.
  • Alertmanager route matchers.
  • Receiver configuration.
  • Silences and inhibitions.
  • Webhook, email, Slack, PagerDuty, or Ocean delivery errors.
  • Expired or rotated tokens, without sharing token values in Slack.

More details:

Escalate After These Checks

Escalate only after the relevant symptom checklist above has been completed, or when the request requires Sauron-team action such as restore, retention change, unlock, re-enable, provisioning, plugin enablement, SMTP enablement, or mTLS enablement.

When you escalate in #sauron-support, include enough detail for the Sauron team to identify the affected instance and reproduce the symptom:

Sauron URL:
Component:
Endpoint:
Auth method:
Start time:
Expected behavior:
Actual behavior:
Exact error/status or screenshot:
Recent changes:
Production impact:
Checks completed:
Page followed:

Do not paste passwords, bearer tokens, private keys, SMTP credentials, webhook tokens, or other secret values in Slack.

I am getting [WARNING] Certificate did not match expected hostname error when accessing my Sauron endpoint

Your Sauron was identified as possibly being no longer in use. We made best efforts to contact the owner which was unsuccessful. It was disabled due to lack of use. Please submit a request to re-enable Sauron on #sauron-support and provide multiple emails of contacts for this Sauron, so we have correct email ids on record.

When I try to login, I get error after SSO login Internal Server Error: Failed to authenticate: no attribute with name 'odxUserRoles'

(Due to IDCS restriction) when a user tries to access a Sauron and he is not part of any Sauron OIM entitlement, Sauron will throw this error in browser.

Please make sure you have the correct OIM entitlement needed to access this endpoint. Contact owner of your Sauron and submit request using OIM to get access. Once your request is approved, try again in an incognito browser.

I am not able to access any Sauron URL using SSO, getting Access denied message.

Assuming your team name is MyTeam, you need to belong to one of these OIM entitlements - MyTeamAdmin or MyTeamViewer or MyTeamEditor - to access your Sauron endpoints, else your access will get Access Denied error. Contact owner of your Sauron and submit request using OIM to get access. Once your request is approved, try again in an incognito browser.

If that doesn't resolve your issue, you need to make sure your ssoGroup for your Sauron matches your team name MyTeam. See SSO documentation for more details.

Note: MyTeamEditor entitlement is applicable only to Grafana endpoint. To access your non-Grafana endpoints, you will also need "MyTeamAdmin" or "MyTeamViewer" entitlement.

Please see https://help.stage.apatil.developers.oracledx.com/oim/ for more details