Oracle Cloud Infrastructure Documentation

Scenario: Analyzing Logs

Send log data to Log Analytics.

This scenario involves creating a log group and a connector. The connector (Connector Hub)  processes and moves log data from Logging to the log group in Log Analytics.

Required IAM Policy

If you're a member of the Administrators group, you already have the required access to execute this scenario. Otherwise, you need access to Log Analytics to create the log group and access to Connector Hub to create the connector.

The workflow for creating the connector includes a default policy when needed to provide permission for writing to the target service. If you're new to policies, see IAM Policies Overview.

Setting Up This Scenario

Setup is easy in the Console. Alternatively, you can use the Oracle Cloud Infrastructure CLI or API, which lets you execute the individual operations yourself.

This example walks through creating a connector that sends log data from Logging to the log group you created using Log Analytics. In this example, the connector filters VCN flow log.

For help with troubleshooting, see Troubleshooting Connectors.

  • Note

    For complete descriptions of fields on the Create connector page, see Creating a Connector.

    Before creating the connector, create the log group using Log Analytics. For instructions, see Create Log Groups.

    On the Connectors list page, select Create connector. If you need help finding the list page, see Listing Connectors.

    1. Basic Connector Information

    Enter identifying information.

    • Connector name: Enter a user-friendly name for the new connector and an optional description. Avoid entering confidential information. Example: VCN Flow Log Error Analysis
    • Select a compartment: Select the compartment to store the new connector in.

    Select Next.

    2. Configure Connector Source

    Select your VCN flow log.

    • Select source: Select Logging.
    • Configure source: Select the compartment, log group, and your VCN flow log.
    • Log filter task: Filter the log data to rejected traffic.
      • Property: data.action
      • Operator: =
      • Value: REJECT

      If you're interested in rejected traffic for a particular port or address, add another filter. For example, select the property data.destinationPort or data.destinationAddress

    Select Next.

    3. Configure Connector Task

    Select Next (skip this section).

    4. Configure Connector Target

    Select the log group that you created.

    • Target: Select Logging Analytics.
    • Configure target: Select the compartment and the log group that you created.
    • Create policy: Select to accept the default policy provided for the entered target configuration.

    Select Next.

    Connector Preview

    Review the connector configuration and then select Create.

    The creation process begins, and its progress is displayed. On completion, the connector's details page opens.

  • This section walks through creating the log group and connector using the CLI.

    For information about using the API and signing requests, see REST API documentation and Security Credentials. For information about SDKs, see SDKs and the CLI.

    1. Create a log group: Open a command prompt and run the oci log-analytics log-group create command: 

      oci log-analytics log-group create --display-name "<display_name>" --compartment-id <compartment_OCID> --namespace-name "<namespace_name>"

    2. Create a connector: Open a command prompt and run the oci sch service-connector create command: 

      oci sch service-connector create --display-name "<display_name>" --compartment-id <compartment_OCID> --source [<source_in_JSON>] --tasks [<tasks_in_JSON>] --target [<targets_in_JSON>]

  • This section walks through creating the log group and connector using the API.

    For information about using the API and signing requests, see REST API documentation and Security Credentials. For information about SDKs, see SDKs and the CLI.

    Use the following operations:

    1. CreateLogAnalyticsLogGroup: Create a log group.

      Example CreateLogAnalyticsLogGroup request
      post /20200601/namespaces/<namespaceName>/logAnalyticsLogGroups
Host: loganalytics.us-phoenix-1.oci.oraclecloud.com
<authorization and other headers>
{
  "compartmentId": "<compartment_OCID>",
  "displayName": "My Log Group"
}

    2. CreateServiceConnector: Create a connector.

      Example CreateServiceConnector request
      POST /20200909/serviceConnectors
Host: service-connector-hub.us-phoenix-1.oraclecloud.com
<authorization and other headers>
{
  "compartmentId": "<compartment_OCID>",
  "description": "My connector description",
  "displayName": "My Connector",
  "source": {
    "kind": "logging",
    "logSources": [
      {
        "compartmentId": "<compartment_OCID>",
        "logGroupId": "<log_group_OCID>",
        "logId": "<log_OCID>"
      }
    ]
  },
  "target": {
    "compartmentId": "<compartment_OCID>",
    "kind": "loggingAnalytics",
    "logGroupId": "<logging_analytics_log_group_OCID>"
  },
  "tasks": [
    {
      "condition": "data.action='REJECT'",
      "kind": "logRule"
    }
  ]
  }
}