Skip to main content

Nginx Plus

Thumbnail icon

The Sumo Logic App for Nginx Plus supports logs as well as Metrics for Nginx Plus, which is a web server that can be used as a reverse proxy, load balancer, mail proxy, and HTTP cache.

The Nginx Plus app is an unified logs and metrics app that helps you monitor the availability, performance, health and resource utilization of your Nginx Plus web servers. Preconfigured dashboards and searches provide insight into server status, location zones, server zones, upstreams, resolvers, visitor locations, visitor access types, traffic patterns, errors, web server operations and access from known malicious sources.

Log and Metrics Types​

The Sumo Logic App for Nginx Plus assumes the NCSA extended/combined log file format for Access logs and the default Nginx error log file format for error logs.

All Dashboards (except the Error logs Analysis dashboard) assume the Access log format. The Error logs Analysis Dashboard assumes both Access and Error log formats, so as to correlate information between the two. For more details on Nginx/NginxPlus logs, see Module ngx_http_log_module.

The Sumo Logic App for Nginx Plus assumes Prometheus format Metrics for Requests and Connections. For Nginx Plus Server metrics, API Module from Nginx Configuration is used. For more details on Nginx Plus Metrics, see Module ngx_http_api_module.

Sample Log Messages​

Access Log Example
{
    "timestamp":1620821977736,
    "log":"10.244.0.132 - - [12/May/2021:12:19:28 +0000] \"GET //demo-index.html HTTP/1.1\" 200 8777 \"-\" \"curl/7.68.0\"",
    "stream":"stdout",
    "time":"2021-05-12T12:19:28.975861476Z"
}
Error Log Example
{
    "timestamp":1620821977737,
    "log":"2021/05/12 12:19:36 [error] 7#7: *8192 upstream timed out (110: Connection timed out) while connecting to upstream, health check \"\" of peer 44.240.53.50:12345 in upstream \"stream_backend2\"",
    "stream":"stderr",
    "time":"2021-05-12T12:19:36.344706832Z"
}

Sample Queries​

This sample Query is from the Nginx Plus - Overview dashboard > Responses Over Time panel.

_sourceCategory=Labs/Nginx/Logs
| json auto maxdepth 1 nodrop
| if (isEmpty(log), _raw, log) as nginx_log_message
| parse regex field=nginx_log_message "(?<Client_Ip>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})"
| parse regex field=nginx_log_message "(?<Method>[A-Z]+)\s(?<URL>\S+)\sHTTP/[\d\.]+\"\s(?<Status_Code>\d+)\s(?<Size>[\d-]+)\s\"(?<Referrer>.*?)\"\s\"(?<User_Agent>.+?)\".*"
| where _sourceHost matches "{{Server}}" and Client_Ip matches "{{Client_Ip}}" and Method matches "{{Method}}" and URL matches "{{URL}}" and Status_Code matches "{{Status_Code}}"
| if(Status_Code matches "2*", 1, 0) as Successes
| if(Status_Code matches "3*", 1, 0) as Redirects
| if(status_code matches "4*", 1, 0) as Client_Errors
| if(Status_Code matches "5*", 1, 0) as Server_Errors
| timeslice by 5m
| sum(Successes) as Successes, sum(Client_Errors) as Client_Errors,  sum(Redirects) as Redirects, sum(Server_Errors) as Server_Errors by _timeslice
| sort by _timeslice asc

Collecting Logs and Metrics for Nginx Plus​

This section provides instructions for configuring log and metric collection for the Sumo Logic App for Nginx Plus. Sumo Logic supports a collection of logs and metrics data from Nginx Plus in both Kubernetes and non-Kubernetes environments. Click on the appropriate links below based on the environment where your Nginx Plus servers are hosted.

For Kubernetes environments​

In Kubernetes environments, we use the Telegraf Operator, which is packaged with our Kubernetes collection. You can learn more about it here. The diagram below illustrates how data is collected from Nginx Plus in Kubernetes environments. In the architecture shown below, there are four services that make up the metric collection pipeline: Telegraf, Telegraf Operator, Prometheus, and Sumo Logic Distribution for OpenTelemetry Collector.


Web servers

The first service in the pipeline is Telegraf. Telegraf collects metrics from Nginx Plus. Note that we’re running Telegraf in each pod we want to collect metrics from as a sidecar deployment: i.e. Telegraf runs in the same pod as the containers it monitors. Telegraf uses the Nginx Plus input plugin to obtain metrics. For simplicity, the diagram doesn’t show the input plugins. The injection of the Telegraf sidecar container is done by the Telegraf Operator. Prometheus pulls metrics from Telegraf and sends them to Sumo Logic Distribution for OpenTelemetry Collector, which enriches metadata and sends metrics to Sumo Logic.

In the logs pipeline, Sumo Logic Distribution for OpenTelemetry Collector collects logs written to standard out and forwards them to another instance of Sumo Logic Distribution for OpenTelemetry Collector, which enriches metadata and sends logs to Sumo Logic.

Collect Logs for Nginx Plus in Kubernetes environment​

Nginx Plus app supports the default access logs and error logs format.

  1. Before you can configure Sumo Logic to ingest logs, you must configure the logging of errors and processed requests in NGINX Open Source and NGINX Plus. For instructions, refer to the Ngnix Plus Configuring Logging documentation.
  2. Use the Sumologic-Kubernetes-Collection, to send the logs to Sumologic. For more information, visit.
  3. Identifying the logs metadata: For example, to get Logs data from the pod, you can use the following source _sourcecategory = "kubernetes/default/nginx" where kubernetes is Cluster name, default is Namespace, nginx is application.
  4. To get log data from Nginx Pods - all nginx logs must be redirected to standard output “stdout” and standard error “stderr”.

Collect Metrics for Nginx Plus in Kubernetes environment​

Nginx Plus app supports the metrics for Nginx Plus.

The following steps assume you are collecting Nginx Plus metrics from a Kubernetes environment. In Kubernetes environments, we use the Telegraf Operator, which is packaged with our Kubernetes collection. You can learn more about this here.

  1. Before you can configure Sumo Logic to ingest metrics, you must enable the API module to expose metrics in NGINX Plus.
  2. Set up Kubernetes Collection with the Telegraf Operator.
  3. On your Nginx Plus Pods, add the following annotations to configure Telegraf.
    annotations:
         telegraf.influxdata.com/inputs: |+
         [[inputs.nginx_plus_api]]
            urls = ["http://localhost:8080/api"]
            response_timeout = "5s"
            api_version = 6
         telegraf.influxdata.com/class: sumologic-prometheus
         prometheus.io/scrape: "true"
         prometheus.io/port: "9273"
  • telegraf.influxdata.com/inputs - This contains the required configuration for the Telegraf Nginx Plus Input plugin. Please refer to this doc for more information on configuring the Nginx input plugin for Telegraf. Note since telegraf will be run as a sidecar the host should always be localhost
  • telegraf.influxdata.com/class: sumologic-prometheus - This instructs the Telegraf operator what output to use. This should not be changed.
  • prometheus.io/scrape: "true" - This ensures our Prometheus will scrape the metrics.
  • prometheus.io/port: "9273" - This tells Prometheus what ports to scrape on. This should not be changed.

Field Extraction Rules​

Field Extraction Rules (FERs) tell Sumo Logic which fields to parse out automatically. For instructions, on creating them, see Create a Field Extraction Rule.

Nginx assumes the NCSA extended/combined log file format for Access logs and the default Nginx error log file format for error logs.

Both the parse expressions can be used for logs collected from Nginx Plus Server running on Local or container-based systems.

For FER for Access Logs, use the following Parse Expression:

| json field=_raw "log" as nginx_log_message nodrop
| if (isEmpty(nginx_log_message), _raw, nginx_log_message) as nginx_log_message
| parse regex field=nginx_log_message "(?<Client_Ip>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})"
| parse regex field=nginx_log_message "(?<Method>[A-Z]+)\s(?<URL>\S+)\sHTTP/[\d\.]+\
"\s(?<Status_Code>\d+)\s(?<Size>[\d-]+)\s\"(?<Referrer>.*?)\"\s\"(?<User_Agent>.+?)\".*"

For FER for Error Logs, use the following Parse Expression:

| json field=_raw "log" as nginx_log_message nodrop
| if (isEmpty(nginx_log_message), _raw, nginx_log_message) as nginx_log_message
| parse regex field=nginx_log_message "\s\[(?<Log_Level>\S+)\]\s\d+#\d+:\s(?:\*\d+\s|)(?<Message>[A-Za-z][^,]+)(?:,|$)"
| parse field=nginx_log_message "client: *, server: *, request: \"* * HTTP/1.1\", host:
\"*\"" as Client_Ip, Server, Method, URL, Host nodrop

Installing Nginx Plus Monitors​

To install these alerts, you need to have the Manage Monitors role capability.

Alerts can be installed by either importing them via a JSON or via a Terraform script.

Method A: Importing a JSON file​

  1. Download the JSON file describing all the monitors.
  2. Replace $$logs_data_source and $$metric_data_source with logs and metrics data sources respectively. For example, _sourceCategory=Labs/Nginx/Plus/Logs.
  3. Go to Manage Data > Alerts > Monitors.
  4. Click Add.
  5. Click Import to import monitors from the JSON above.
note

The monitors are disabled by default. Once you have installed the alerts via this method, navigate to the Nginx Plus folder under Monitors to configure them. Refer to this document to enable monitors, to configure each monitor, to send notifications to teams or connections.

Method B: Using a Terraform script​

  1. Generate a Sumo Logic access key and access ID for a user that has the Manage Monitors role capability in Sumo Logic using these instructions. Please identify your Sumo Logic deployment.
  2. Download and install Terraform 0.13 or later
  3. Download the Sumo Logic Terraform package for Nginx Plus alerts. The alerts package is available in the Sumo Logic github repository. You can either download it via the “git clone” command or as a zip file.
  4. Alert Configuration: After the package has been extracted, navigate to the package directory terraform-sumologic-sumo-logic-monitor/monitor_packages/nginx-plus/. Edit the nginxplus.auto.tfvars file as per below instructions:
    1. Add the Sumo Logic Access Key, Access Id, Deployment from Step 1.
      access_id  = "<YOUR SUMO ACCESS ID>"
      access_key = "<YOUR SUMO ACCESS KEY>"
      environment = "<DEPLOYMENT>"
    2. Add the data source values.
    • Metric_data_source - Sumo Logic data source for nginx plus metrics.
    • Logs_data_source - Sumo Logic data source for logs.
    1. All monitors are disabled by default on installation. If you would like to enable all the monitors, set the parameter monitors_disabled to false.
    2. All monitors are configured in a monitor folder called “Nginx Plus”, if you would like to change the name of the folder, update the parameter folder.
  5. Email and Connection Notification Configuration Examples: Modify the file nginxplus.auto.tfvars and populate connection_notifications and email_notifications as per below examples.
Pagerduty Connection Example
connection_notifications = [
    {
      connection_type       = "PagerDuty",
      connection_id         = "<CONNECTION_ID>",
      payload_override      = "{\"service_key\": \"your_pagerduty_api_integration_key\",\"event_type\": \"trigger\",\"description\": \"Alert: Triggered {{TriggerType}} for Monitor {{Name}}\",\"client\": \"Sumo Logic\",\"client_url\": \"{{QueryUrl}}\"}",
      run_for_trigger_types = ["Critical", "ResolvedCritical"]
    },
    {
      connection_type       = "Webhook",
      connection_id         = "<CONNECTION_ID>",
      payload_override      = "",
      run_for_trigger_types = ["Critical", "ResolvedCritical"]
    }
  ]

Replace <CONNECTION_ID> with the connection id of the webhook connection. The webhook connection id can be retrieved via calling the Monitors API.

For overriding payload for different connection types, refer to this document.

Email Notifications Example
email_notifications = [
    {
      connection_type       = "Email",
      recipients            = ["abc@example.com"],
      subject               = "Monitor Alert: {{TriggerType}} on {{Name}}",
      time_zone             = "PST",
      message_body          = "Triggered {{TriggerType}} Alert on {{Name}}: {{QueryURL}}",
      run_for_trigger_types = ["Critical", "ResolvedCritical"]
    }
  ]
}
  1. Install the Alerts:
    1. Navigate to the package directory terraform-sumologic-sumo-logic-monitor/monitor_packages/nginx-plus/ and run terraform init. This will initialize Terraform and will download the required components.
    2. Run terraform plan to view the monitors resources which will be created/modified by Terraform.
    3. Run terraform apply.
  2. Post Installation steps: If you haven’t enabled alerts and/or configured notifications via the terraform procedure outlined above, we highly recommend enabling alerts of interest and configuring each enabled alert to send notifications to other people or services. This is detailed in Step 4.

Note: There are limits to how many alerts can be enabled - please see the Alerts FAQ.

Installing the Ngnix Plus App​

This section has instructions for installing the Sumo App for Nginx Plus. The instructions assume you have already set up the collection as described above.

Locate and install the app you need from the App Catalog. If you want to see a preview of the dashboards included with the app before installing, click Preview Dashboards.

  1. From the App Catalog, search for and select the app.
  2. Select the version of the service you're using and click Add to Library. Version selection is applicable only to a few apps currently. For more information, see the Install the Apps from the Library.
  3. To install the app, complete the following fields.
    1. App Name. You can retain the existing name, or enter a name of your choice for the app.

    2. Data Source. Select either of these options for the data source.

      • Choose Source Category, and select a source category from the list.

      • Choose Enter a Custom Data Filter, and enter a custom source category beginning with an underscore. Example: (_sourceCategory=MyCategory).

    3. Advanced. Select the Location in Library (the default is the Personal folder in the library), or click New Folder to add a new folder.
  4. Click Add to Library.

Once an app is installed, it will appear in your Personal folder, or other folder that you specified. From here, you can share it with your organization.

Panels will start to fill automatically. It's important to note that each panel slowly fills with data matching the time range query and received since the panel was created. Results won't immediately be available, but with a bit of time, you'll see full graphs and maps.

Nginx Plus Alerts​

Sumo Logic has provided out-of-the-box alerts available via Sumo Logic monitors to help you quickly determine if the Nginx Plus server is available and performing as expected. These alerts are built based on logs and metrics datasets and have preset thresholds based on industry best practices and recommendations. They are as follows:

NameDescriptionAlert ConditionRecover Condition
Nginx Plus - Dropped ConnectionsThis alert fires when we detect dropped connections for a given Nginx Plus server. > 0 < = 0
Nginx Plus - Critical Error MessagesThis alert fires when we detect critical error messages for a given Nginx Plus server. > 0 < = 0
Nginx Plus - Access from Highly Malicious SourcesThis alert fires when an Nginx Plus is accessed from highly malicious IP addresses. > 0 < = 0
Nginx Plus - High Client (HTTP 4xx) Error RateThis alert fires when there are too many HTTP requests (>5%) with a response status of 4xx. > 0 < = 0
Nginx Plus - High Server (HTTP 5xx) Error RateThis alert fires when there are too many HTTP requests (>5%) with a response status of 5xx. > 0 < = 0

Viewing Nginx Plus Dashboards​

Filter with template variables

Template variables provide dynamic dashboards that can rescope data on the fly. As you apply variables to troubleshoot through your dashboard, you view dynamic changes to the data for a quicker resolution to the root cause. You can use template variables to drill down and examine the data on a granular level. For more information, see Filter with template variables.

Overview​

The Nginx Plus - Overview dashboard provides an at-a-glance view of the Nginx Plus server access locations, error logs along with connection metrics.

Use this dashboard to:

  • Gain insights into originated traffic location by region. This can help you allocate computer resources to different regions according to their needs.
  • Gain insights into your Nginx health using Critical Errors and Status of Nginx Server.
  • Get insights into Active and dropped connections.
Nginx-Plus-Overview

Error Logs Analysis​

The Nginx Plus - Error Logs Analysis dashboard provides a high-level view of log level breakdowns, comparisons, and trends. The panels also show the geographic locations of clients and clients with critical messages, new connections and outliers, client requests, request trends, and request outliers.

Use this dashboard to:

  • Track requests from clients. A request is a message asking for a resource, such as a page or an image.
  • Track and view client geographic locations generating errors.
  • Track critical alerts and emergency error alerts.
Nginx-Plus-Error-Logs-Analysis

Logs Timeline Analysis​

The Nginx Plus - Logs Timeline Analysis dashboard provides a high-level view of the activity and health of Nginx servers on your network. Dashboard panels display visual graphs and detailed information on traffic volume and distribution, responses over time, as well as time comparisons for visitor locations and server hits.

Use this dashboard to:

  • Understand the traffic distribution across servers, provide insights for resource planning by analyzing data volume and bytes served.
  • Gain insights into originated traffic location by region. This can help you allocate compute resources to different regions according to their needs.
tk

Outlier Analysis​

The Nginx Plus - Outlier Analysis dashboard provides a high-level view of Nginx server outlier metrics for bytes served, number of visitors, and server errors. You can select the time interval over which outliers are aggregated, then hover the cursor over the graph to display detailed information for that point in time.

Use this dashboard to:

  • Detects outliers in your infrastructure with Sumo Logic’s machine learning algorithm.
  • To identify outliers in incoming traffic and the number of errors encountered by your servers.

You can use schedule searches to send alerts to yourself whenever there is an outlier detected by Sumo Logic.

tk

Threat Intel​

The Nginx Plus - Threat Intel dashboard provides an at-a-glance view of threats to Nginx servers on your network. Dashboard panels display the threat count over a selected time period, geographic locations where threats occurred, source breakdown, actors responsible for threats, severity, and a correlation of IP addresses, method, and status code of threats.

Use this dashboard to:

  • To gain insights and understand threats in incoming traffic and discover potential IOCs. Incoming traffic requests are analyzed using the Sumo - Crowdstrikes threat feed.
tk

Web Server Operations​

The Nginx Plus - Web Server Operations dashboard provides a high-level view combined with detailed information on the top ten bots, geographic locations, and data for clients with high error rates, server errors over time, and non 200 response code status codes. Dashboard panels also show information on server error logs, error log levels, error responses by a server, and the top URIs responsible for 404 responses.

Use this dashboard to:

  • Gain insights into Client, Server Responses on Nginx Server. This helps you identify errors in Nginx Server.
  • To identify geo locations of all Client errors. This helps you identify client location causing errors and helps you to block client IPs.
tk

Visitor Access Types​

The Nginx Plus - Visitor Access Types dashboard provides insights into visitor platform types, browsers, and operating systems, as well as the most popular mobile devices, PC and Mac versions used.

Use this dashboard to:

  • Understand which platform and browsers are used to gain access to your infrastructure. These insights can be useful for planning in which browsers, platforms, and operating systems (OS) should be supported by different software services.
tk

Visitor Locations​

The Nginx Plus - Visitor Locations dashboard provides a high-level view of Nginx visitor geographic locations both worldwide and in the United States. Dashboard panels also show graphic trends for visits by country over time and visits by US region over time.

Use this dashboard to:

  • Gain insights into geographic locations of your user base. This is useful for resource planning in different regions across the globe.
tk

Visitor Traffic Insight​

The Nginx Plus - Visitor Traffic Insight dashboard provides detailed information on the top documents accessed, top referrers, top search terms from popular search engines, and the media types served.

Use this dashboard to:

  • To understand the type of content that is frequently requested by users.
  • It helps in allocating IT resources according to the content types.
tk

Caches​

The Nginx Plus - Caches metrics dashboard provides insight into cache states, cache hit rate and cache disk usage over time.

Use this dashboard to:

  • Gain information about the number of caches used, how many of them are in active (hot) state and what is the hit rate of the cache.
  • Gain information about how much disk space is used for cache.
tk

HTTP Location Zones​

The Nginx Plus - HTTP Location Zones metrics dashboard provides detailed statistics on the frontend performance, showing traffic speed, responses/requests count and various error responses.

Use this dashboard to:

  • Gain information about Location http zones traffic: received and sent; speed, requires/responses amount, discarded traffic.
  • Gain information about Location http zones error responses: percentage of responses by server, percentage of each type of error responses.
tk

HTTP Server Zones​

The Nginx Plus - HTTP Server Zones metrics dashboard provides detailed statistics on the frontend performance, showing traffic speed, responses/requests count and various error responses.

Use this dashboard to:

  • Gain information about Server http zones traffic: received and sent; speed, requires/responses amount, discarded traffic.
  • Gain information about Server http zones error responses: percentage of responses by server, percentage of each type of error responses.
tk

HTTP Upstreams​

The Nginx Plus - HTTP Upstreams metrics dashboard provides information about each upstream group for HTTP and HTTPS traffic, showing number of HTTP upstreams, servers, back-up servers, error responses and health monitoring.

Use this dashboard to:

  • Gain information about HTTP upstreams, servers and back-up servers.
  • Gain information about HTTP upstreams traffic: received and sent; speed, requires/responses amount, downtime and response time.
  • Gain information about HTTP upstreams error responses: percentage of responses by server, percentage of each type of error responses.
  • Gain information about HTTP upstreams health monitoring.
tk

Resolvers​

The Nginx Plus - Resolvers metrics dashboard provides DNS server statistics of requests and responses per each DNS status zone.

Use this dashboard to:

  • Gain information about the total number of zones, responses and requests speed.
  • Gain information about error responses by each type of error.
tk

TCP/UDP Upstreams​

The Nginx Plus - TCP/UDP Upstreams metrics dashboard provides information about each upstream group for TCP and UDP traffic, showing number of TCP and UDP upstreams, servers, back-up servers, error responses and health monitoring.

Use this dashboard to:

  • Gain information about TCP and UDP upstreams, servers and back-up servers.
  • Gain information about TCP and UDP upstreams traffic: received and sent; speed, requests/responses amount, downtime and response time.
  • Gain information about TCP and UDP upstreams error responses: percentage of responses by server, percentage of each type of error responses.
  • Gain information about TCP and UDP upstreams health monitoring.
tk

TCP/UDP Zones​

The Nginx Plus - TCP/UDP Zones metrics dashboard provides TCP and UDP status zones with charts for connection limiting.

Use this dashboard to:

  • Gain information about TCP and UDP traffic: received and sent; speed, requires/responses amount, discarded traffic.
  • Gain information about TCP and UDP error responses: percentage of responses by server, percentage of each type of error responses.
tk
Legal
Privacy Statement
Terms of Use

Copyright © 2023 by Sumo Logic, Inc.