Squid Proxy - OpenTelemetry Collector

The Squid Proxy app is a logs app that helps you monitor activity in Squid Proxy. The preconfigured dashboards provide insight into served and denied requests; HTTP response codes; URLs experiencing redirects, client errors, and server errors; and quality of service data that helps you understand your users' experience.

Squid logs are sent to Sumo Logic through OpenTtelemetry filelog receiver.

Prerequisites

By default, the squid proxy will write the access log to the log directory that was configured during installation. For example, on Linux, the default log directory is /var/log/squid/access.log. If the access log is disabled then you must enable the access log by following these instructions.

Fields creation in Sumo Logic for Squid

Following are the Fields which will be created as part of Squid App install if not already present:

• webengine.cluster.name. User configured. Enter a name to identify this Squid cluster. This cluster name will be shown in the Sumo Logic dashboards.
• webengine.system. Has a fixed value of squidproxy.
• sumo.datasource. Has a fixed value of squidproxy.

Collection Configuration and App installation

As part of data collection setup and app installation, you can select the App from App Catalog and click on Install App. Follow the steps below.

Step 1: Set up Collector

note

If you want to use an existing OpenTelemetry Collector, you can skip this step by selecting the Use an existing Collector option.

To create a new Collector:

1. Select the Add a new Collector option.
2. Select the platform where you want to install the Sumo Logic OpenTelemetry Collector.

This will generate a command that you can execute in the machine environment you need to monitor. Once executed, it will install the Sumo Logic OpenTelemetry Collector.

Step 2: Configure integration

OpenTelemetry works with a configuration yaml file which has all the details with respect to the data which needs to be collected. For example, it specifies the location of a log file that is read and sent to the Sumo Logic platform..

In this step, you will configure the yaml required for Squid Collection. The path of the log file configured to capture Squid logs is needed to be given here.

The files are typically located in /var/log/squid/access.log. Refer to the Prerequisites section for more details.

You can add any custom fields which you want to tag along with the data ingested in Sumo. Click on the Download YAML File button to get the yaml file.

Step 3: Send logs to Sumo

Once you have downloaded the yaml file as described in the previous step, follow the below steps based on your platform.

Linux

1. Copy the yaml at /etc/otelcol-sumo/conf.d/ folder in the Squid instance which needs to be monitored.
2. Restart the otelcol-sumo process using the below command:

   sudo systemctl restart otelcol-sumo

macOS

1. Copy the yaml at /etc/otelcol-sumo/conf.d/ folder in the Squid instance which needs to be monitored.
2. Restart the otelcol-sumo process using the below command:

     otelcol-sumo --config /etc/otelcol-sumo/sumologic.yaml --conf "glob:/etc/otelcol-sumo/conf.d/*.yaml"

After successfully executing the above command, Sumo Logic will start receiving data from your host machine.

Click Next. This will install the app (dashboards and monitors) to your Sumo Logic Org.

Dashboard panels will start to fill automatically. It's important to note that each panel fills with data matching the time range query and received since the panel was created. Results won't immediately be available, but within 20 minutes, you'll see full graphs and maps.

Sample Log Messages in Non-Kubernetes environments

1674483805.439 44 192.168.100.40 TCP_REFRESH_MODIFIED/301 514 GET http://openstack.org/ - HIER_DIRECT/192.168.100.40 text/html

Sample Query

This sample Query is from the Squid Proxy - HTTP Response Analysis > URLs Experiencing Redirections panel.

Query String

%"sumo.datasource"=squidproxy %"webengine.cluster.name"=*  host.name=* %"webengine.system"=squidproxy
| json "log" as message nodrop 
| if (isEMpty(message), _raw, message) as message
| parse regex field = message "(?<time>[\d.]+)\s+(?<elapsed>[\d]+)\s+(?<remotehost>[^\s]+)\s+(?<action>[^/]+)/(?<status_code>[\d]+)\s+(?<bytes>[\d]+)\s+(?<method>[^\s]+)\s+(?<url>[^\s]+)\s(?<rfc931>[^\s]+)\s+(?<peerstatus>[^/]+)/(?<peerhost>[^\s]+)\s+(?<type>[^\s|$]+?)(?:\s+|$)" nodrop | parse field = message "Connection: *\\r\\n" as connection_status nodrop | parse field = message "Host: *\\r\\n" as host nodrop | parse field = message "User-Agent: *\\r\\n" as user_agent nodrop | parse field = message "TE: *\\r\\n" as te nodrop
| where status_code matches "3*"
| count as eventCount by url, status_Code
| sort by status_code, eventCount, url

Viewing Squid Dashboards

Squid Proxy - Overview

The Squid Proxy - Overview Dashboard provides an at-a-glance view of the activity and health of the SquidProxy clusters and servers by destination locations, error and denied requests, URLs accessed.

Squid Proxy - Activity Trend

Squid Proxy - Activity Trend dashboard provides trends around denied request trend, action trend, time spent to serve, success and non-success response, remote hosts.

Squid Proxy - HTTP Response Analysis

The Squid Proxy - HTTP Response Analysis dashboard provides insights into HTTP response, HTTP code, the number of client errors, server errors, redirections outlier, URLs experiencing server errors.

Squid Proxy - Quality of Service

The Squid Proxy - Quality of Service dashboard provides insights into latency, the response time of requests according to HTTP action, and the response time according to location.