NewRelic Monitoring Setup

When it comes to monitoring there are variety of solutions provided by New Relic to be considered:

Infrastructure monitoring

Infrastructure monitoring is used to collect health and performance data from servers, virtual machines, containers, databases, and other backend components in a tech stack.

There are also numerous on-host and cloud integrations available.

Application Performance Monitoring (APM)

APM provides monitoring of your web or non-web application's performance. It supports apps using several programming languages. APM agents give you real-time observability matched with trending data about your application's performance and the user experience. Agents reveal what is happening deep in your code with end to end transaction tracing and a variety of color-coded charts and reports.

Synthetics

New Relic's synthetic monitoring simulates user traffic around the world so you can detect and resolve poor performance and outages. Use the suite of automated, scriptable tools to monitor your external and internal websites, critical business transactions, and API endpoints.

Browser monitoring

Many websites contain dynamic content that is loaded after the initial page has finished loading, and complex JavaScript code increases the need for error reporting. Browser monitoring provides a Real User Monitoring (RUM) solution that measures the speed and performance of your end users as they navigate to your site from different web browsers, devices, operating systems, and networks.

Logging

New Relic offers a fast, scalable log management platform so you can connect your logs with the rest of your telemetry and infrastructure data in a single place. With the log management solution, you have direct control over what data is reported to New Relic. To ensure data privacy, and to limit the types of information New Relic receives, no customer data is captured except what you supply in API calls or log forwarder configuration. All data for the logs service is then reported to New Relic over HTTPS.

Alerts

Alerts is a single tool to manage alert policies and alert conditions, letting you focus on the metrics you care about most.

Cloudaware provides automation for different New Relic monitoring activities.

Next follows the instructions on using the aforementioned solutions:

Infrastructure Monitoring

In order to setup Infrastructure Monitoring for EC2 instances of an AWS account, please create a ticket and provide AWS Account ID and the New Relic Account Name you wish to map it to. We will configure Breeze so that all agents which belong to the given account automatically install Breeze monitoring plugin and ingest metrics to the appropriate New Relic account.

Java APM

A preconfigured Java APM Agent is being shipped to each instance with Breeze Agent installed. The agent location is /opt/cloudaware

In order to start using Java APM Agent, please include “-javaagent:/opt/cloudaware/newrelic/newrelic.jar“ in your Java application command line options.

Example:

java -javaagent:/opt/cloudaware/newrelic/newrelic.jar -Dnewrelic.config.app_name=preferencedynamo-dev -jar /nyl/eis/ms/app/dynamodb-stream-2-kafka-1.0.64.RELEASE.jar --spring.profiles.active=dev --dynamodb.sourceTable=eis-nonprod-dev-preference --kafka.targetTopic=com.newyorklife.edm.preference.management

Whenever Cloudaware ships a new version of the APM agent to an agent, please make sure to restart the application.

.Net APM

Add the following tag to the instance with the .Net application hosted in IIS:

dot_net_apm_enabled = true

2. Allow 30-60 minutes for Breeze automation to complete the job. After that navigate to Control Panel > Programs > Programs and Features and confirm that New Relic .NET Agent is installed.

3. Restart IIS. This can be done via a command line:

3.1. From the Start menu, click Run.

3.2. In the Open box, type cmd, and click OK.

3.3. At the command prompt, type: iisreset /noforce

3.4. IIS attempts to stop all services before restarting. The IISReset command-line utility waits up to one minute for all services to stop. If the services cannot be stopped within one minute, all IIS services are terminated, and IIS restarts.

4. Generate some traffic to the hosted application, e.g. navigate to the website URL in the browser.

5. Allow about 5 minutes for APM data to be pushed to New Relic.

Alerts and Synthetic Monitoring

In order to create alerts or synthetic monitors, please refer to the New Relic official documentation:

https://docs.New Relic.com/docs/alerts-applied-intelligence?toc=true

https://docs.New Relic.com/docs/synthetics?toc=true

Note: When creating alerts do not forget to update the Notification channel to include Pagerduty integration.

Kubernetes Monitoring

In order to deploy New Relic monitoring to a kubernetes cluster using Breeze, you need to configure kubectl on one of the EC2 instances with Breeze agent installed (a chosen instance should have permissions to access and deploy to a desired cluster) and apply the following tags to the instance:

k8s_cluster_name
kube_config_path

Alternatively, you may follow the instructions on manual installation.

Install kube-state-metrics and get it running on the cluster. For example:

curl -L -o kube-state-metrics-1.9.5.zip https://github.com/kubernetes/kube-state-metrics/archive/v1.9.5.zip

unzip kube-state-metrics-1.9.5.zip

kubectl apply -f kube-state-metrics-1.9.5/examples/standard

2. Download the manifest file:

curl -O https://download.newrelic.com/infrastructure_agent/integrations/kubernetes/newrelic-infrastructure-k8s-latest.yaml

3. In the manifest, add your New Relic license key and a cluster name to identify your Kubernetes cluster. Both values are required.

4. Confirm that kube-state-metrics is installed:

kubectl get pods --all-namespaces | grep kube-state-metrics

5. Create the DaemonSet:

kubectl create -f New Relic-infrastructure-k8s-latest.yaml

6. Confirm that the DaemonSet has been created successfully by looking for New Relic-infra in the results generated by this command:

kubectl get daemonsets

Steps to install Kubernetes plugin for Logs:

git clone https://github.com/newrelic/kubernetes-logging
edit new-relic-fluent-plugin.yml, replace the placeholder value <LICENSE_KEY > with your New Relic license key.
Run: kubectl apply -f .

Optional: Configure parsing: https://docs.fluentbit.io/manual/

Steps to install Kubernetes events integration:

Download the manifest file: curl -O https://download.New Relic.com/infrastructure_agent/integrations/kubernetes/nri-kube-events-latest.yaml
Edit the nri-kube-events-latest.yaml manifest file, adding both a cluster name, to identify your Kubernetes cluster (required), and your New Relic license key (required).
Run: kubectl apply -f nri-kube-events-latest.yaml

To confirm that the integration has been configured correctly, wait a few minutes, then go to one.New Relic.com > Chart builder, and run the following NRQL query to see if data has been reported:

FROM InfrastructureEvent SELECT count(*) WHERE clusterName = 'YOUR_CLUSTER_NAME' since 1 hour ago

APM for Kubernetes

Make sure to check the requirements;
You containerized application should contain APM agent;
Download the following yaml file:
curl -O http://download.newrelic.com/infrastructure_agent/integrations/kubernetes/k8s-metadata-injection-latest.yaml
Edit this file, replacing <YOUR_CLUSTER_NAME> with the name of your cluster and apply the yaml file to your Kubernetes cluster:
kubectl apply -f k8s-metadata-injection-latest.yaml

Alternatively, you can install metadata injector using Helm. It is being shipped as a part of nri-bundle chart. For APM to work it is enough to install the injector webhook only, however, you are free to stick to the default chart configuration as well. Remember to replace the placeholders.
Option 1: Install injector webhook only:
helm upgrade --install newrelic newrelic/nri-bundle
--namespace newrelic --create-namespace
--set global.licenseKey=<YOUR_NEW_RELIC_LICENSE_KEY>
--set global.cluster=<CLUSTER_NAME>
--set ksm.enabled=false
--set newrelic-infrastructure.privileged=false
--set infrastructure.enabled=false
--set prometheus.enabled=false
--set webhook.enabled=true
--set kubeEvents.enabled=false
--set logging.enabled=false
Option 2: Default configuration:
helm upgrade --install newrelic newrelic/nri-bundle
--namespace newrelic --create-namespace
--set global.licenseKey=<YOUR_NEW_RELIC_LICENSE_KEY>
--set global.cluster=<CLUSTER_NAME> --set ksm.enabled=true \ --set newrelic-infrastructure.privileged=true \ --set infrastructure.enabled=true \ --set prometheus.enabled=true \ --set webhook.enabled=true \ --set kubeEvents.enabled=true \ --set logging.enabled=true
Each application pod/deployment which contains APM agent should be restarted for the New Relic environment variables to be injected by the webhook. After restarting the application you can verify that the webhook (responsible for injecting the metadata) was installed correctly by running the command:
kubectl exec <pod_name> -- env | grep NEW_RELIC_METADATA_KUBERNETES
You should see NEW_RELIC_METADATA_KUBERNETES_* environmental variables in the output;
You can optionally limit the injection of metadata to specific namespaces in your cluster, or configure the metadata injection to work with custom certificates, if you’re using them. For more information, including steps for validating and troubleshooting metadata injection, see the documentation. After deploying the metadata injection file to our Kubernetes cluster, custom parameters from Kubernetes begin to appear in the New Relic UI.

Browser Monitoring

Browser monitoring provides a Real User Monitoring (RUM) solution that measures the speed and performance of your end users as they navigate to your site from different web browsers, devices, operating systems, and networks. Instrumentation is compatible with most SPA frameworks, such as Angular, Backbone, Ember, and React

Steps to enable Browser Monitoring: