APM logs in context connects your logs with all of your telemetry data for your apps, hosts, and other entities. Bringing all of this data together in a single tool helps you quickly:
Cut through the noise of thousands of logs when troubleshooting time-critical issues, so you automatically see only the most relevant logs.
Navigate within multiple types of telemetry data, and have the data correlate back to the original issue.
Easily drill down into more detailed information from the same place in the UI.
Find the log lines that you need to identify and resolve a problem.
For more information, including examples, learn how to get started with APM logs in context.
Automatic logs in context options
You have three options to configure APM logs in context to send your app's logs and linking metadata automatically to New Relic.
This is the simplest approach, and it's a great choice for developers who may not have the access or interest in setting up a log forwarder, or for accounts that want to see the power of logs and other linking metadata in context of their apps, without a lot of overhead.
Using this option, your logs will include span.id, trace.id, hostname, entity.guid, and entity.name. These attributes automatically link your logs to spans, traces, and other telemetry, making it easier to troubleshoot.
All you need to do is install an agent version with log forwarding capabilities (.NET agent 9.7.0 or higher), and update your configuration to enable forwarding:
Note: The environment variable value, if present, takes precedence over the config file value.
Beginning with agent version 9.8.0, this feature is enabled by default.
Optional adjustments:
Once this is enabled, you also have control over the maximum number of logs sent to New Relic every minute. The default value is 10,000. If more than 10,000 logs are received in a 60-second window, your logs will begin to be sampled.
Set it to a higher number to receive more logs. Set it to a lower number to receive fewer logs. Any integer is valid.
Note: The environment variable value, if present, takes precedence over the config file value.
If you have an existing log forwarding solution and are updating your agent to use automatic logs in context, be sure to disable your manual log forwarder. Otherwise, your app will be sending double log lines. Depending on your account, this could result in double billing. For more information, follow the procedures to disable your specific log forwarder.
Already have a log forwarder you like? We've got you covered! Language agents can decorate your logs with the linking metadata needed to provide access to automatic logs-in-context features.
This option should not be used with in-agent forwarding. Using an external log forwarder to send logs to New Relic while in-agent forwarding is enabled will cause your logs to be sent up twice to New Relic. Depending on your account, this may result in double billing.
Important
Local log decoration for the .NET agent does not directly alter log messages. Your logging framework configuration will need to be updated to write out the NR-LINKING token in messages.
If you want to use this option, make sure you have the in-agent forwarding configuration option disabled.
Enable log decorating in your configuration, then relaunch the agent to start decorating your logs.
Note: The environment variable value, if present, takes precedence over the config file value.
Our decorator adds five attributes to every log message: entity.guid, entity.name, hostname, trace.id, and span.id. Example:
This is my log message. NR-LINKING|{entity.guid}|{hostname}|{trace.id}|{span.id}|{entity.name}
Some attributes may be empty if the log occurred outside a transaction or if they are not applicable to your application's context. We recommend this option over the manual process to use a decorating formatter.
Important
The key used to read out the NR-LINKING token from a logging framework is NR_LINKING not NR-LINKING in order to support logging frameworks that do not allow hyphens in key names.
log4net configuration
For log4net, the .NET agent stores the NR-LINKING token as a property called NR_LINKING in the log event object. The following common layout examples demonstrate how to configure log4net to write out out the NR-LINKING token. Other layouts are supported, but not listed below. SimpleLayout is not supported since it cannot be altered.
PatternLayout/DynamicPatternLayout:
Update the conversionPattern to include the NR_LINKING property at the end of the line.
log4net.Appender.AdoNetAppender:
This appender has a slightly different configuration than other layouts. You will need to update the "message" parameter to include the metadata. The following example demonstrates this using a PatternLayout.
For Serilog, the .NET agent stores the NR-LINKING token as a property called NR_LINKING in the log event object. The following common formatter and sink examples demonstrate how to configure Serilog to write out out the NR-LINKING token. Other sinks and formatters are supported, but not listed below.
Plain text using an outputTemplate
Update the outputTemplate to include the NR_LINKING property at the end of the line.
For Microsoft.Extensions.Logging, the .NET agent stores the NR-LINKING token with the message in a newly created a Scope. Since Microsoft.Extensions.Logging relies on other frameworks for output, please see either the log4net or Serilog configuration above for details.
NLog configuration
For NLog, the .NET agent is able to automatically append the NR-LINKING token directly to the end of the message so no additional configuration is required.
Before language agents had the ability to forward and decorate logs, you could enable logs in context by updating your application to send linking metadata.
This option is still supported, but is no longer encouraged. For instructions to use this approach, see Manual logs in context option.
Also, this method requires that you install a log forwarder before enabling logs in context. If you do not have a log forwarder, the New Relic UI will prompt you to use our infrastructure agent.
If you decide to use your existing log forwarding solution and later decide to update your agent to use automatic logs in context, be sure to disable your manual log forwarder. Otherwise, your app will be sending double log lines. Depending on your account, this could result in double billing. For more information, follow the procedures to disable your specific log forwarder.
Secure your data
Your logs may include sensitive information protected by HIPAA or other compliance protocols. By default we obfuscate number patterns that appear to be for items such as credit cards or Social Security numbers, but you may need to hash or mask additional information.
For more information, see our documentation about obfuscation expressions and rules. You can hash or mask your log data by using the New Relic UI or by using NerdGraph, our GraphQL API.
Explore your data
To make the most of your logging data:
On the APM Summary page, click your Web transaction time chart to view logs associated with a specific point in time.
Check your app's Errors inbox to view the logs associated with your errors.
Typically your logs will start to appear less than a minute after you enable APM logs in context. Check your app's Triage > Logs section. You will also start seeing log patterns related to the error there.
If you don't see any logs for errors or traces, there may not be any for your app. Try refreshing the UI page, or change the selected time period.
Disable automatic logging
APM logs in context automatically forwards APM agent log data and is enabled by default. This can have a negative impact on your security, compliance, billing, or system performance. For more information, or if you need to adjust the default setting, follow the procedures to disable automatic logging.
Manual logs in context option
If you need to use the manual process to set up logs in context for .NET, follow these steps:
Make sure you have already set up logging in New Relic. This includes configuring a supported log forwarder that collects your application logs and extends the metadata that is forwarded to New Relic.
The following diagram illustrates the flow of log messages through Apache log4net, highlighting specific components of the New Relic log4net extension. Many log forwarders are available. This example uses Fluentd.
Appender: The NewRelicAppender adds contextual information from the .NET agent (using the API) to the log events generated by the application. This contextual information, known as linking metadata, is used by New Relic to link log messages to the transactions and spans from which they were created. This appender will pass the enriched log events to downstream appenders for further processing.
Since the NewRelicAppender is ForwardingAppender type, it needs to be the first appender in the chain. It also requires another appender that can write to an actual output destination as its child in order to work.
Layout: The NewRelicLayout formats the enriched log events into the JSON format expected by New Relic. The appender, which this layout is assigned to, instructs log4net to output the JSON to a file in the location that the log forwarder expects.
Log Forwarder: The log forwarder monitors an output folder and incrementally sends the properly formatted and enriched log information to the New Relic logging endpoint.
Log4net uses appender and layout to store and format log messages. NewRelicAppender enriches log messages with contextual information from the New Relic .NET agent if it is attached to your application. The appender passes enriched log messages to downstream appenders to handle specific use cases for log messages.
For more information about logging with log4net, see the Apache log4net Getting started documentation.
To configure logs in context with the log4net extension:
In your log4net configuration file, update your logging configuration to use the NewRelicAppender as the first level appender, and reference your existing appenders as its children. Also replace the layout of the appender that writes log messages to an output destination with the NewRelicLayout.
The following log4net configuration example enriches log events with New Relic linking metadata. In addition to the existing log files, it outputs new log files in a specific JSON format at C:\logs\log4netExample.log.json for consumption by the log forwarder:
After you configure the log4net extension and update your logging file, you can configure your extension to send data to New Relic. There are several options for forwarding your logs. Here is an example configuration using the infrastructure agent for logs in context:
logs:
-name: application-log
file: C:\logs\log4netExample.log.json # Path to a single log file
You can use our NLog 4.5 or higher extension to link to your log data with related data across the rest of the New Relic platform.
The New Relic NLog extension provides a NewRelicJsonLayout that formats a log event in the way required by the New Relic logging endpoint. Next, it adds contextual information from the .NET agent when attached to your application. Then, a target can be configured to write logging data to an output folder. The log forwarder can monitor this folder and incrementally send log information to New Relic.
The following diagram illustrates the flow of log messages through NLog, highlighting specific components of the New Relic NLog extension.
New Relic JSON Layout: The NewRelicJsonLayout adds contextual information from the .NET agent (using the API) to the log events generated by the application, and outputs log messages in the JSON format expected by New Relic. This contextual information, known as linking metadata, is used by New Relic to link log messages to the transactions and spans where they were created.
Since the NewRelicAppender is ForwardingAppender type, it needs to be the first appender in the chain. It also requires another appender that can write to an actual output destination as its child in order to work.
File Target: A FileTarget defines a file on disk where log messages are written. Adding the NewRelicJsonLayout to that target allows the output to be formatted correctly for forwarding to New Relic.
Log Forwarder: The log forwarder is configured to send the properly formatted and enriched log data from the FileTarget output to the New Relic logging endpoint.
In your application code, update your logging configuration to add the NewRelicJsonLayout and decide if you want to collect MappedDiagnosticsContext (MDC) or the MappedDiagnosticsLogicalContext (MDLC) data.
The following configuration examples result in new JSON files that are written to disk. Some of these configuration options may be useful for managing the amount of disk space used and/or the performance of the target.
archiveAboveSize
maxArchiveFiles
bufferSize
enableArchiveFileCompression
autoFlush
concurrentWrites
Although the NLog AsyncWrapper Target is not required, it may help improve performance by performing formatting and output of log files on a different thread.
Don't collect MDC or the MDLC data (default):
The following code example enriches log events with New Relic linking metadata, but not with MDC or the MDLC data. In addition to the existing log files, it outputs new log files in a specific JSON format at C:\logs\NLogExample.log.json for consumption by the log forwarder:
var loggerConfig =newLoggingConfiguration();
var newRelicFileTarget =newFileTarget("NewRelicFileTarget");
If your application uses the MDC or the MDLC, you can configure the NewRelicJsonLayout to include items in those collections. The following code example adds the additional configuration to enable collecting MDC and MDLC data. As in the previous example, it outputs new log files in a specific JSON format at C:\logs\NLogExample.log.json for consumption by the log forwarder:
var loggerConfig =newLoggingConfiguration();
var newRelicFileTarget =newFileTarget("NewRelicFileTarget");
Once you have configured the NLog extension and updated your logging file, you can configure your extension to send data to New Relic. There are several options for forwarding your logs. Here is an example configuration using the infrastructure agent for logs in context:
logs:
-name: application-log
file: C:\logs\log4netExample.log.json # Path to a single log file
You can also configure the New Relic NLog extension with file-based configuration providers. The folowing example code creates a logger based on settings contained in an App.config file.
Serilog is a structured logging framework that records log messages from your application and creates a LogEvent to store the message data. Using Enrichers, you can add additional information to the log events. Sinks and Formatters allow you to format and output those log events for downstream consumption and viewing.
The following diagram illustrates the flow of log messages through Serilog, highlighting specific components of the New Relic Serilog extension. Many log forwarders are available. This example uses Fluentd.
New Relic Enricher: The NewRelicEnricher adds contextual information from the .NET agent (using the API) to the log events generated by the application. This contextual information, called linking metadata, is used by New Relic to link log messages to the transactions and spans where they were created.
New Relic Formatter: The NewRelicFormatter translates enriched log events into the JSON format expected by New Relic. A sink instructs Serilog to output the JSON to a file in the location that the log forwarder expects.
New Relic Log Forwarder: The log forwarder is configured to send the properly formatted and enriched log data from the FileTarget output to the New Relic logging endpoint.
In your application code, update your logging configuration to add the NewRelicEnricher and NewRelicFormatter.
The following code example enriches log events with New Relic linking metadata. In addition to the existing log files, it outputs new log files in a specific JSON format at C:\logs\SerilogExample.log.json for consumption by the log forwarder:
This configuration results in new JSON files that are written to disk. Some of these configuration options may be useful for managing the amount of disk space used and/or the performance of the sink.
restrictedToMinimumLevel
buffered
rollingInterval
rollOnFileSizeLimit
retainedFileCountLimit
Although not required, using the Serilog Asynchronous Sink Wrapper may help improve the performance by performing formatting and output of log files on a different thread.
Once you have configured the Serilog extension and updated your logging file, there are several options for forwarding your logs. Here is an example configuration using the infrastructure agent for logs in context:
logs:
-name: application-log
file: C:\logs\log4netExample.log.json # Path to a single log file
The following example code creates a logger based on settings contained in a web.config file. The Serilog.Settings.AppSettings NuGet Package is required.