Sometimes it is very hard to track all logs or the next or previous ones that are related to a specific log in distributed microservice.
We are going to use Mapped Diagnostic Context(MDC). if you know nothing about MDC, don’t worry learn this and come back.
Here we go
First thing first let's learn a bit about API request flow.Request -> App/Web server -> filter -> Servlet -> Interceptor -> Controller or Url
We can add a unique request id in :
1. Filter
2. Servlet
3. Interceptor
I have chosen Interceptor to add the unique request id because the filter should be responsible for only filtering the request (SOLID Principle).
You can add anywhere you want.
Steps 1: Create an Interceptor from there we can add a unique request id.
@Component
public class MdcInterceptor implements HandlerInterceptor {
@Override
public boolean preHandle(HttpServletRequest request, HttpServletResponse response, Object handler) {
MDC.put("requestId", StringUtils.getRandomString.apply(null));
return true;
}
@Override
public void afterCompletion(HttpServletRequest request, HttpServletResponse response, Object handler, Exception ex) {
MDC.remove("requestId");
}
}
In preHandle
simply adding uniqueness requestId
to MDC.
In afterCompletion
removing the added requestId
from MDC.
Steps 2: Register the MdcInterceptor
This step is helpful because we are telling spring to consider MdcInterceptor
as interceptor and execute preHandle
on request arrival and execute afterCompletion
on process completion.
@Component
@AllArgsConstructor
public class WebMvcConfiguration implements WebMvcConfigurer {
private final MdcInterceptor mdcInterceptor;
@Override
public void addInterceptors(InterceptorRegistry registry) {
registry.addInterceptor(mdcInterceptor);
}
}
Steps 3: Add a Random unique id generator function
public Function<Void, String> getRandomString = (nothing) -> UUID.randomUUID().toString().replaceAll("-","");
Random String : 26c322bc3ffa48488a89f8755ca896c2
Steps 4: Add “requestId” into your log config (logback-spring.xml)
<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<pattern>%d{ISO8601} %X{requestId} [%thread] %-5level %logger{36} - %msg%n</pattern>
</encoder>
</appender>
Now we are ready to use.
Log structure before
Log 1:2022–09–06 10:12:34,685 [http-nio-8080-exec-4] INFO c.s.s.impl.StudentServiceImpl — Fetching all student records
After
Log 2:2022-09-06 10:19:41,687 26c322bc3ffa48488a89f8755ca896c2[http-nio-8080-exec-4] INFO c.s.s.impl.StudentServiceImpl — Fetching all student records
We have successfully gotten the unique identifier with a log. But
There is a problem with the above flow. We have added “requestId” in MDC that is only applicable in sync flow not in async flow.
Because MDC doesn’t visible or accessible to the thread pool.
Let's assume "Fetching all student records" Log written inside a async method then Log 1 would have printed.
It won’t work in async flow.
The reason behind it is simple:
MDC is only visible and accessible from the flow where thread context is nowhere used.
Example:
@Async
public void notifyStudent(Long studentId){
log.info("sending welcome email to student {} ", studentId);
}
Whenever we run notifyStudent in both cases whether we have added a unique request id in the request or not. the printed log must be the same.
2022–09–06 10:12:34,685 [http-nio-8080-exec-4] INFO c.s.s.impl.StudentServiceImpl — sending welcome email to student 12
Request id missing in the log.
To solve the issue
Steps 5: Add MDCAsyncTaskDecorator
public class MDCAsyncTaskDecorator implements TaskDecorator {
@Override
public Runnable decorate(Runnable runnable) {
Map<String, String> contextMap = MDC.getCopyOfContextMap();
if (contextMap == null)
contextMap = new HashMap<>();
try {
MDC.setContextMap(contextMap);
runnable.run();
} finally {
MDC.clear();
}
return runnable;
}
}
In MDCAsyncTaskDecorator
we are performing two steps
1. Copy MDC context Map
2. Add Copied context Map to thread context
Now thread executor can easily access requestId
from MDC.
Step 6: Register MDCAsyncTaskDecorator
as task decorator in executor config.
ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
executor.setTaskDecorator(new MDCAsyncTaskDecorator());
Now we are ready. Run your application and test it.
Note: It is only valid for API requests. Not for
1. Jobs: https://stackoverflow.com/questions/34801432/how-to-get-mdc-logging-working-for-spring-batch2. Listeners:
Ref Link :
RabbitMQ: https://stackoverflow.com/questions/60286491/set-mdc-properties-only-for-rabbitmq-events3. Socket Events
Happy Learning !!! Hope you like it.