Basics of Logging
What is logging? Recording events and data from your application for debugging, monitoring, and analysis. Good logging is your safety net when things go wrong and your insights when optimizing performance.
Why log? Debugging production issues, monitoring application health, understanding user behavior, compliance requirements, and performance optimization. Without logs, you're flying blind.
When to log? During errors and exceptions, important business events, performance bottlenecks, security events, and state changes. Don't log in tight loops or overly verbose debug information in production.
What Should Be Logged
Always Log These Events
Errors and exceptions - Stack traces, error messages, and context that led to the failure.
Authentication events - Login attempts, password changes, privilege escalations.
Business-critical actions - User registrations, purchases, data modifications, file uploads.
Performance issues - Slow database queries, API timeouts, high memory usage.
Security events - Failed login attempts, suspicious IP addresses, access denied events.
Context to Include
Request information - HTTP method, URL, headers, IP address, user agent.
User context - User ID, session ID, roles/permissions, organization.
System context - Server name, environment, application version, timestamp.
Business context - Transaction IDs, order numbers, account numbers, feature flags.
Never Log These
Sensitive data - Passwords, credit card numbers, social security numbers, API keys.
Personal information - Full addresses, phone numbers (unless required for compliance).
Large payloads - Entire file contents, massive JSON objects, binary data.
Structure of a Log
Standard Log Format (JSON)
{
"timestamp": "2024-06-23T14:30:25.123Z",
"level": "INFO",
"message": "User login successful",
"service": "auth-service",
"version": "1.2.3",
"environment": "production",
"requestId": "req_abc123",
"userId": "user_456",
"context": {
"email": "user@example.com",
"ip": "192.168.1.100",
"userAgent": "Mozilla/5.0...",
"loginMethod": "password"
},
"metadata": {
"duration": 245,
"endpoint": "/api/auth/login"
}
}
Log Levels Explained
ERROR - Application failures, exceptions, critical issues that need immediate attention.
WARN - Recoverable issues, deprecated feature usage, configuration problems.
INFO - Business events, successful operations, system state changes.
DEBUG - Detailed diagnostic information, variable values, execution flow.
TRACE - Very detailed debugging, function entry/exit, step-by-step execution.
Essential Fields
// Minimum required fields for every log entry
const logEntry = {
timestamp: new Date().toISOString(), // When it happened
level: 'INFO', // Severity level
message: 'User created account', // Human-readable description
service: 'user-service', // Which service/component
requestId: 'req_xyz789', // Correlation ID
userId: 'user_123', // Who was involved
action: 'account_creation', // What happened
context: { // Additional relevant data
email: 'user@example.com',
source: 'web_app'
}
};
Where to Log
Cloud Logging Services
Axiom - Fast, cost-effective, great for high-volume applications. Excellent query performance and real-time analytics.
// Axiom integration
const axiom = require('@axiomhq/node');
const client = new axiom.Client({
token: process.env.AXIOM_TOKEN,
orgId: process.env.AXIOM_ORG_ID,
});
await client.ingest('my-dataset', [logEntry]);
Datadog - Comprehensive monitoring with APM integration. Best for full observability stack.
New Relic - Application performance monitoring with good log correlation.
Elastic Stack (ELK) - Self-hosted option with powerful search and visualization.
AWS CloudWatch - Native AWS integration, good for AWS-heavy infrastructure.
Google Cloud Logging - Integrated with Google Cloud Platform services.
Self-Hosted Options
Grafana Loki - Lightweight, cost-effective, designed for logs. Works well with Grafana dashboards.
Fluentd + Elasticsearch - Flexible log collection and storage. Good for complex routing needs.
Syslog servers - Traditional option for system logs and compliance requirements.
Local Development
Console logging - Built-in browser and Node.js console methods.
File logging - Write to local files during development and testing.
// Development vs Production logging
const logger = winston.createLogger({
level: process.env.LOG_LEVEL || 'info',
format: winston.format.json(),
transports: [
// Always log errors to file
new winston.transports.File({
filename: 'error.log',
level: 'error'
}),
// Console in development, external service in production
process.env.NODE_ENV === 'production'
? new winston.transports.Http({
host: 'logs.axiom.co',
port: 443,
path: `/api/v1/datasets/${dataset}/ingest`,
})
: new winston.transports.Console({
format: winston.format.simple()
})
]
});
Choosing a Logging Solution
Volume and cost - Axiom and Loki are cost-effective for high volume. Datadog and New Relic are expensive but feature-rich.
Integration needs - Choose based on your existing monitoring stack and cloud provider.
Query requirements - Elasticsearch excels at complex searches. Axiom is fast for time-series queries.
Retention policies - Consider how long you need to keep logs and compliance requirements.
Team expertise - Self-hosted solutions require more maintenance but offer more control.
Core Logging Principles
Use structured logging - Always log in JSON format for better searchability and parsing.
Include context - Add request IDs, user IDs, and relevant metadata to every log entry.
Log levels matter - Use appropriate levels: ERROR for failures, WARN for recoverable issues, INFO for business events, DEBUG for development.
Backend Logging (Node.js/Express)
Basic Setup with Winston
const winston = require('winston');
const logger = winston.createLogger({
level: process.env.LOG_LEVEL || 'info',
format: winston.format.combine(
winston.format.timestamp(),
winston.format.errors({ stack: true }),
winston.format.json()
),
transports: [
new winston.transports.File({ filename: 'error.log', level: 'error' }),
new winston.transports.File({ filename: 'combined.log' }),
new winston.transports.Console({
format: winston.format.simple()
})
]
});
module.exports = logger;
Request Middleware
const { v4: uuidv4 } = require('uuid');
const requestLogger = (req, res, next) => {
req.requestId = uuidv4();
logger.info('Request started', {
requestId: req.requestId,
method: req.method,
url: req.url,
userAgent: req.get('User-Agent'),
ip: req.ip
});
const start = Date.now();
res.on('finish', () => {
const duration = Date.now() - start;
logger.info('Request completed', {
requestId: req.requestId,
statusCode: res.statusCode,
duration: `${duration}ms`
});
});
next();
};
app.use(requestLogger);
Error Logging
// Async error wrapper
const asyncHandler = (fn) => (req, res, next) => {
Promise.resolve(fn(req, res, next)).catch(next);
};
// Route handler
app.get('/api/users/:id', asyncHandler(async (req, res) => {
try {
const user = await User.findById(req.params.id);
if (!user) {
logger.warn('User not found', {
requestId: req.requestId,
userId: req.params.id
});
return res.status(404).json({ error: 'User not found' });
}
logger.info('User retrieved successfully', {
requestId: req.requestId,
userId: user.id
});
res.json(user);
} catch (error) {
logger.error('Failed to retrieve user', {
requestId: req.requestId,
userId: req.params.id,
error: error.message,
stack: error.stack
});
res.status(500).json({ error: 'Internal server error' });
}
}));
// Global error handler
app.use((err, req, res, next) => {
logger.error('Unhandled error', {
requestId: req.requestId,
error: err.message,
stack: err.stack,
url: req.url,
method: req.method
});
res.status(500).json({ error: 'Something went wrong' });
});
Database Query Logging
// Mongoose middleware
userSchema.post('save', function(doc) {
logger.info('User saved', {
userId: doc._id,
email: doc.email,
action: 'user_created'
});
});
userSchema.post('findOneAndUpdate', function(doc) {
logger.info('User updated', {
userId: doc._id,
action: 'user_updated'
});
});
// SQL query logging (with Sequelize)
const sequelize = new Sequelize(database, username, password, {
logging: (sql, timing) => {
logger.debug('Database query', {
sql: sql,
duration: timing,
timestamp: new Date().toISOString()
});
}
});
Frontend Logging (React/JavaScript)
Client-Side Logger Setup
class ClientLogger {
constructor() {
this.endpoint = '/api/logs';
this.sessionId = this.generateSessionId();
this.queue = [];
this.flushInterval = 5000; // 5 seconds
this.startPeriodicFlush();
}
generateSessionId() {
return Date.now().toString(36) + Math.random().toString(36).substr(2);
}
log(level, message, context = {}) {
const logEntry = {
level,
message,
context: {
...context,
sessionId: this.sessionId,
timestamp: new Date().toISOString(),
url: window.location.href,
userAgent: navigator.userAgent
}
};
// Console log for development
if (process.env.NODE_ENV === 'development') {
console[level] || console.log(logEntry);
}
this.queue.push(logEntry);
// Immediate flush for errors
if (level === 'error') {
this.flush();
}
}
error(message, context) { this.log('error', message, context); }
warn(message, context) { this.log('warn', message, context); }
info(message, context) { this.log('info', message, context); }
debug(message, context) { this.log('debug', message, context); }
async flush() {
if (this.queue.length === 0) return;
const logs = [...this.queue];
this.queue = [];
try {
await fetch(this.endpoint, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ logs })
});
} catch (error) {
// Don't log the logging error to avoid infinite loops
console.error('Failed to send logs:', error);
// Put logs back in queue for retry
this.queue.unshift(...logs);
}
}
startPeriodicFlush() {
setInterval(() => this.flush(), this.flushInterval);
// Flush on page unload
window.addEventListener('beforeunload', () => this.flush());
}
}
export const logger = new ClientLogger();
React Error Boundary
import React from 'react';
import { logger } from './logger';
class ErrorBoundary extends React.Component {
constructor(props) {
super(props);
this.state = { hasError: false };
}
static getDerivedStateFromError(error) {
return { hasError: true };
}
componentDidCatch(error, errorInfo) {
logger.error('React component error', {
error: error.message,
stack: error.stack,
componentStack: errorInfo.componentStack,
component: this.props.componentName || 'Unknown'
});
}
render() {
if (this.state.hasError) {
return <h2>Something went wrong.</h2>;
}
return this.props.children;
}
}
// Usage
function App() {
return (
<ErrorBoundary componentName="App">
<Header />
<Main />
<Footer />
</ErrorBoundary>
);
}
API Call Logging
// API wrapper with logging
class ApiClient {
constructor() {
this.baseURL = process.env.REACT_APP_API_URL;
}
async request(method, endpoint, data = null) {
const requestId = Date.now().toString(36);
const startTime = Date.now();
logger.info('API request started', {
requestId,
method,
endpoint,
hasData: !!data
});
try {
const response = await fetch(`${this.baseURL}${endpoint}`, {
method,
headers: {
'Content-Type': 'application/json',
'X-Request-ID': requestId
},
body: data ? JSON.stringify(data) : null
});
const duration = Date.now() - startTime;
if (!response.ok) {
const errorText = await response.text();
logger.error('API request failed', {
requestId,
method,
endpoint,
status: response.status,
statusText: response.statusText,
duration,
error: errorText
});
throw new Error(`API Error: ${response.status}`);
}
const result = await response.json();
logger.info('API request completed', {
requestId,
method,
endpoint,
status: response.status,
duration
});
return result;
} catch (error) {
const duration = Date.now() - startTime;
logger.error('API request exception', {
requestId,
method,
endpoint,
duration,
error: error.message
});
throw error;
}
}
get(endpoint) { return this.request('GET', endpoint); }
post(endpoint, data) { return this.request('POST', endpoint, data); }
put(endpoint, data) { return this.request('PUT', endpoint, data); }
delete(endpoint) { return this.request('DELETE', endpoint); }
}
export const api = new ApiClient();
User Action Tracking
// Custom hook for action logging
import { useCallback } from 'react';
import { logger } from './logger';
export function useActionLogger() {
const logAction = useCallback((action, context = {}) => {
logger.info('User action', {
action,
...context,
timestamp: new Date().toISOString()
});
}, []);
return logAction;
}
// Usage in components
function LoginForm() {
const logAction = useActionLogger();
const handleSubmit = async (formData) => {
logAction('login_attempt', { email: formData.email });
try {
await api.post('/auth/login', formData);
logAction('login_success');
} catch (error) {
logAction('login_failure', { error: error.message });
}
};
return (
<form onSubmit={handleSubmit}>
{/* form fields */}
</form>
);
}
Log Management Best Practices
Environment-Specific Configuration
// config/logging.js
const configs = {
development: {
level: 'debug',
console: true,
file: false
},
staging: {
level: 'info',
console: true,
file: true
},
production: {
level: 'warn',
console: false,
file: true,
external: true // Send to external service
}
};
export const loggingConfig = configs[process.env.NODE_ENV] || configs.development;
Log Sanitization
// Remove sensitive data from logs
const sanitizeLogData = (data) => {
const sensitive = ['password', 'token', 'secret', 'key', 'authorization'];
const sanitized = { ...data };
const sanitizeObject = (obj) => {
Object.keys(obj).forEach(key => {
if (sensitive.some(s => key.toLowerCase().includes(s))) {
obj[key] = '[REDACTED]';
} else if (typeof obj[key] === 'object' && obj[key] !== null) {
sanitizeObject(obj[key]);
}
});
};
if (typeof sanitized === 'object') {
sanitizeObject(sanitized);
}
return sanitized;
};
// Use in logger
logger.info('User login', sanitizeLogData({
email: user.email,
password: user.password, // Will be redacted
loginTime: new Date()
}));
Performance Monitoring
// Performance logging utility
class PerformanceLogger {
static startTimer(name) {
const start = performance.now();
return () => {
const duration = performance.now() - start;
logger.info('Performance metric', {
metric: name,
duration: `${duration.toFixed(2)}ms`,
timestamp: new Date().toISOString()
});
};
}
static async trackAsync(name, asyncFunction) {
const endTimer = this.startTimer(name);
try {
const result = await asyncFunction();
endTimer();
return result;
} catch (error) {
endTimer();
logger.error('Performance tracked function failed', {
metric: name,
error: error.message
});
throw error;
}
}
}
// Usage
const endTimer = PerformanceLogger.startTimer('database_query');
const users = await User.findAll();
endTimer();
// Or with async tracking
const result = await PerformanceLogger.trackAsync('api_call',
() => api.get('/users')
);
Key Takeaways
Structure everything - Use consistent JSON format across frontend and backend.
Include correlation IDs - Track requests across your entire stack with unique identifiers.
Log business events - Not just errors, but successful user actions and system events.
Sanitize sensitive data - Never log passwords, tokens, or personal information.
Use appropriate levels - ERROR for failures, WARN for issues, INFO for events, DEBUG for development.
Monitor performance - Log slow queries and API calls to identify bottlenecks.
Centralize logs - Send both frontend and backend logs to the same system for easier debugging.