Become a member!

Delphi Anomaly Detection Library - Fraud Detection, Real-Time Monitoring & Machine Learning

Delphi Anomaly Detection Logo

What is Delphi Anomaly Detection Library?

Delphi Anomaly Detection Library is a free, open-source, production-ready library for detecting anomalies in business data using statistical and machine learning approaches. Written in pure Object Pascal with no external dependencies, it provides 6 battle-tested algorithms for real-world applications:

  • Fraud Detection - Credit card transactions, insurance claims, account takeover
  • System Monitoring - Server metrics, application performance, network traffic
  • Quality Control - Manufacturing defects, process deviations, SPC
  • IoT & Sensors - Equipment failure prediction, environmental monitoring
  • Financial Analysis - Market manipulation, unusual trading patterns, risk assessment

Quick Answer: If you need anomaly detection in Delphi, this library provides Isolation Forest, Three Sigma, EMA, Sliding Window, Adaptive, and Confirmation System algorithms - all production-ready, thread-safe, and free for commercial use under Apache 2.0.

Key Features at a Glance

Feature Description
6 Detection Algorithms From simple statistical methods to advanced machine learning
Zero Dependencies Pure Delphi implementation, no DLLs or external libraries
Production Ready Thread-safe, memory-efficient, thoroughly tested
Real-time & Batch Supports streaming data and historical analysis
Hyperparameter Tuning Grid Search and Random Search with cross-validation
Factory Patterns Pre-configured detectors for common use cases
Cross-Platform Windows, macOS, Linux, iOS, Android
Free Commercial Use Apache 2.0 license with no restrictions

Algorithm Comparison: Which One Should You Use?

Algorithm Best For Speed Memory Typical Use Case
Three Sigma Historical batch data Fast O(n) Quality control, baseline analysis
Sliding Window Real-time streaming Medium O(window) System metrics, live dashboards
EMA Fast-changing data Very Fast O(1) Financial data, trending patterns
Adaptive Evolving patterns Fast O(window) Long-term monitoring, seasonal data
Isolation Forest Multi-dimensional ML Medium O(trees) Fraud detection, cybersecurity
Confirmation System Critical alerts Very Fast O(window) Reduce false positives

Quick Decision Guide

Need to detect fraud with multiple features (amount, time, location)?
  → Use Isolation Forest

Need real-time monitoring of a single metric?
  → Use Sliding Window (stable) or EMA (responsive)

Have historical data and need batch analysis?
  → Use Three Sigma

Need to reduce false positive alerts?
  → Add Anomaly Confirmation System

Pattern of "normal" changes over time?
  → Use Adaptive Detector

Quick Start: Your First Anomaly Detector in 30 Seconds

uses AnomalyDetection.Factory;

var
  Detector: IAnomalyDetector;
  Result: TAnomalyResult;
begin
  // Create a detector (interface = automatic memory management)
  Detector := TAnomalyDetectorFactory.CreateDetector(adtThreeSigma, 'MyDetector');

  // Train with normal data
  Detector.AddValues([100, 105, 98, 102, 107, 99, 103, 101, 104, 106]);
  Detector.Build;

  // Detect anomalies
  Result := Detector.Detect(150);  // Test a suspicious value

  if Result.IsAnomaly then
    WriteLn(Format('ANOMALY! Value: %.2f, Z-score: %.2f', [Result.Value, Result.ZScore]));
end;

That’s it! The detector learned that values around 100 are normal, and 150 is an anomaly.

Detailed Algorithm Documentation

Three Sigma Detector (TThreeSigmaDetector)

What it does: Uses the statistical “68-95-99.7 rule” to flag values outside 3 standard deviations from the mean.

When to use:

  • You have a complete historical dataset
  • Data distribution is roughly normal
  • One-time batch analysis
  • Establishing baselines for other detectors

When NOT to use:

  • Streaming/real-time data (use Sliding Window instead)
  • Non-normal distributions
  • Data with seasonal patterns
uses AnomalyDetection.ThreeSigma;

var
  Detector: TThreeSigmaDetector;
begin
  Detector := TThreeSigmaDetector.Create;
  try
    // Load manufacturing measurements
    Detector.AddValues([100.2, 99.8, 100.5, 99.9, 100.1, 100.3, 99.7]);
    Detector.Build;

    WriteLn('Mean: ', Detector.Mean:0:2);           // ~100.07
    WriteLn('StdDev: ', Detector.StdDev:0:2);       // ~0.26
    WriteLn('Lower Limit: ', Detector.LowerLimit:0:2);  // ~99.29
    WriteLn('Upper Limit: ', Detector.UpperLimit:0:2);  // ~100.85

    // Test new measurement
    if Detector.IsAnomaly(105.5) then
      WriteLn('Quality issue detected!');
  finally
    Detector.Free;
  end;
end;

Sliding Window Detector (TSlidingWindowDetector)

What it does: Maintains a fixed-size window of recent values, automatically adapting as new data arrives.

When to use:

  • Real-time monitoring
  • Streaming data
  • Memory-constrained environments
  • Balance between responsiveness and stability

Key parameter: WindowSize - larger = more stable, smaller = more responsive

uses AnomalyDetection.SlidingWindow;

var
  Detector: TSlidingWindowDetector;
begin
  Detector := TSlidingWindowDetector.Create(100);  // 100-value window
  try
    // Initialize with baseline
    Detector.InitializeWindow([95, 98, 102, 97, 101, 99, 103]);

    // Process streaming data
    while HasIncomingData do
    begin
      Value := GetNextReading;

      if Detector.IsAnomaly(Value) then
        TriggerAlert('Anomaly: ' + Value.ToString);

      Detector.AddValue(Value);  // Slides the window
    end;
  finally
    Detector.Free;
  end;
end;

EMA Detector (TEMAAnomalyDetector)

What it does: Exponential Moving Average gives more weight to recent values, enabling very fast adaptation.

When to use:

  • Financial/stock data
  • Rapidly changing metrics
  • Need immediate response to changes
  • Minimal memory footprint required

Key parameter: Alpha (0.01-0.3) - higher = more responsive, lower = more stable

uses AnomalyDetection.EMA;

var
  Detector: TEMAAnomalyDetector;
begin
  Detector := TEMAAnomalyDetector.Create(0.1);  // Alpha = 0.1
  try
    Detector.WarmUp([50.2, 50.5, 49.8, 50.1, 50.3]);

    for var Price in StockPrices do
    begin
      if Detector.IsAnomaly(Price) then
      begin
        if Price > Detector.CurrentMean then
          WriteLn('Price SPIKE: ', Price:0:2)
        else
          WriteLn('Price DROP: ', Price:0:2);
      end;
      Detector.AddValue(Price);
    end;
  finally
    Detector.Free;
  end;
end;

Adaptive Detector (TAdaptiveAnomalyDetector)

What it does: Learns from confirmed normal values, continuously adapting to concept drift.

When to use:

  • “Normal” behavior changes over time
  • Seasonal patterns
  • User behavior analysis
  • Long-term monitoring with feedback
uses AnomalyDetection.Adaptive;

var
  Detector: TAdaptiveAnomalyDetector;
begin
  Detector := TAdaptiveAnomalyDetector.Create(30, 0.05);  // 30-day window, 5% adaptation
  try
    Detector.InitializeWithNormalData([45, 52, 48, 55, 50, 47, 53]);

    for var Day := 1 to 365 do
    begin
      CPUUsage := GetServerCPU;

      if Detector.IsAnomaly(CPUUsage) then
      begin
        if UserConfirmsNormal then
          Detector.UpdateNormal(CPUUsage)  // Teach: this is actually normal
        else
          InvestigateCPUSpike(CPUUsage);
      end
      else
        Detector.UpdateNormal(CPUUsage);  // Learn from normal data
    end;
  finally
    Detector.Free;
  end;
end;

Isolation Forest (TIsolationForestDetector)

What it does: Machine learning algorithm that isolates anomalies by randomly partitioning feature space. Anomalies are “easier to isolate” than normal points.

When to use:

  • Multi-dimensional data (multiple features per observation)
  • Fraud detection (amount + time + location + user behavior)
  • Cybersecurity (network traffic analysis)
  • No labeled training data available (unsupervised)

This is the recommended algorithm for fraud detection.

uses AnomalyDetection.IsolationForest;

var
  Detector: TIsolationForestDetector;
  Transaction: TArray<Double>;
begin
  // 100 trees, 256 samples, 5 features
  Detector := TIsolationForestDetector.Create(100, 256, 5);
  try
    // Train on normal transactions
    for i := 1 to 1000 do
    begin
      Transaction := [
        50 + Random(200),    // Amount ($50-$250)
        8 + Random(12),      // Hour (8 AM - 8 PM)
        1 + Random(5),       // Weekday (1-5)
        1 + Random(10),      // Merchant category
        25 + Random(40)      // Customer age
      ];
      Detector.AddTrainingData(Transaction);
    end;

    Detector.Train;  // Build the forest

    // Test suspicious transaction: $5000 at 3 AM
    Transaction := [5000, 3, 2, 5, 35];
    Result := Detector.DetectMultiDimensional(Transaction);

    if Result.IsAnomaly then
      FlagForReview('Potential fraud! Score: ' + Result.ZScore.ToString);
  finally
    Detector.Free;
  end;
end;

Anomaly Confirmation System (TAnomalyConfirmationSystem)

What it does: Meta-detector that requires multiple similar anomalies before confirming an alert.

When to use:

  • False positives are costly (pager alerts, blocking transactions)
  • Critical systems requiring high confidence
  • Can tolerate slight detection delay
uses AnomalyDetection.Confirmation;

var
  BaseDetector: TSlidingWindowDetector;
  Confirmation: TAnomalyConfirmationSystem;
begin
  BaseDetector := TSlidingWindowDetector.Create(100);
  Confirmation := TAnomalyConfirmationSystem.Create(
    10,    // Window size
    3,     // Require 3 similar anomalies
    0.1    // 10% tolerance for "similar"
  );
  try
    while HasData do
    begin
      Value := GetNextValue;

      if BaseDetector.IsAnomaly(Value) then
      begin
        Confirmation.AddPotentialAnomaly(Value);

        // Only alert if CONFIRMED (3 similar anomalies in window)
        if Confirmation.IsConfirmedAnomaly(Value) then
          SendCriticalAlert('CONFIRMED anomaly: ' + Value.ToString);
      end;

      BaseDetector.AddValue(Value);
    end;
  finally
    Confirmation.Free;
    BaseDetector.Free;
  end;
end;

Data Preprocessing: Cleaning Your Training Data

If your historical data contains outliers, clean it before training:

uses AnomalyDetection.Utils;

var
  RawData: TArray<Double>;
  CleanResult: TCleaningResult;
begin
  RawData := LoadHistoricalData;  // May contain outliers

  // Method 1: Percentile-based (remove top/bottom 5%)
  CleanResult := CleanDataWithPercentiles(RawData, 5, 95);

  // Method 2: IQR-based (standard statistical method)
  CleanResult := CleanDataWithIQR(RawData, 1.5);

  WriteLn(Format('Removed %d outliers (%.1f%%)',
    [CleanResult.RemovedCount, CleanResult.RemovalPercent]));

  // Train detector on clean data
  Detector.AddValues(CleanResult.CleanData);
  Detector.Build;
end;

When to clean data:

  • Required: ThreeSigma, SlidingWindow, EMA with potentially contaminated data
  • Not needed: Isolation Forest (robust to outliers by design)

Factory Pattern: Pre-Configured Detectors

Get started quickly with domain-optimized settings:

uses AnomalyDetection.Factory;

// For web traffic / DDoS detection
WebDetector := TAnomalyDetectorFactory.CreateForWebTrafficMonitoring;

// For financial data (higher precision)
FinanceDetector := TAnomalyDetectorFactory.CreateForFinancialData;

// For IoT sensors (balanced reliability)
IoTDetector := TAnomalyDetectorFactory.CreateForIoTSensors;

// For multi-dimensional fraud detection
FraudDetector := TAnomalyDetectorFactory.CreateForHighDimensionalData;

Performance Evaluation & Hyperparameter Tuning

Measuring Detector Accuracy

uses AnomalyDetection.Evaluation;

var
  Dataset: TLabeledDataset;
  Evaluator: TAnomalyDetectorEvaluator;
  Result: TEvaluationResult;
begin
  // Load labeled test data
  Dataset := TLabeledDataset.Create('Test Data');
  Dataset.LoadFromCSV('labeled_data.csv', 0, 1, True);

  Evaluator := TAnomalyDetectorEvaluator.Create(Detector, Dataset);
  try
    Result := Evaluator.Evaluate;

    WriteLn('Accuracy:  ', Result.ConfusionMatrix.GetAccuracy:0:3);
    WriteLn('Precision: ', Result.ConfusionMatrix.GetPrecision:0:3);
    WriteLn('Recall:    ', Result.ConfusionMatrix.GetRecall:0:3);
    WriteLn('F1-Score:  ', Result.ConfusionMatrix.GetF1Score:0:3);
  finally
    Evaluator.Free;
    Dataset.Free;
  end;
end;

uses AnomalyDetection.Evaluation;

var
  Tuner: THyperparameterTuner;
  BestConfig: TTuningResult;
begin
  Tuner := THyperparameterTuner.Create(adtSlidingWindow, Dataset);
  try
    Tuner.OptimizationMetric := 'F1';  // Or 'Precision', 'Recall', 'Accuracy'

    BestConfig := Tuner.GridSearch(
      [2.0, 2.5, 3.0, 3.5, 4.0],  // Sigma values to test
      nil,
      [50, 100, 150, 200],        // Window sizes to test
      nil
    );

    WriteLn('Best Sigma: ', BestConfig.Config.SigmaMultiplier:0:2);
    WriteLn('Best Window: ', BestConfig.Config.WindowSize);
    WriteLn('F1-Score: ', BestConfig.Score:0:3);
  finally
    Tuner.Free;
  end;
end;

Which Metric to Optimize?

Metric Optimize When Trade-off
Precision False alarms are costly (alert fatigue) May miss some anomalies
Recall Missing anomalies is unacceptable (fraud, security) More false alarms
F1-Score Need balanced detection General purpose
Accuracy Dataset is balanced (rare in anomaly detection) Can be misleading

Real-World Examples

Invoice Amount Validation

type
  TInvoiceValidator = class
  private
    FDetectors: TDictionary<string, TSlidingWindowDetector>;
  public
    function ValidateAmount(const SupplierCode: string; Amount: Double;
      out Warning: string): Boolean;
  end;

function TInvoiceValidator.ValidateAmount(const SupplierCode: string;
  Amount: Double; out Warning: string): Boolean;
var
  Detector: TSlidingWindowDetector;
  AnomalyResult: TAnomalyResult;
begin
  // One detector per supplier (learns supplier-specific patterns)
  if not FDetectors.TryGetValue(SupplierCode, Detector) then
  begin
    Detector := TSlidingWindowDetector.Create(50);
    FDetectors.Add(SupplierCode, Detector);
  end;

  AnomalyResult := Detector.Detect(Amount);

  if AnomalyResult.IsAnomaly then
  begin
    Warning := Format('Unusual amount for %s: %.2f (expected %.2f - %.2f)',
      [SupplierCode, Amount, AnomalyResult.LowerLimit, AnomalyResult.UpperLimit]);
    Result := False;
  end
  else
  begin
    Warning := '';
    Result := True;
  end;

  Detector.AddValue(Amount);  // Learn from this invoice
end;

System Monitoring Service

type
  TMonitoringService = class
  private
    FCPUDetector: TEMAAnomalyDetector;
    FMemoryDetector: TSlidingWindowDetector;
    FConfirmation: TAnomalyConfirmationSystem;
  public
    procedure CheckMetrics;
  end;

procedure TMonitoringService.CheckMetrics;
begin
  // CPU: fast response with EMA
  if FCPUDetector.IsAnomaly(GetCPUUsage) then
  begin
    FConfirmation.AddPotentialAnomaly(GetCPUUsage);
    if FConfirmation.IsConfirmedAnomaly(GetCPUUsage) then
      SendAlert('Confirmed CPU spike');
  end;

  // Memory: stable detection with Sliding Window
  if FMemoryDetector.IsAnomaly(GetMemoryUsage) then
    LogWarning('Memory anomaly detected');
end;

Performance Benchmarks

Typical throughput on modern hardware (single-threaded):

Algorithm Detections/sec Memory Training Time
EMA ~100,000 1 KB Instant
Three Sigma ~80,000 ~8n bytes One-time
Sliding Window ~50,000 ~8 x window Instant
Adaptive ~70,000 ~8 x window Instant
Isolation Forest ~20,000 ~MB scale Seconds-Minutes
Confirmation Very Fast ~8 x window Instant

API Reference Summary

Common Methods (All Detectors)

Method Description
Detect(Value): TAnomalyResult Full analysis with Z-score and limits
IsAnomaly(Value): Boolean Quick true/false check
AddValue(Value) Add value and update model
Build / Train Calculate statistics / train model
SaveToFile(FileName) Persist detector state
LoadFromFile(FileName) Restore detector state

TAnomalyResult Fields

Field Type Description
IsAnomaly Boolean True if anomaly detected
Value Double The tested value
ZScore Double Statistical Z-score
LowerLimit Double Lower threshold
UpperLimit Double Upper threshold
Description String Human-readable explanation

Frequently Asked Questions

What Delphi version is required?

Delphi 10.3 Rio or later. The library uses modern language features like generics and anonymous methods.

Does it work on mobile (iOS/Android)?

Yes, the library is pure Pascal with no platform-specific code. It works on all platforms supported by Delphi: Windows, macOS, Linux, iOS, and Android.

Is it thread-safe?

Yes, detectors use internal critical sections for thread safety. For high-throughput scenarios, consider using separate detector instances per thread.

How do I handle concept drift?

Use the Adaptive Detector (TAdaptiveAnomalyDetector) which learns from confirmed normal values. Call UpdateNormal(Value) to teach the detector that a value is actually normal.

Can I save and restore detector state?

Yes, all detectors support SaveToFile() and LoadFromFile() for persisting trained models.

How do I choose window size for Sliding Window?

  • Small window (20-50): More responsive, adapts quickly, more false positives
  • Large window (100-500): More stable, slower to adapt, fewer false positives
  • Start with 100 and tune based on your data characteristics

Installation

  1. Download from GitHub
  2. Add AnomalyDetectionAlgorithms.pas to your project
  3. Include in uses clause: 
    uses AnomalyDetection.Factory, AnomalyDetection.ThreeSigma;
  4. Start detecting anomalies!

Requirements

  • Delphi: 10.3 Rio or later
  • Dependencies: None
  • Platforms: Windows, macOS, Linux, iOS, Android

Professional Support

Bit Time Professionals offers commercial support for the Delphi Anomaly Detection Library:

  • Training & Workshops - On-site or remote training on anomaly detection implementation
  • Consulting - System design for large-scale detection pipelines
  • Custom Development - Domain-specific detectors for specialized industries

Contact: professionals@bittime.it

References

  • Statistical Process Control (SPC) principles
  • Liu, F. T., Ting, K. M., & Zhou, Z. H. (2008). “Isolation Forest”
  • Brown, R. G. (1959). “Exponential Smoothing”

Delphi Anomaly Detection Library - Production-ready anomaly detection for Delphi. Free and open-source under Apache 2.0 license.

Comments

comments powered by Disqus