Skip to content

touskar/openafis

BranchesTags

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

24 Commits
.idea
.idea
 
 
docs
docs
 
 
k8s
k8s
 
 
src/main
src/main
 
 
target
target
 
 
test_ressources
test_ressources
 
 
.env.sample
.env.sample
 
 
.gitignore
.gitignore
 
 
CLAUDE.md
CLAUDE.md
 
 
LICENSE
LICENSE
 
 
OpenAFIS.iml
OpenAFIS.iml
 
 
README.md
README.md
 
 
TEST_PROGRESSION.json
TEST_PROGRESSION.json
 
 
TEST_RESULT.json
TEST_RESULT.json
 
 
URLS.md
URLS.md
 
 
clear_aerospike.py
clear_aerospike.py
 
 
dependency-reduced-pom.xml
dependency-reduced-pom.xml
 
 
dev-k8s.sh
dev-k8s.sh
 
 
integration_test_results.json
integration_test_results.json
 
 
integration_test_results.json.bak
integration_test_results.json.bak
 
 
pom.xml
pom.xml
 
 
run_javalin.sh
run_javalin.sh
 
 
skaffold.yaml
skaffold.yaml
 
 
test_dpi_fixes.class
test_dpi_fixes.class
 
 

Repository files navigation

OpenAFIS

⚠️ Development Status: This application is currently under active development and is NOT ready for production use. Features may be incomplete, APIs may change, and stability is not guaranteed.

Open Automated Fingerprint Identification System

A high-performance, scalable distributed biometric identification system built with modern microservices architecture. OpenAFIS provides comprehensive REST APIs for fingerprint enrollment, template extraction, matching, and large-scale identification operations.

Built on top of industry-leading technologies including Apache Spark for distributed computing, Aerospike for high-performance NoSQL storage, Kubernetes for container orchestration, and SourceAFIS (robertvazan/sourceafis-java) for advanced fingerprint matching algorithms.

Java Maven Apache Spark Aerospike Kubernetes License Build

🚀 Features

Core Capabilities

  • Fingerprint Enrollment - Multi-finger enrollment with quality assessment
  • Template Extraction - Advanced minutiae extraction and template generation
  • 1:1 Verification - High-speed fingerprint verification
  • 1:N Identification - Large-scale database searching
  • Multi-finger Processing - Simultaneous processing of multiple fingerprints
  • Async Job Queue - Distributed background processing for large operations

Technical Features

  • RESTful APIs - Comprehensive OpenAPI 3.0 specification
  • Microservices Architecture - Scalable, cloud-ready design
  • Real-time Processing - Sub-second response times
  • Distributed Computing - Apache Spark integration for large-scale operations
  • Advanced Logging - Elasticsearch + Kibana integration
  • Production Ready - Health checks, metrics, monitoring

🏗️ Architecture

Technology Stack

  • Framework: Javalin 5.6.3 (Lightweight Java web framework)
  • Build System: Apache Maven
  • Runtime: Java 17+
  • Database: Aerospike (High-performance NoSQL)
  • Search & Analytics: Elasticsearch + Kibana
  • Big Data: Apache Spark
  • Container: Docker + Kubernetes ready

System Components

OpenAFIS operates as a secure backend service consumed by custom developer platforms. The top layer represents custom applications built by developers that leverage OpenAFIS biometric capabilities through a secure API gateway with RBAC (Role-Based Access Control). These platforms can be web applications, mobile apps, or enterprise systems that need fingerprint identification services.

                    ┌─────────────────┐    ┌─────────────────┐    ┌─────────────────┐
                    │ Custom Web App  │    │Custom Mobile App│    │ Developer Tools │
                    │   (with RBAC)   │    │   (with RBAC)   │    │   (with RBAC)   │
                    └─────────────────┘    └─────────────────┘    └─────────────────┘
                             │                       │                       │
                             └───────────────────────┼───────────────────────┘
                                                     │
                                        ┌─────────────────────┐
                                        │    API Gateway      │
                                        │ (Authentication &   │
                                        │  Authorization)     │
                                        └─────────────────────┘
                                                     │
    ════════════════════════════════════════════════════════════════════════════════
                                   ISOLATED NETWORK
    ════════════════════════════════════════════════════════════════════════════════
                                                     │
                                        ┌─────────────────────┐
                                        │   OpenAFIS API      │
                                        │   (Javalin Core)    │
                                        └─────────────────────┘
                                                     │
                                     ┌───────────────────┐
                                     │                   │
                            ┌─────────────────┐      ┌─────────────────┐
                            │   Aerospike     │      │ Apache Spark    │
                            │ (All Data & IDs)│      │ (Processing)    │
                            └─────────────────┘      └─────────────────┘
                                     │                   │
                                     └───────────────────┘
                                                     │
                                        ┌─────────────────────┐
                                        │  Elasticsearch      │
                                        │   + Kibana          │
                                        │  (Logs & Analytics) │
                                        └─────────────────────┘

📋 Prerequisites

Required Software

  • Java 17+ - Runtime environment
  • Apache Maven 3.8+ - Build system
  • Docker - Container runtime
  • kubectl - Kubernetes CLI (for infrastructure)

Infrastructure Dependencies

  • Aerospike - High-performance NoSQL database
  • Aerospike Data Browser - SQL-based data exploration with table views
  • Elasticsearch - Search and analytics
  • Kibana - Log visualization
  • Apache Spark - Distributed computing

🛠️ Quick Start

1. Clone Repository

git clone https://github.com/touskar/openafis.git
cd openafis

2. Configure Environment

# Copy sample configuration
cp .env.sample .env

# Edit configuration as needed
nano .env

3. Setup Infrastructure

# Start all required services (Aerospike, Elasticsearch, Spark)
./dev-k8s.sh

4. Setup Database Indexes

# Setup Elasticsearch and Kibana
./k8s/elasticsearch/create_index_pattern.sh

5. Run Application

# Development mode (auto-compile, development features)
./run_javalin.sh

# Production mode
./run_javalin.sh --prod

# Custom port
./run_javalin.sh -p 8080

6. Verify Installation

# Basic health check
curl http://localhost:8085/health

# Detailed health with service status
curl http://localhost:8085/health/detailed

# Prometheus metrics
curl http://localhost:8085/metrics/prometheus

# API documentation (Swagger UI)
open http://localhost:8085/docs

# API request builder
open http://localhost:8085/request-builder

# Kibana dashboard
open http://localhost:5601

# Spark Web UI (when running in cluster mode)
open http://localhost:4040

📊 API Documentation

Core Endpoints

Health & System

GET  /                        # Service information and endpoints
GET  /health                  # Basic health check
GET  /health/detailed         # Detailed system status with service health
GET  /metrics/prometheus      # Prometheus-compatible metrics
GET  /queue/status            # Job queue status

Candidate Management

POST /candidates                      # Create new candidate
GET  /candidates/{id}                 # Get candidate details
PUT  /candidates/{id}/fingerprints    # Update fingerprints
DELETE /candidates/{id}               # Delete candidate
GET  /candidates                      # List candidates (paginated)

Fingerprint Operations

POST /fingerprints/verify                    # 1:1 verification
POST /fingerprints/identify                  # 1:N identification
POST /fingerprints/identify-async            # Async identification
GET  /fingerprints/identify-async/{job_id}   # Check async job status
DELETE /fingerprints/identify-async/{job_id}/cancel  # Cancel async job

Multi-finger Processing

POST /fingerprints/multi-finger      # Multi-finger search

Documentation

GET  /docs               # Swagger UI
GET  /openapi.json       # OpenAPI specification
GET  /request-builder    # API request builder

Authentication

Currently configured for development use. Production deployments should implement:

  • JWT token authentication
  • API key management
  • Rate limiting
  • Input validation

🚀 Deployment

Development

# Quick development start
./run_javalin.sh

Docker

# Build image
docker build -t openafis:latest .

# Run container
docker run -p 8085:8085 --env-file .env openafis:latest

Kubernetes

# Deploy infrastructure
kubectl apply -f k8s/

# Deploy application
kubectl apply -f k8s/openafis-deployment.yaml

Production Considerations

  • Scaling: Horizontal pod autoscaling configured
  • Monitoring: Prometheus metrics endpoint available
  • Logging: Structured logging to Elasticsearch
  • Security: Configure authentication and authorization
  • Performance: 17 optimized Aerospike indexes for query performance

⚙️ Configuration

Environment Variables

Application Settings

APP_NAME=OpenAFIS                    # Application name
APP_ENV=development                  # Environment (development/production)
APP_PORT=8085                       # HTTP server port

Database Configuration

# Aerospike  
AEROSPIKE_HOST=localhost
AEROSPIKE_PORT=3000
AEROSPIKE_NAMESPACE=openafis

# Elasticsearch
ELASTICSEARCH_URL=http://localhost:9200
ELASTIC_USERNAME=elastic
ELASTIC_PASSWORD=password

Apache Spark Configuration

# Spark Cluster Connection
SPARK_MASTER_URL=local[*]                  # Local mode (all CPU cores) or spark://localhost:7077 for cluster
SPARK_UI_URL=http://localhost:8080         # Spark Web UI URL (cluster) or http://localhost:4040 (local)

# Spark Resource Allocation
SPARK_EXECUTOR_MEMORY=2g                   # Memory per executor
SPARK_EXECUTOR_CORES=2                     # CPU cores per executor
SPARK_DRIVER_MEMORY=4g                     # Driver memory
SPARK_DRIVER_MAX_RESULT_SIZE=2g            # Max result size

# Spark Serialization (optimized for biometric data)
SPARK_SERIALIZER=org.apache.spark.serializer.KryoSerializer
SPARK_KRYO_REGISTRATOR=com.openafis.spark.BiometricKryoRegistrator
SPARK_KRYO_UNSAFE=true                     # Enable unsafe operations
SPARK_KRYO_BUFFER_MAX=256m                 # Max Kryo buffer size

# Spark Network & Timeouts
SPARK_NETWORK_TIMEOUT=120s                 # Network timeout
SPARK_RPC_ASK_TIMEOUT=120s                 # RPC timeout
SPARK_TASK_MAX_FAILURES=3                  # Max task failures
SPARK_EXCLUDE_ON_FAILURE_ENABLED=false     # Disable node exclusion

# Spark Performance Tuning
SPARK_SQL_ADAPTIVE_ENABLED=true            # Enable adaptive query
SPARK_SQL_ADAPTIVE_COALESCE_PARTITIONS=true # Enable partition coalescing
SPARK_SPECULATION=false                    # Disable speculative execution
SPARK_DYNAMIC_ALLOCATION_ENABLED=false     # Fixed resource allocation
SPARK_LOCALITY_WAIT=3s                     # Data locality wait time

# Spark Fallback Configuration
ALLOW_FALLBACK_TO_LOCAL_VERIFICATION_IF_SPARK_UNAVAILABLE=true
ALLOW_FALLBACK_TO_LOCAL_IDENTIFY_IF_SPARK_UNAVAILABLE=true
SPARK_NUMBER_OF_CLUSTER=2                  # Number of Spark worker nodes

⚠️ Important: Spark Configuration Modes

  • Local Mode (Default): SPARK_MASTER_URL=local[*] - Runs on local machine using all CPU cores
  • Cluster Mode: SPARK_MASTER_URL=spark://localhost:7077 - Connects to Kubernetes Spark cluster
  • Version Compatibility: Application uses Spark 3.5.6 (matches bitnami/spark:3.5 image)
  • Serialization Issues: Different Spark distributions may have RPC incompatibilities. Use local mode if connection fails.
  • To Update: Change <spark.version>3.5.6</spark.version> in pom.xml to match your cluster version

Fingerprint Matching Configuration

# Matching Thresholds
FINGERPRINT_MATCH_THRESHOLD=40.0           # General match threshold
FINGERPRINT_HIGH_CONFIDENCE_THRESHOLD=60.0 # High confidence threshold
FINGERPRINT_VERIFICATION_THRESHOLD=30.0    # 1:1 verification threshold
FINGERPRINT_MIN_QUALITY_THRESHOLD=40.0     # Minimum image quality

# Template Processing
FINGERPRINT_MIN_TEMPLATE_SIZE=32           # Minimum template size (bytes)
FINGERPRINT_OPTIMAL_TEMPLATE_SIZE=1024     # Optimal template size (bytes)
FINGERPRINT_MIN_IMAGE_SIZE=32768           # Minimum image size (bytes)
FINGERPRINT_DEFAULT_TOP_N=5               # Default top N results

# Computation Methods
VERIFY_COMPUTATION_METHOD=local            # local|spark
IDENTIFY_COMPUTATION_METHOD=spark          # local|spark

Performance Tuning

# Queue System
QUEUE_WORKER_THREADS=4              # Parallel processing threads
QUEUE_BATCH_SIZE=10                 # Batch processing size
QUEUE_POLL_INTERVAL_MS=1000         # Queue polling interval

# Aerospike Timeouts
AEROSPIKE_READ_TIMEOUT=2000         # Read timeout (ms)
AEROSPIKE_WRITE_TIMEOUT=5000        # Write timeout (ms)
AEROSPIKE_MAX_CONNS_PER_NODE=100    # Connection pool size

Database Schema

Aerospike Sets

  • candidates - Candidate enrollment records
  • templates - Fingerprint templates and minutiae
  • async_tasks - Background job queue
  • match_results - Matching results and scores
  • locks - Distributed locking
  • indexes - Index tracking and metadata

Performance Indexes

The system creates 17 optimized secondary indexes:

  • High Priority: task_status, candidate_external_id, template_candidate_id
  • Performance: task_worker_instance, task_priority, task_queue_name
  • Cleanup: task_created_at, candidate_enrollment_date

📈 Monitoring & Observability

Application Metrics

  • Health Endpoints: Basic and detailed health checks
  • Performance Metrics: Response times, throughput, error rates
  • Queue Metrics: Job processing stats, queue depth, worker utilization
  • Database Metrics: Connection pool stats, query performance

Logging

  • Structured Logging: JSON format for machine processing
  • Multiple Outputs: Console, file, Elasticsearch
  • Log Levels: Configurable per component
  • Request Tracing: Full request lifecycle tracking

Visualization

  • Kibana Dashboards: Pre-configured log visualization
  • Grafana Integration: Performance metrics dashboards
  • Alerting: Configurable alerts for critical events

🧪 Testing

Running Tests

# Unit tests
mvn test

# Integration tests  
mvn verify

# Performance tests
mvn test -Dtest=PerformanceTests

API Testing

# Using curl
curl -X POST http://localhost:8085/candidates \
  -H "Content-Type: application/json" \
  -d '{"external_id":"test001","name":"Test User"}'

# Using HTTPie
http POST localhost:8085/candidates external_id=test001 name="Test User"

Load Testing

# Apache Bench
ab -n 1000 -c 10 http://localhost:8085/health

# JMeter test plans available in test/jmeter/

🤝 Contributing

Development Setup

  1. Fork the repository
  2. Create feature branch: git checkout -b feature/new-feature
  3. Follow coding standards (see CONTRIBUTING.md)
  4. Add tests for new functionality
  5. Submit pull request

Code Standards

  • Java 17 features and best practices
  • Clean Code principles
  • Comprehensive testing (unit + integration)
  • API documentation updates
  • Performance considerations

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

🆘 Support

Documentation

Community

Professional Support

For enterprise support, consulting, and custom development:

OpenAFIS - Building the future of biometric identification systems.

Made with ❤️ by the OpenAFIS Community

About

No description, website, or topics provided.

Resources

Readme

License

View license
Activity

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases

No releases published

Packages

No packages published

Languages