Skip to content

Commit a8561c5

Browse files
11PRIMUS11PRIMUS
authored
Merge pull request #197 from that-ar-guy/add/anamoly
Time-Series-Anomaly-Detection
2 parents 8cc3f04 + ec659bc commit a8561c5

File tree

2 files changed

+180
-0
lines changed

2 files changed

+180
-0
lines changed

‎docs/projects/deep-learning/anamoly-detection.md

Lines changed: 171 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,171 @@
1+
# 📜 Time-Series Anomaly Detection
2+
3+
<div align="center">
4+
<img src="https://fr.mathworks.com/help/examples/nnet/win64/TimeSeriesAnomalyDetectionUsingDeepLearningExample_08.png" />
5+
</div>
6+
7+
## 🎯 AIM
8+
To detect anomalies in time-series data using Long Short-Term Memory (LSTM) networks.
9+
10+
## 📊 DATASET LINK
11+
[NOT USED]
12+
13+
## 📓 KAGGLE NOTEBOOK
14+
[https://www.kaggle.com/code/thatarguy/lstm-anamoly-detection/notebook](https://www.kaggle.com/code/thatarguy/lstm-anamoly-detection/notebook)
15+
16+
??? Abstract "Kaggle Notebook"
17+
18+
</iframe>
19+
<iframe src="https://www.kaggle.com/embed/thatarguy/lstm-anamoly-detection?kernelSessionId=222020820" height="800" style="margin: 0 auto; width: 100%; max-width: 950px;" frameborder="0" scrolling="auto" title="lstm anamoly detection"></iframe>
20+
21+
## ⚙️ TECH STACK
22+
23+
| **Category** | **Technologies** |
24+
|--------------------------|---------------------------------------------|
25+
| **Languages** | Python |
26+
| **Libraries/Frameworks** | TensorFlow, Keras, scikit-learn, numpy, pandas, matplotlib |
27+
| **Tools** | Jupyter Notebook, VS Code |
28+
29+
---
30+
31+
## 📝 DESCRIPTION
32+
33+
!!! info "What is the requirement of the project?"
34+
- The project focuses on identifying anomalies in time-series data using an LSTM autoencoder. The model learns normal patterns and detects deviations indicating anomalies.
35+
36+
??? info "Why is it necessary?"
37+
- Anomaly detection is crucial in various domains such as finance, healthcare, and cybersecurity, where detecting unexpected behavior can prevent failures, fraud, or security breaches.
38+
39+
??? info "How is it beneficial and used?"
40+
- Businesses can use it to detect irregularities in stock market trends.
41+
- It can help monitor industrial equipment to identify faults before failures occur.
42+
- It can be applied in fraud detection for financial transactions.
43+
44+
??? info "How did you start approaching this project? (Initial thoughts and planning)"
45+
- Understanding time-series anomaly detection methodologies.
46+
- Generating synthetic data to simulate real-world scenarios.
47+
- Implementing an LSTM autoencoder to learn normal patterns and detect anomalies.
48+
- Evaluating model performance using Mean Squared Error (MSE).
49+
50+
??? info "Mention any additional resources used (blogs, books, chapters, articles, research papers, etc.)."
51+
- Research paper: "Deep Learning for Time-Series Anomaly Detection"
52+
- Public notebook: LSTM Autoencoder for Anomaly Detection
53+
54+
---
55+
56+
## 🔍 PROJECT EXPLANATION
57+
58+
### 🧩 DATASET OVERVIEW & FEATURE DETAILS
59+
60+
??? example "📂 Synthetic dataset"
61+
62+
- The dataset consists of a sine wave with added noise.
63+
64+
| Feature Name | Description | Datatype |
65+
|--------------|-------------|:------------:|
66+
| time | Timestamp | int64 |
67+
| value | Sine wave value with noise | float64 |
68+
69+
---
70+
71+
### 🛤 PROJECT WORKFLOW
72+
73+
!!! success "Project workflow"
74+
75+
``` mermaid
76+
graph LR
77+
A[Start] --> B{Generate Data};
78+
B --> C[Normalize Data];
79+
C --> D[Create Sequences];
80+
D --> E[Train LSTM Autoencoder];
81+
E --> F[Compute Reconstruction Error];
82+
F --> G[Identify Anomalies];
83+
```
84+
85+
=== "Step 1"
86+
- Generate synthetic data (sine wave with noise)
87+
- Normalize data using MinMaxScaler
88+
- Split data into training and validation sets
89+
90+
=== "Step 2"
91+
- Create sequential data using a rolling window approach
92+
- Reshape data for LSTM compatibility
93+
94+
=== "Step 3"
95+
- Implement LSTM autoencoder for anomaly detection
96+
- Optimize model using Adam optimizer
97+
98+
=== "Step 4"
99+
- Compute reconstruction error for anomaly detection
100+
- Identify threshold for anomalies using percentile-based method
101+
102+
=== "Step 5"
103+
- Visualize detected anomalies using Matplotlib
104+
105+
---
106+
107+
### 🖥 CODE EXPLANATION
108+
109+
=== "LSTM Autoencoder"
110+
- The model consists of an encoder, bottleneck, and decoder.
111+
- It learns normal time-series behavior and reconstructs it.
112+
- Deviations from normal patterns are considered anomalies.
113+
114+
---
115+
116+
### ⚖️ PROJECT TRADE-OFFS AND SOLUTIONS
117+
118+
=== "Reconstruction Error Threshold Selection"
119+
- Setting a high threshold may miss subtle anomalies, while a low threshold might increase false positives.
120+
- **Solution**: Use the 95th percentile of reconstruction errors as the threshold to balance false positives and false negatives.
121+
122+
---
123+
124+
## 🖼 SCREENSHOTS
125+
126+
!!! tip "Visualizations and EDA of different features"
127+
128+
=== "Synthetic Data Plot"
129+
![img](https://github.com/user-attachments/assets/e33a0537-9e23-4e21-b0e5-153a78ac4000)
130+
131+
132+
??? example "Model performance graphs"
133+
134+
=== "Reconstruction Error Plot"
135+
![img](https://github.com/user-attachments/assets/4ff144a9-756a-43e3-aba2-609d92cbacd2)
136+
---
137+
138+
## 📉 MODELS USED AND THEIR EVALUATION METRICS
139+
140+
| Model | Reconstruction Error (MSE) |
141+
|------------------|---------------------------|
142+
| LSTM Autoencoder | 0.015 |
143+
144+
---
145+
146+
## ✅ CONCLUSION
147+
148+
### 🔑 KEY LEARNINGS
149+
150+
!!! tip "Insights gained from the data"
151+
- Time-series anomalies often appear as sudden deviations from normal patterns.
152+
153+
??? tip "Improvements in understanding machine learning concepts"
154+
- Learned about LSTM autoencoders and their ability to reconstruct normal sequences.
155+
156+
??? tip "Challenges faced and how they were overcome"
157+
- Handling high reconstruction errors by tuning model hyperparameters.
158+
- Selecting an appropriate anomaly threshold using statistical methods.
159+
160+
---
161+
162+
### 🌍 USE CASES
163+
164+
=== "Financial Fraud Detection"
165+
- Detect irregular transaction patterns using anomaly detection.
166+
167+
=== "Predictive Maintenance"
168+
- Identify equipment failures in industrial settings before they occur.
169+
170+
171+

‎docs/projects/deep-learning/index.md

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,15 @@
1111
<p style="font-size: 12px;">📅 2025-01-10 | ⏱️ 10 mins</p>
1212
</div>
1313
</a>
14+
<!-- LSTM Autoencoder for Time Series Anomaly Detection -->
15+
<a href="anamoly-detection" style="padding: 0 2px 0 16px; background-color: rgba(39, 39, 43, 0.4); border: 1px solid rgba(76, 76, 82, 0.4); border-radius: 10px; box-shadow: 0 4px 8px rgba(0,0,0,0.1); overflow: hidden; transition: transform 0.2s; display: flex; align-items: center;">
16+
<img src="https://fr.mathworks.com/help/examples/nnet/win64/TimeSeriesAnomalyDetectionUsingDeepLearningExample_08.png" alt="" style="width: 300px; height: 150px; object-fit: cover; border-radius: 10px;" />
17+
<div style="padding: 15px;">
18+
<h2 style="margin: 0; font-size: 20px;">LSTM Autoencoder for Time Series Anomaly Detection</h2>
19+
<p style="font-size: 16px;">A deep learning approach to detect anomalies in time series data.</p>
20+
<p style="font-size: 12px;">📅 2025-02-12 | ⏱️ 10 mins</p>
21+
</div>
22+
</a>
1423

1524

1625
</div>

0 commit comments

Comments
 (0)