# Malware Crawler - AWS Free Tier Deployment

This guide shows you how to deploy the malware crawler application on AWS using the **Free Tier** configuration, which is perfect for testing, learning, and small-scale deployments.

## 🆓 **AWS Free Tier Benefits**

- **$0.00 - $5.00/month** estimated cost
- **No Load Balancer** required (direct access via public IP)
- **Simplified architecture** using default VPC
- **Free tier eligible resources** only
- **Perfect for testing and development**

## 🏗️ **Free Tier Architecture**

```
┌─────────────────────────────────────────────────────────────┐
│                    AWS Free Tier Deployment                 │
├─────────────────────────────────────────────────────────────┤
│                                                             │
│  ┌─────────────────┐    ┌─────────────────┐                │
│  │   ECS Fargate   │    │   RDS Postgres  │                │
│  │   (0.25 vCPU)   │    │   (db.t3.micro) │                │
│  │   (0.5 GB RAM)  │    │   (20 GB)       │                │
│  └─────────────────┘    └─────────────────┘                │
│           │                       │                        │
│           │              ┌─────────────────┐                │
│           │              │  ElastiCache    │                │
│           │              │  Redis          │                │
│           │              │  (cache.t3.micro)│               │
│           │              └─────────────────┘                │
│           │                       │                        │
│  ┌─────────────────┐    ┌─────────────────┐                │
│  │   S3 Buckets    │    │   ECR Registry  │                │
│  │   (Files/Logs)  │    │   (Container)   │                │
│  └─────────────────┘    └─────────────────┘                │
│                                                             │
│  🌐 Direct Access via Public IP (No Load Balancer)         │
└─────────────────────────────────────────────────────────────┘
```

## 🚀 **Quick Start (Free Tier)**

### **Step 1: Prerequisites**
- AWS Free Tier account
- GitHub repository with the code
- Basic understanding of AWS services

### **Step 2: AWS Account Setup**
1. **Create AWS Account** (if you don't have one)
2. **Enable Free Tier** in AWS Console
3. **Create IAM User** with programmatic access
4. **Note down credentials** (Access Key ID, Secret Access Key)

### **Step 3: GitHub Repository Setup**
1. **Fork or clone** this repository
2. **Go to Settings → Secrets and variables → Actions**
3. **Add these secrets:**
   ```
   AWS_ACCESS_KEY_ID: your-access-key-id
   AWS_SECRET_ACCESS_KEY: your-secret-access-key
   VIRUSTOTAL_API_KEY: your-virustotal-api-key (optional)
   SHODAN_API_KEY: your-shodan-api-key (optional)
   ```

### **Step 4: API Services Setup (Optional)**
For enhanced malware detection, you can set up API keys:

#### **VirusTotal API (Recommended)**
1. **Go to** [VirusTotal](https://www.virustotal.com/gui/my-apikey)
2. **Create account** (free tier available)
3. **Get API key** from your account settings
4. **Add to GitHub secrets** as `VIRUSTOTAL_API_KEY`

#### **Shodan API (Optional)**
1. **Go to** [Shodan](https://account.shodan.io/)
2. **Create account** (free tier available)
3. **Get API key** from your account
4. **Add to GitHub secrets** as `SHODAN_API_KEY`

### **Step 5: Deploy via GitHub Actions**
1. **Go to Actions tab** in your repository
2. **Select "Deploy to AWS Free Tier" workflow**
3. **Click "Run workflow"**
4. **Choose "free-tier" environment**
5. **Click "Run workflow"**

### **Step 6: Access Your Application**
1. **Wait 5-10 minutes** for deployment to complete
2. **Check the workflow output** for the application URL
3. **Access your application** at: `http://[PUBLIC_IP]:5000`

## 📋 **Free Tier Configuration**

### **Resources Used (Free Tier Eligible)**
- **ECS Fargate**: 0.25 vCPU, 0.5 GB RAM
- **RDS PostgreSQL**: db.t3.micro, 20 GB storage
- **ElastiCache Redis**: cache.t3.micro
- **S3 Storage**: 5 GB standard storage
- **ECR**: 500 MB container storage
- **CloudWatch Logs**: 5 GB log storage

### **Free Tier Limits**
- **750 hours/month** ECS Fargate
- **750 hours/month** RDS db.t3.micro
- **750 hours/month** ElastiCache cache.t3.micro
- **5 GB** S3 standard storage
- **5 GB** CloudWatch logs

## 🔧 **Manual Deployment (Alternative)**

If you prefer to deploy manually:

### **Step 1: Configure Terraform**
```bash
cd terraform
cp free-tier.tfvars.example terraform.tfvars
# Edit terraform.tfvars with your settings
```

### **Step 2: Deploy Infrastructure**
```bash
terraform init
terraform plan
terraform apply
```

### **Step 3: Build and Push Container**
```bash
# Build Docker image
docker build -t malware-crawler .

# Tag for ECR
docker tag malware-crawler:latest [ACCOUNT_ID].dkr.ecr.[REGION].amazonaws.com/malware-crawler-app-free:latest

# Push to ECR
docker push [ACCOUNT_ID].dkr.ecr.[REGION].amazonaws.com/malware-crawler-app-free:latest
```

### **Step 4: Update ECS Service**
```bash
# Update ECS service with new image
aws ecs update-service --cluster malware-crawler-cluster-free --service malware-crawler-service-free --force-new-deployment
```

## 🌐 **Accessing Your Application**

### **Direct Access (No Load Balancer)**
- **URL**: `http://[PUBLIC_IP]:5000`
- **Port**: 5000
- **Protocol**: HTTP (no SSL/TLS in free tier)

### **Finding Your Public IP**
1. **AWS Console** → ECS → Clusters → malware-crawler-cluster-free
2. **Services** → malware-crawler-service-free
3. **Tasks** → Click on running task
4. **Network** → Public IP address

### **Application Features**
- **Web Interface**: Upload files, enter URLs to crawl
- **Malware Detection**: Scan files against known signatures
- **VirusTotal Integration**: Multi-engine malware detection
- **Shodan Integration**: IP intelligence and network analysis
- **Reports**: View scan results and findings
- **Database**: Store findings and malware signatures
- **Context Analysis**: Extract contextual information from websites

## 💰 **Cost Monitoring**

### **Free Tier Monitoring**
1. **AWS Console** → Billing → Free Tier
2. **Check usage** against free tier limits
3. **Set up billing alerts** for $5.00 threshold

### **Cost Optimization Tips**
- **Stop ECS service** when not in use
- **Delete unused resources** regularly
- **Monitor S3 storage** usage
- **Use CloudWatch** for resource monitoring

## 🔒 **Security Considerations**

### **Free Tier Limitations**
- **No SSL/TLS** (HTTP only)
- **No encryption** for RDS/Redis
- **No backups** enabled
- **Public IP access** (no private networking)

### **Security Best Practices**
- **Use strong passwords** for database
- **Limit access** to your IP addresses
- **Regular updates** of container images
- **Monitor access logs**

## 🛠️ **Troubleshooting**

### **Common Issues**

#### **"No statefile was found" Error**
```bash
# Run the fix script
./scripts/fix-no-statefile.sh
```

#### **ECS Task Not Starting**
1. **Check CloudWatch logs** for errors
2. **Verify ECR image** exists and is accessible
3. **Check security groups** allow traffic
4. **Verify task definition** is correct

#### **Database Connection Issues**
1. **Check RDS security groups** allow ECS access
2. **Verify database credentials** in secrets
3. **Check VPC configuration** (using default VPC)

#### **Application Not Accessible**
1. **Verify public IP** is correct
2. **Check security groups** allow port 5000
3. **Wait for container** to fully start (2-3 minutes)
4. **Check ECS service** is running

### **Getting Help**
1. **Check GitHub Actions logs** for detailed error messages
2. **Review CloudWatch logs** for application errors
3. **Verify AWS Free Tier** limits haven't been exceeded
4. **Check AWS Service Health** dashboard

## 📊 **Monitoring and Maintenance**

### **Health Checks**
- **ECS Service**: Automatic health checks
- **Application**: `/health` endpoint
- **Database**: Connection monitoring
- **Redis**: Connection monitoring

### **Logs and Monitoring**
- **CloudWatch Logs**: Application logs
- **ECS Service**: Task status and events
- **RDS**: Database performance metrics
- **S3**: Access logs and metrics

### **Maintenance Tasks**
- **Weekly**: Check resource usage
- **Monthly**: Review costs and optimize
- **Quarterly**: Update container images
- **As needed**: Clean up unused resources

## 🚨 **Important Notes**

### **Free Tier Limitations**
- **12 months** from account creation
- **Limited resources** (see AWS documentation)
- **No support** for free tier accounts
- **Automatic charges** after free tier expires

### **Production Considerations**
- **Not suitable** for production workloads
- **No high availability** or redundancy
- **Limited security** features
- **No backup** or disaster recovery

### **Upgrading from Free Tier**
When ready to upgrade:
1. **Use the full deployment** configuration
2. **Enable encryption** and security features
3. **Add load balancer** for production access
4. **Implement backup** and monitoring

## 🎯 **Next Steps**

1. **Deploy the application** using the free tier configuration
2. **Test the functionality** with sample files and URLs
3. **Monitor costs** and usage in AWS Console
4. **Explore the codebase** to understand the architecture
5. **Consider upgrading** to production configuration when ready

## 📚 **Additional Resources**

- [AWS Free Tier Documentation](https://aws.amazon.com/free/)
- [ECS Fargate Pricing](https://aws.amazon.com/fargate/pricing/)
- [RDS Free Tier](https://aws.amazon.com/rds/free/)
- [S3 Free Tier](https://aws.amazon.com/s3/pricing/)
- [CloudWatch Free Tier](https://aws.amazon.com/cloudwatch/pricing/)

---

**Happy crawling! 🕷️🔍**
