# Malware Website Crawler - Scripts

This directory contains utility scripts for managing the Malware Website Crawler deployment.

## Scripts Overview

### Deployment Scripts

- **`load_known_malware.py`** - Loads known malware signatures into the database
- **`teardown.sh`** - Complete AWS resource teardown (Linux/Mac)
- **`teardown.ps1`** - Complete AWS resource teardown (Windows PowerShell)

### Management Scripts

- **`selective-teardown.sh`** - Interactive selective resource teardown
- **`emergency-teardown.sh`** - Emergency teardown for immediate cost reduction
- **`cost-monitor.sh`** - AWS cost monitoring and optimization recommendations

## Usage

### Loading Malware Signatures

```bash
# Load known malware signatures into the database
python scripts/load_known_malware.py
```

### Complete Teardown

```bash
# Linux/Mac
./scripts/teardown.sh

# Windows PowerShell
.\scripts\teardown.ps1
```

### Selective Teardown

```bash
# Interactive menu for selective resource removal
./scripts/selective-teardown.sh
```

### Emergency Teardown

```bash
# ⚠️ WARNING: Force deletes all resources immediately
./scripts/emergency-teardown.sh
```

### Cost Monitoring

```bash
# Monitor AWS costs and get optimization recommendations
./scripts/cost-monitor.sh
```

## Script Details

### teardown.sh / teardown.ps1

**Purpose**: Complete removal of all AWS resources created by Terraform

**Features**:
- Confirmation prompt before destruction
- Terraform state cleanup
- Local file cleanup
- Resource summary

**Usage**:
```bash
./scripts/teardown.sh
```

### selective-teardown.sh

**Purpose**: Interactive selective removal of specific AWS resources

**Options**:
1. Application only (ECS, ALB, ECR)
2. Database only (RDS, Redis)
3. Storage only (S3 buckets)
4. Networking only (VPC, subnets, NAT gateways)
5. Security only (Security groups, IAM roles)
6. Everything except database
7. Everything except storage
8. Custom selection
9. Show resources again

**Usage**:
```bash
./scripts/selective-teardown.sh
```

### emergency-teardown.sh

**Purpose**: Aggressive cleanup when normal teardown fails

**⚠️ WARNING**: This script will:
- Force delete S3 buckets with all contents
- Force delete ECR repositories with all images
- Immediately stop all running services
- Delete all data without snapshots

**Usage**:
```bash
./scripts/emergency-teardown.sh
# Type 'EMERGENCY' to confirm
```

### cost-monitor.sh

**Purpose**: Monitor AWS costs and identify optimization opportunities

**Features**:
- Current month costs by service
- Resource counts and status
- Estimated daily/monthly costs
- Expensive resource identification
- Cost optimization recommendations

**Usage**:
```bash
./scripts/cost-monitor.sh
```

### load_known_malware.py

**Purpose**: Load known malware signatures from JSON file into database

**Features**:
- Reads from `known_malware.json`
- Updates existing signatures
- Creates new signatures
- Database transaction management

**Usage**:
```bash
python scripts/load_known_malware.py
```

## Prerequisites

### Required Tools

- **AWS CLI**: Configured with appropriate permissions
- **Terraform**: For infrastructure management
- **Python 3.11+**: For Python scripts
- **jq**: For JSON processing (optional, for better output formatting)
- **bc**: For calculations (optional, for cost estimates)

### AWS Permissions

The scripts require the following AWS permissions:

```json
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "ecs:*",
                "rds:*",
                "elasticache:*",
                "s3:*",
                "ec2:*",
                "iam:*",
                "logs:*",
                "ce:*",
                "ecr:*",
                "elbv2:*"
            ],
            "Resource": "*"
        }
    ]
}
```

## Safety Features

### Confirmation Prompts

All destructive scripts include confirmation prompts:
- `teardown.sh`: Requires "yes" confirmation
- `emergency-teardown.sh`: Requires "EMERGENCY" confirmation
- `selective-teardown.sh`: Interactive menu with clear options

### Resource Validation

Scripts validate resources before deletion:
- Check if resources exist
- Verify resource ownership
- Display resource counts before deletion

### Error Handling

All scripts include error handling:
- Graceful failure on missing resources
- Continue on non-critical errors
- Clear error messages and exit codes

## Best Practices

### Before Teardown

1. **Backup Important Data**
   ```bash
   # Create database snapshots
   aws rds create-db-snapshot --db-instance-identifier malware-crawler-db --db-snapshot-identifier backup-$(date +%Y%m%d)
   ```

2. **Monitor Costs**
   ```bash
   # Check current costs
   ./scripts/cost-monitor.sh
   ```

3. **Verify Resources**
   ```bash
   # List all resources
   terraform state list
   ```

### During Teardown

1. **Use Selective Teardown First**
   - Start with non-critical resources
   - Keep databases until last
   - Monitor costs during teardown

2. **Monitor Progress**
   - Watch AWS console during teardown
   - Check for any stuck resources
   - Verify costs are decreasing

### After Teardown

1. **Verify Complete Removal**
   ```bash
   # Check for remaining resources
   ./scripts/cost-monitor.sh
   ```

2. **Clean Up Local Files**
   - Remove Terraform state files
   - Clean up local environment files
   - Remove Docker images

## Troubleshooting

### Common Issues

1. **Permission Denied**
   - Check AWS CLI configuration
   - Verify IAM permissions
   - Ensure correct AWS profile

2. **Resource Dependencies**
   - Some resources have dependencies
   - Use `terraform destroy` for proper ordering
   - Check AWS console for stuck resources

3. **Script Execution Errors**
   - Ensure scripts are executable: `chmod +x scripts/*.sh`
   - Check for required tools (AWS CLI, Terraform)
   - Verify network connectivity

### Getting Help

1. **Check Script Logs**
   - Scripts provide detailed output
   - Look for error messages
   - Check AWS CloudTrail for API calls

2. **AWS Console**
   - Verify resource status in AWS console
   - Check for any stuck resources
   - Review billing for unexpected charges

3. **Terraform State**
   ```bash
   # Check Terraform state
   terraform state list
   terraform show
   ```

## Security Considerations

### Data Protection

- Scripts handle sensitive data (passwords, keys)
- Use environment variables for sensitive information
- Clean up temporary files after execution

### Access Control

- Scripts require appropriate AWS permissions
- Use least privilege principle
- Consider using AWS IAM roles instead of access keys

### Audit Trail

- All destructive operations are logged
- AWS CloudTrail tracks all API calls
- Scripts provide detailed output for auditing
