|
| 1 | +# Data Retention |
| 2 | + |
| 3 | +Coder supports configurable retention policies that automatically purge old |
| 4 | +Audit Logs, Connection Logs, and API keys. These policies help manage database |
| 5 | +growth by removing records older than a specified duration. |
| 6 | + |
| 7 | +## Overview |
| 8 | + |
| 9 | +Large deployments can accumulate significant amounts of data over time. |
| 10 | +Retention policies help you: |
| 11 | + |
| 12 | +- **Reduce database size**: Automatically remove old records to free disk space. |
| 13 | +- **Improve performance**: Smaller tables mean faster queries and backups. |
| 14 | +- **Meet compliance requirements**: Configure retention periods that align with |
| 15 | + your organization's data retention policies. |
| 16 | + |
| 17 | +> [!NOTE] |
| 18 | +> Retention policies are disabled by default (set to `0`) to preserve existing |
| 19 | +> behavior. The only exception is API keys, which defaults to 7 days. |
| 20 | +
|
| 21 | +## Configuration |
| 22 | + |
| 23 | +You can configure retention policies using CLI flags, environment variables, or |
| 24 | +a YAML configuration file. |
| 25 | + |
| 26 | +### Settings |
| 27 | + |
| 28 | +| Setting | CLI Flag | Environment Variable | Default | Description | |
| 29 | +| --------------- | ----------------------------- | --------------------------------- | ---------------- | ------------------------------------------------------------------------- | |
| 30 | +| Global | `--global-retention` | `CODER_GLOBAL_RETENTION` | `0` (disabled) | Default retention for all data types. Individual settings override this. | |
| 31 | +| Audit Logs | `--audit-logs-retention` | `CODER_AUDIT_LOGS_RETENTION` | `0` (use global) | How long to retain Audit Log entries. | |
| 32 | +| Connection Logs | `--connection-logs-retention` | `CODER_CONNECTION_LOGS_RETENTION` | `0` (use global) | How long to retain Connection Log entries. | |
| 33 | +| API Keys | `--api-keys-retention` | `CODER_API_KEYS_RETENTION` | `7d` | How long to retain expired API keys. | |
| 34 | + |
| 35 | +### Duration Format |
| 36 | + |
| 37 | +Retention durations support days (`d`) and weeks (`w`) in addition to standard |
| 38 | +Go duration units (`h`, `m`, `s`): |
| 39 | + |
| 40 | +- `7d` - 7 days |
| 41 | +- `2w` - 2 weeks |
| 42 | +- `30d` - 30 days |
| 43 | +- `90d` - 90 days |
| 44 | +- `365d` - 1 year |
| 45 | + |
| 46 | +### CLI Example |
| 47 | + |
| 48 | +```bash |
| 49 | +coder server \ |
| 50 | + --global-retention=90d \ |
| 51 | + --audit-logs-retention=365d \ |
| 52 | + --api-keys-retention=7d |
| 53 | +``` |
| 54 | + |
| 55 | +### Environment Variables Example |
| 56 | + |
| 57 | +```bash |
| 58 | +export CODER_GLOBAL_RETENTION=90d |
| 59 | +export CODER_AUDIT_LOGS_RETENTION=365d |
| 60 | +export CODER_API_KEYS_RETENTION=7d |
| 61 | +``` |
| 62 | + |
| 63 | +### YAML Configuration Example |
| 64 | + |
| 65 | +```yaml |
| 66 | +retention: |
| 67 | + global: 90d |
| 68 | + audit_logs: 365d |
| 69 | + connection_logs: 0s |
| 70 | + api_keys: 7d |
| 71 | +``` |
| 72 | +
|
| 73 | +## How Retention Works |
| 74 | +
|
| 75 | +### Background Purge Process |
| 76 | +
|
| 77 | +Coder runs a background process that periodically deletes old records. The |
| 78 | +purge process: |
| 79 | +
|
| 80 | +1. Runs approximately every 10 minutes. |
| 81 | +2. Processes records in batches to avoid database lock contention. |
| 82 | +3. Deletes records older than the configured retention period. |
| 83 | +4. Logs the number of deleted records for monitoring. |
| 84 | +
|
| 85 | +### Effective Retention |
| 86 | +
|
| 87 | +For each data type, the effective retention is determined as follows: |
| 88 | +
|
| 89 | +1. If the individual setting is non-zero, use that value. |
| 90 | +2. If the individual setting is zero, use the global retention value. |
| 91 | +3. If both are zero, retention is disabled (data is kept indefinitely). |
| 92 | +
|
| 93 | +### API Keys Special Behavior |
| 94 | +
|
| 95 | +API key retention only affects **expired** keys. A key is deleted only when: |
| 96 | +
|
| 97 | +1. The key has expired (past its `expires_at` timestamp). |
| 98 | +2. The key has been expired for longer than the retention period. |
| 99 | + |
| 100 | +Setting `--api-keys-retention=7d` deletes keys that expired more than 7 days |
| 101 | +ago. Active keys are never deleted by the retention policy. |
| 102 | + |
| 103 | +Keeping expired keys for a short period allows Coder to return a more helpful |
| 104 | +error message when users attempt to use an expired key. |
| 105 | + |
| 106 | +## Best Practices |
| 107 | + |
| 108 | +### Recommended Starting Configuration |
| 109 | + |
| 110 | +For most deployments, we recommend: |
| 111 | + |
| 112 | +```yaml |
| 113 | +retention: |
| 114 | + global: 90d |
| 115 | + audit_logs: 365d |
| 116 | + connection_logs: 0s # Use global |
| 117 | + api_keys: 7d |
| 118 | +``` |
| 119 | + |
| 120 | +### Compliance Considerations |
| 121 | + |
| 122 | +> [!WARNING] |
| 123 | +> Audit Logs provide critical security and compliance information. Purging |
| 124 | +> Audit Logs may impact your organization's ability to investigate security |
| 125 | +> incidents or meet compliance requirements. Consult your security and |
| 126 | +> compliance teams before configuring Audit Log retention. |
| 127 | + |
| 128 | +Common compliance frameworks have varying retention requirements: |
| 129 | + |
| 130 | +- **SOC 2**: Typically requires 1 year of audit logs. |
| 131 | +- **HIPAA**: Requires 6 years for certain records. |
| 132 | +- **PCI DSS**: Requires 1 year of audit logs, with 3 months immediately |
| 133 | + available. |
| 134 | +- **GDPR**: Requires data minimization but does not specify maximum retention. |
| 135 | + |
| 136 | +### External Log Aggregation |
| 137 | + |
| 138 | +If you use an external log aggregation system (Splunk, Datadog, etc.), you can |
| 139 | +configure shorter retention periods in Coder since logs are preserved |
| 140 | +externally. See |
| 141 | +[Capturing/Exporting Audit Logs](../security/audit-logs.md#capturingexporting-audit-logs) |
| 142 | +for details on exporting logs. |
| 143 | + |
| 144 | +### Database Maintenance |
| 145 | + |
| 146 | +After enabling retention policies, you may want to run a `VACUUM` operation on |
| 147 | +your PostgreSQL database to reclaim disk space. See |
| 148 | +[Maintenance Procedures](../security/audit-logs.md#maintenance-procedures-for-the-audit-logs-table) |
| 149 | +for guidance. |
| 150 | + |
| 151 | +## Disabling Retention |
| 152 | + |
| 153 | +Setting a retention value to `0` means "use global retention", not "disable". |
| 154 | +To disable all automatic purging, set global to `0` and leave individual |
| 155 | +settings at `0`: |
| 156 | + |
| 157 | +```yaml |
| 158 | +retention: |
| 159 | + global: 0s |
| 160 | + audit_logs: 0s |
| 161 | + connection_logs: 0s |
| 162 | + api_keys: 0s |
| 163 | +``` |
| 164 | + |
| 165 | +There is no way to disable retention for a specific data type while global |
| 166 | +retention is enabled. If you need to retain one data type longer than others, |
| 167 | +set its individual retention to a larger value. |
| 168 | + |
| 169 | +## Monitoring |
| 170 | + |
| 171 | +The purge process logs deletion counts at the `DEBUG` level. To monitor |
| 172 | +retention activity, enable debug logging or search your logs for entries |
| 173 | +containing the table name (e.g., `audit_logs`, `connection_logs`, `api_keys`). |
| 174 | + |
| 175 | +## Related Documentation |
| 176 | + |
| 177 | +- [Audit Logs](../security/audit-logs.md): Learn about Audit Logs and manual |
| 178 | + purge procedures. |
| 179 | +- [Connection Logs](../monitoring/connection-logs.md): Learn about Connection |
| 180 | + Logs and monitoring. |
0 commit comments