Log and event storage policy

1) Purpose and scope

Purpose: To provide legal, secure, and cost-effective log/event storage, support AML/KYC investigations, audits, reporting, and platform resilience.
Coverage: all environments (prod/stage/dev), applications and microservices, anti-fraud and payments, CCM/sanctions, RG, infrastructure (K8s/cloud/CDN/WAF), partners/vendors (PSP, KYC, anti-fraud, analytics).

2) Log classes and minimum field composition

1. Security (SecOps/Identity): authentication, ATO/anti-fraud signals, role and policy changes, access to PII.

Поля: `actor`, `subject`, `action`, `result`, `ip`, `device`, `geo`, `risk_score`, `trace_id`.

2. Transactions/payments: deposits/withdrawals, chargebacks, anti-fraud rules.

Поля: `tx_id`, `amount`, `currency`, `psp`, `status`, `rule_hits[]`, `evidence_ref`.
3. CCM/sanctions/PEP: initiations, results, provider/version of lists, decisions (true/false positive).
4. Operations/SRE: SLO metrics, releases, autocat, incidents, alerts.
5. Marketing/CRM (optional): opt-in/unsubscribe events, campaigns (no extra PII).
6. Data access audit: reading/exporting/deleting sets from PII; references to DSAR/AML cases.

3) Retention periods and retention levels (Hot/Warm/Cold/WORM)

4) Time synchronization and traceability

Single time base: NTP/Chrony, store 'ts _ utc' (UTC) + 'ts _ local' (for reporting).
Correlation: Include 'trace _ id '/' span _ id' and 'source _ service' in each log.
Time zones: reports/export - with explicit indication of TZ.

5) Access, encryption and segregation of duties

Encryption: at rest (KMS; key rotation at least 90 days for secret spaces) and in transit (TLS 1. 2+).
RBAC/ABAC: minimum access; separate roles for reading audit logs.
Break-glass: temporary access with multi-factor authorization and auto-closing.
Segmentation: logs with PII/finance - separate indices/tanks, separate keys.
Logs of access to logs: all reads/exports are recorded and reviewed.

6) Privacy and masking

It is strictly forbidden to log: passwords, tokens, PAN (in full), CVV/CVC, full document numbers, "raw" biometric data.
Default masking: email → 'p @ domain. com`; telephone → '+ XXX123'; IBAN/PAN → tokens/last 4 digits.
Aliasing: replace 'user _ id' with a strong token in analytical/marketing logs.
Cookies/SDK: log only technical identifiers with consent (CMP) and without gluing with PII, if there is no legal basis.
DSAR compatibility: store a reference to the source of the set and the ability to selectively extract/delete.

7) Data Quality and Formatting

Scheme-as-code: centralized JSON schemes/event protocols, versioning.
Validations: not null/ranges/regexes; rejected events - to a quarantine queue with a reason label.
Deduplication: by '(trace_id, ts, source)'; idempotency levels for retrays.
Enrichment: strictly deterministic; geo/device attributes - indicating the version of dictionaries.

8) Architecture and storage tiers

Hot: indexable storage/search clusters (operational investigations, SIEM).
Warm: object storage with accelerated access/colds.
Cold: object/archive storage (glacier class/analogue), requests via batch.
WORM/Legal Hold: unchangeable retention buckets/policies and "legal holds" with no removal/change before expiration.

9) Deletion, archiving and Legal Hold (SOP)

1. The daily scheduler calculates candidates by time.
2. Check for active incidents/investigations/Legal Hold.
3. Archiving - Migrate to Cold/WORM as needed.
4. Delete: safe purge + log ('dataset', 'range', 'actor', 'hash _ before/after').
5. Report to Compliance/Data at the end of the batch.

10) Compliance Integration (GDPR/AML/PCI/ISO)

GDPR: minimization, goals/bases in RoPA; DSAR availability; 72-hour notifications rely on audit logs.
AML: storage of logs of sanctions checks, STR/SAR links; terms of 5-10 years (by country).
PCI DSS (if applicable): disable sensitive authentication data; segregation of payment perimeter logs.
ISO 27001/ISMS: logging policy as a mandatory document; annual audits and tests.

11) Vendors and sub-processors

DPA/SLA: register retention periods, geography, TOMs, export format, WORM/Legal Hold, incident response time.
Audit: questionnaires, PII selective access logs, incident/notification test.
Offboarding: deletion/return of logs, closing act, confirmation of destruction of copies/backups.

12) Monitoring and alerts

KRIs: validation failure growth> X%, ingestion> Y lags, ETL failure <99%, out-of-window access attempts.
KPI: logging coverage ≥ 95% of services; MTTD of pipeline failure ≤ 15 min; The percentage of requests to Hot completed ≤ 2 seconds is ≥ 95%.
SOAR: auto-tickets in violation of retention/access/masking.

13) RACI

14) Export and reporting

White lists of recipients and formats (CSV/Parquet/JSON) with depersonalization by default.
Signature/hash of each archive, download log.
Regulatory report templates: summaries of sanctions/PEP, KYC, AML alerts, PII access, incidents.

15) Requirements for development and operation

Log meaningfully: key actions/decisions, not all traffic.
Level standards: 'DEBUG' is not allowed in prod; 'INFO' for business events; 'WARN/ERROR' for anomalies.
Redaction-middleware: a single layer of masking in gateways/SDKs.
Test environments: synthetic data or pseudonymization; disabling copies of prod-logs in dev.
Releases: log/masking checklist in CAB; feature flags for advanced logging.

16) Checklists

16. 1 Weekly monitoring

• Time synchronization without drift
• Ingestion errors
• No direct PII/secrets in samples
• Access/roles are up to date
• ETL success ≥ 99%

16. 2 Monthly audit

• Check retention/removals
• Random selection of exports (signature/hash ok)
• Vendor reviews (access logs, incidents)
• Updating Schemas/Reference Books

16. 3 Before deletion/archive

• No Legal Hold/Incident
• Export related artifacts (if required)
• Destruction protocol formed

17) Logging incidents (fast playbook)

PII/secrets were found in the logs → immediately enable redaction rules, restrict access, start key cleaning/rotation, estimate the scale (DPO/Legal), and, if necessary, notifications.
Failure of the pipeline of logs → switching to buffering, alert SRE, restart ingestion, post-mortem.

18) Implementation Roadmap

Weeks 1-2: inventory of sources, agreement on dates, basic retention matrix, scheme-as-code.
Weeks 3-4: masking/revision implementation, index separation with PII, NTP/trace identifiers, WORM for critical sets.
Month 2: automation of deletion/archiving, KRIs/KPIs and alerts, SOAR playbooks.
Month 3 +: vendor audits, cost optimization (tiering), quarterly reviews of deadlines and requirements of jurisdictions.

TL; DR

Unified log policy = clear timing matrix + masking and encryption + RBAC and access audit + WORM/Legal Hold + quality and time synchronization. This reduces risk (GDPR/AML/PCI), reduces storage costs and speeds up investigations.