Skip to content

Enterprise Content Moderation Pipeline ​

examples/support/content-moderation.halo policies |! people

A hardened moderation pipeline with multiple defense layers, audit trails, escalating human review, and budget-safe AI model selection. Shows how Halo handles compliance.

Open in the playground and play with it →

halo
// ============================================================
// Enterprise Content Moderation Pipeline
// ============================================================
// A hardened moderation pipeline with multiple defense layers,
// audit trails, escalating human review, and budget-safe
// AI model selection. Shows how Halo handles compliance.
// ============================================================

// --- GUARDRAILS: Hard boundaries ---
// Personal data in a submission is masked before any model reads it. Whether content is
// harmful is a judgment no pattern can make: the triage score and the review route decide it.
policy PIIProtection   : Redact(PII)
policy CostControl     : MaxSpend(2.00)

// --- TOOLS ---
tool ModerationAPI  : POST("api.openai.com/v1/moderations")
tool AuditLogger    : POST("api.example.com/audit", apiKey: Env.AUDIT_KEY)
tool EscalationQueue: POST("api.jira.com/rest/issue")

// --- MEMORY ---
tool PolicyRules    : VectorStore(Env.PINECONE_KEY, namespace: "content-policy-v3")
tool PastCases      : VectorStore(Env.PINECONE_KEY, namespace: "moderation-history")

// --- MINDS ---
// The triage agent is cheap — handles bulk content fast
agent TriageAgent [openai/gpt-4o-mini, temp: 0.0]
  system """
    You are a content triage specialist. Rapidly screen
    submissions for obvious policy violations. Classify as
    safe, borderline, or blocked. Optimize for throughput
    — most content should auto-clear in under 500ms.
  """
  requires PIIProtection, CostControl

// The deep-review agent handles edge cases with precision
@timeout(60s)
agent ReviewAgent [google/gemini-flash, temp: 0.1]
  system """
    You are a nuanced content review agent. Handle edge cases
    that triage couldn't auto-resolve. Consider cultural
    context, satire, and intent. Report risk level with
    detailed reasoning for borderline decisions.
  """
  requires PIIProtection, CostControl

// The policy expert knows regulations inside-out
agent PolicyExpert [anthropic/claude-sonnet-4, temp: 0.2]
  system """
    You are a regulatory compliance specialist. Verify content
    against platform policies and applicable regulations
    (GDPR, COPPA, DSA). Cite specific policy clauses in
    every decision. Never approve content you're uncertain about.
  """
  equips PolicyRules
  requires PIIProtection, CostControl

// --- The Pipelines ---
// Every submission gets exactly one outcome: the branch (|? { ... })
// sends it down the first route whose condition holds.
@timeout(5m)
@retry(3, 10s)
pipeline ModerateContent(submission: JSON) -> Result:
  submission
    // Layer 1: Quick triage scores every submission cheaply
    |> TriageAgent.screen(
        text: context.data.submission.body,
        user_flags: context.data.submission.user_flags
      ) as screen

    // Layer 2: Route it. Clearly fine content publishes, clearly toxic
    // content is removed, and everything in between gets a deep review.
    |? {
        (context.data.screen.toxicity_score < 0.3) => System.Publish(context.data.submission),
        (context.data.screen.toxicity_score > 0.9) => System.Notify("removed", reason: context.data.screen.reason),
        else => DeepReview(context.data.submission),
      } as outcome

    // Layer 3: Log every decision to the audit trail
    |> AuditLogger.record(
        event: "moderation_decision",
        screen: context.data.screen,
        outcome: context.data.outcome
      )
    |> System.Print(context.data.outcome)

// Ambiguous content: a careful model reviews it against policy.
@timeout(2m)
pipeline DeepReview(submission: JSON) -> Result:
  submission
    |> ReviewAgent.evaluate(
        content: context.data.submission.body,
        policy: PolicyRules,
        history: PastCases
      ) as review

    // High risk goes to a human; the rest gets a final policy check
    |? {
        (context.data.review.risk_level == "high") => HumanReview(context.data.submission, context.data.review),
        else => PolicyExpert.final_review(content: context.data.submission.body, review: context.data.review),
      }

// High-risk content: a human moderator makes the call.
pipeline HumanReview(submission: JSON, review: JSON) -> Result:
  review
    |! yield(
        queue: EscalationQueue,
        context: context.data.review,
        timeout: 1h
      )
    ~> listen(EscalationQueue.Reaction) as human
    |? {
        (context.data.human.action == "approve") => PolicyExpert.final_review(
            content: context.data.submission.body,
            review: context.data.review,
            moderator: context.data.human
          ),
        else => System.Notify("removed by moderator", review: context.data.review),
      }

curl -fsSL https://halolang.com/install.sh | sh