Why this post (and why now)
Last week’s post was beginner-friendly. This week, we turn the dial up. If you already know what billing is, you’re probably wondering: what actually happens inside the billing engine after the network spits out usage records? How do those raw records turn into a clean, compliant invoice that a customer can understand and pay?
Think of the billing engine as a factory production line.
-> Raw materials (usage data) come in through one door.
-> Machines (mediation, rating, billing, doc generation) transform, assemble, polish, and package that data.
-> Finally, finished goods (invoices) roll out, ready to be delivered by truck (email, portal, print/post, or e-invoicing rails).
Across the industry, the blueprint is remarkably consistent—
whether you’re running a large suite from a well-known vendor (Oracle BRM, Amdocs, Netcracker, etc.) or a modern microservices-based stack. We’ll keep this article generic and pattern-focused, but I’ll name-check those systems to anchor the conversation in reality.
What you’ll get
- A story-driven walkthrough: Mediation → Rating → Billing → Document generation → Invoice delivery
- Diagrams, Pseudo Code, typical input and output
- A worked example that shows exactly how a single call becomes a priced charge, then a bill, then a formatted invoice
- Practical tips for scale, accuracy, and operations
Quick recap: Billing 101 (the 60-second refresher)
- Mediation: Collects records from networks, cleans and standardizes them.
- Rating: Applies price rules to usage to compute charges.
- Billing: Aggregates charges, adds recurring/one-time fees, proration, taxes, discounts, shares, and produces a bill run.
- Document generation: Formats the bill into a human-friendly invoice (PDF/HTML).
- Delivery: Sends the invoice via email, portal, print, EDI/e-invoicing networks.
- Post-billing (adjacent): Payments, adjustments, dunning, GL posting, revenue recognition.
The factory analogy: Your mental model
- Raw materials: CDRs/EDRs/XDRs/usage events (voice minutes, data bytes, SMS count, content purchases).
- Assembly line:
- Gate 1: Mediation (quality control, cleaning, labeling).
- Station 2: Rating (pricing, discounts).
- Station 3: Billing (packaging, summarization, taxes).
- Station 4: Document generation (box design with brand and legal language).
- Station 5: Logistics (delivery to customer).
- Warehouse: Data stores for audit, rerating, dispute resolution, reporting.
The data backbone: who’s who in the model
- Customer: The human or business consuming services.
- Billing Account: Where invoices are issued and payments applied; may group multiple subscriptions.
- Subscription/Service: The thing being charged (SIM, number, data plan, enterprise circuit).
- Resource: Identifiers (IMSI, MSISDN, IP, MAC).
- Product/Offer: What was sold; links to price rules.
- Price/Discount: How to charge (tiers, bundles, promotions).
- Tax profile: Jurisdiction, exemption flags, VAT/GST settings.
- General Ledger (GL) mapping: Where revenue posts in finance.
Deep dive: Each station on the line
1) Mediation — quality control at the factory gate
What it ingests
- Batch: CDR/EDR files from MSCs/PGWs/IMS, typically CSV, ASN.1, TAP, or vendor-specific formats, via SFTP or object storage.
- Stream: Real-time events via Kafka/HTTP from network functions or charging probes.
What it does
- Parsing: Turn vendor formats into a normalized schema (e.g., JSON/Avro/Parquet).
- Deduplication: Detect duplicates by composite keys (switch_id, record_id, start_time, duration).
- Enrichment:
- Subscriber lookup: Map MSISDN/IMSI to billing account, subscription ID.
- Product mapping: Which price plan was active at event time?
- Geozone/roaming: Home vs visited network, zone-based pricing.
- Timezone normalization: Event timestamps → billing timezone or UTC.
- Correlation: Session start/stop pairs, multi-part CDR stitching.
- Validation: Field sanity checks, negative duration, future time, impossible IMEI patterns.
- Routing: Send valid, enriched records to rating; quarantine bad ones for RA and remediation.
- Idempotency: Every record gets a unique deterministic ID; retries don’t create double charges.
Mini example:
A raw voice CDR becomes a normalized usage record
Raw (CSV from switch):
"msc42","a1b2c3","2025-09-10T10:05:03+02:00","MSISDN=447700900123","DEST=442079460123","DURATION=187","CALL_TYPE=MO"
Normalized (JSON/Avro-like):
{
"event_id": "msc42-a1b2c3",
"timestamp_utc": "2025-09-10T08:05:03Z",
"service_type": "voice",
"direction": "outgoing",
"subscriber": {
"msisdn": "447700900123",
"subscription_id": "SUB-983420",
"billing_account_id": "BA-13877"
},
"usage": {
"destination": "442079460123",
"duration_seconds": 187
},
"context": {
"zone": "domestic",
"roaming": false,
"network_element": "msc42"
},
"hash": "sha256:..."
}
Common pitfalls and fixes
- Late-arriving CDRs: Use event-time processing with a watermark window; feed late records to a rerating lane.
- Clock skew: Normalize to UTC and record original timezone; adjust for DST with Olson/IANATZ tables.
- Exploding file formats: Keep parsers versioned; add feature flags for vendor schema changes.
- Scale: Partition by subscription_id or event_id hash to parallelize without cross-node duplicates.
2) Rating — pricing brain and balance keeper
Inputs and dependencies
- Normalized, enriched usage events from mediation.
- Product catalog and price rules: tiers, bundles, ladders, time-of-day, roaming zones.
- Discounts and allowances: buckets of free units, rollover logic.
- Currency, rounding, tax applicability (for pre-tax vs post-tax discounts).
- GL mapping: revenue chart-of-accounts, deferred vs recognized rules.
What rating does
- Determines the applicable price plan given event time.
- Applies rating rules:
- Units: seconds to billable minutes (ceil, round, step).
- Tiers: first N minutes at $X, next at $Y.
- Bundles/allowances: decrement free units before charging overage.
- Special cases: on-net vs off-net, short codes, roaming, international zones.
- Produces a rated event:
- Base charge, discounts applied, net charge.
- Units consumed from allowances.
- Audit trail: rule IDs used, versioning, exact formula path.
- Rerating:
- When: late CDRs, price plan backdating, discount correction.
- How: mark original charges as reversed, produce corrected rated events; keep idempotent keys.
Example: rating pseudocode (readable and generic)
function rate(event, price_plan, allowances):
billable_units = ceil(event.usage.duration_seconds / 60) / per-minute billing
remaining_free = allowances.voice_minutes.remaining
free_applied = min(billable_units, remaining_free)
chargeable_units = billable_units - free_applied
base_rate = price_plan.domestic_voice_per_minute // e.g., 0.05
base_charge = chargeable_units * base_rate
discounts = sum(apply_discounts(event, price_plan, chargeable_units, base_charge))
net_charge = max(base_charge - discounts, 0)
update_allowance(allowances.voice_minutes, free_applied)
return {
rated_event_id: deterministic_id(event),
subscription_id: event.subscriber.subscription_id,
billable_units: billable_units,
free_applied: free_applied,
chargeable_units: chargeable_units,
base_charge: base_charge,
discounts: discounts,
net_charge: net_charge,
currency: "GBP",
gl_code: "REV.VOICE.DOM",
audit: { rules: [...], versions: [...] }
}
Worked rating calculation (concrete numbers)
- Event: 187 seconds outgoing domestic voice.
- Rounding: Per-minute, ceiling → 187 sec → 4 minutes.
- Allowance: 100 free minutes/month; 98 already used → 2 minutes remaining.
- Price: £0.05 per minute.
- Free applied: 2 minutes → 0 charge on those.
- Chargeable minutes: 2 → 2 x £0.05 = £0.10 base.
- Discounts: none.
- Net charge: £0.10.
3) Billing — assembling the monthly bill
Inputs
- Rated events during the billing window (e.g., 2025-09-01 to 2025-09-30).
- Recurring charges (monthly plan fees), one-time charges (activation), credits/adjustments.
- Tax configuration (jurisdiction, tax type, exemptions).
- Billing account preferences (bill cycle day, invoice split/merge rules, currency, language).
What billing does
- Bill cycle control:
- Pre-bill checks: open disputes, unresolved rating errors, data completeness.
- Cycle freeze: lock price plans/products for the bill period to ensure consistency.
- Aggregation:
- Group rated events by subscription and charge type.
- Summarize as line items (e.g., “Domestic voice calls: 123 minutes, £6.15”).
- Provide itemization granularity preferences (full detail vs summary).
- Recurring and one-time charges:
- Proration for mid-cycle changes (start/stop dates).
- Pro-rate discounts and taxes accordingly.
- Discounts at bill level:
- Family or enterprise pooled discounts (e.g., 10% off all lines).
- Spend-based credits.
- Taxes:
- Calculate VAT/GST/sales tax based on jurisdiction and taxable basis (pre/post discount).
- Itemized tax lines or tax-included pricing depending on locale.
- Invoice composition:
- Opening balance and payments since last bill.
- Current charges: usage + recurring + one-time.
- Taxes and surcharges.
- Total due, due date, currency.
- Rerating and adjustments:
- If late events arrive after bill run, decide on: add to next bill, create adjustment on current account, or perform off-cycle rebill (shadow invoice and credit/debit note).
- Financial posting:
- Map revenue to GL codes.
- Create AR entries: debit AR, credit revenue/tax.
- Support IFRS 15/ASC 606 for allocation of bundled products (if applicable).
Billing example continued
Given our £0.10 rated call above, suppose:
- Monthly plan fee: £15.00 (pro-rated to full month).
- SMS usage: £0.40 (after allowances).
- Taxes: VAT 20% on taxable subtotal £15.50 → £3.10 VAT.
- Previous balance: £0; payments: £0.
Bill summary
- Recurring charges: £15.00
- Usage charges: £0.10 (voice) + £0.40 (SMS) = £0.50
- Subtotal: £15.50
- VAT 20%: £3.10
- Total due: £18.60
4) Document generation — turning a bill into an invoice
Inputs
- Bill run output: line items, totals, taxes, account metadata.
- Templates: branding, language, legal text, regulatory disclosures.
- Localization: currency symbols, decimal separators, date formats.
What it does
- Templating: Merge data with templates to produce PDF/HTML/XML.
- Dynamic sections:
- Itemization details per customer preference.
- Usage graphs, plan summaries, discount highlights.
- Legal and compliance:
- Invoice numbering schemes per jurisdiction.
- Required fields (VAT ID, GSTIN, QR codes for e-invoicing where mandated).
- Accessibility: Proper fonts, alt text for online portals, readable color contrast.
Common template tech choices (keep generic)
- Templating engines with HTML+CSS to PDF (wkhtmltopdf/WeasyPrint class of tools).
- Report designers (JasperReports/BIRT class of tools).
- XSL-FO pipelines if your data is XML-centric.
5) Invoice delivery — getting it to the customer securely
Channels
- Email: PDF attachments with secure links; SPF/DKIM/DMARC configured; bounce tracking.
- Self-care portal/app: HTML view + downloadable PDF; API endpoints.
- Print and post: Print house integration with barcodes and mail sorting.
- EDI/e-invoicing: B2B formats (EDIFACT/UBL/Peppol BIS); country-specific e-invoice rails where mandated.
Operational best practices
- Delivery preferences at account level; suppress paper where consented.
- Encryption at rest and in transit; password-protected PDFs if required.
- Retry and bounce handling; update contact info automatically when bounces persist.
- Track opens/clicks (privacy-aware), measure delivery success rates.
An end-to-end worked example (single event to invoice)
1. Network emits voice CDR at 10:05 local time; mediation ingests at 08:06 UTC.
2. Mediation normalizes, enriches with subscription BA-13877; assigns event_id msc42-a1b2c3.
3. Rating finds active plan “Family 15GB + 100 min”; two free minutes remain; applies:
- Billable units: ceil(187/60)=4
- Free applied: 2
- Chargeable: 2
- Rate: £0.05/min → £0.10 net
4. Billing cycle closes on the 1st:
- Adds recurring plan fee £15.00
- Aggregates usage: Voice £0.10, SMS £0.40
- Subtotal £15.50; VAT 20% £3.10; total £18.60
5. Document generator merges data into template (English, GBP, UK VAT format).
6. Delivery sends PDF via email and posts HTML to portal; log successes; AR and GL entries posted.
Resilience, scale, and accuracy: how to keep the line humming
- Throughput and partitioning:
- Partition by subscription_id for both mediation and rating; keeps ordering per subscriber and improves cache locality for allowances.
- Use backpressure-aware streaming; for batch, size files to fit memory for vectorized parsing.
- Idempotency everywhere:
- Deterministic event_id; Include it in downstream unique indexes; safe retries without double charging.
- Late data and rerating:
- Keep a rerating pipeline. Strategy: reverse-and-reapply rather than mutate in place. Always preserve original rated event for audit.
- Time and timezone hygiene:
- Store event timestamp as UTC plus original tz offset; evaluate price rules in customer’s billing timezone if rules are time-of-day sensitive.
- Testing:
- Golden datasets: build a suite of synthetic CDRs with expected outcomes; version them with your catalog/rules.
- Property-based tests: assert invariants (e.g., discounts never make net charge negative; taxes respect rounding rules).
- Observability:
- KPIs: ingestion lag, dedupe rate, rating exceptions, rerate volume, bill reject rate, delivery success %, dispute volume.
- Tracing: include event_id across all logs; you should be able to trace a single call from network to invoice line.
- Revenue assurance controls:
- Reconcile counts: network CDR count vs mediation accepted vs rated vs billed.
- Value checks: compare expected revenue (based on traffic models) with billed revenue by segment.
- Sampling: itemize a random sample of invoices monthly for manual verification.
Compliance and finance alignment (generic patterns)
- Tax compliance: keep jurisdiction logic externalized and versioned; always store tax basis and calculated tax separately.
- Invoice numbering: deterministic, non-reusable sequence per legal entity; allocate ranges per system instance to avoid collisions.
- GL posting:
- Usage revenue: recognized at event time; billing creates AR entries and maps revenue to GL.
- Bundled contracts: if you must allocate, do it in billing with allocation rules fed by the product catalog.
Rerating in practice: two common cases
- Late event: A roaming CDR appears after bill run. Add it to next bill as “Late usage for Sep 2025” or generate a one-line adjustment on the current account depending on your policy.
- Catalog correction: A discount was misconfigured. Run a controlled rerate: reverse impacted rated events, recalculate, adjust affected invoices via credit/debit notes; never overwrite history.
A note on real-world systems
Large, widely deployed systems such as Oracle BRM, Amdocs, and Netcracker each implement versions of these patterns under the hood. Names for components and specific configuration screens vary, but the flow—mediation, rating, billing, doc gen, delivery, with rerating and RA around the edges—is consistent. This article stays product-agnostic so you can apply the concepts in any stack.
Design choices that shape your implementation
- Batch vs streaming:
- Batch: simpler, great for postpaid; daily files; mediate → rate overnight → bill monthly.
- Streaming: necessary for near-real-time balances, bill shock prevention, and prepaid experiences.
- Convergent charging vs parallel stacks:
- One engine for voice/SMS/data/content vs separate verticals. Convergent simplifies customer experience but needs solid domain modeling.
- Allowances ledger:
- Central shared ledger for buckets (minutes, data) across devices/subscriptions vs per-subscription. Central makes family/enterprise pooling easier.
- Itemization level:
- Full CDR itemization inflates invoice size and can confuse consumers; often better in portal detail with invoice showing summaries.
A short but practical SQL
Assume rated_events table and bill_items table.
-- summarize rated events by charge key
insert into bill_items (bill_run_id, billing_account_id, subscription_id, charge_type, quantity, amount, currency)
select
:bill_run_id,
re.billing_account_id,
re.subscription_id,
re.charge_type, -- e.g., VOICE.DOMESTIC
sum(re.units) as quantity, -- e.g., minutes
sum(re.net_charge) as amount,
re.currency
from rated_events re
where re.event_time between :cycle_start and :cycle_end
and re.status = 'FINAL'
group by re.billing_account_id, re.subscription_id, re.charge_type, re.currency;
Then apply recurring fees, taxes, discounts, and produce invoice headers.
Performance and cost guardrails
- Storage format: Keep raw files in original format for audit; normalized in columnar (Parquet) for analytics; operational store in a relational or key-value DB.
- Caching: Warm caches with price plans and discount rules in the rating tier; version configs and include rule version in audit fields.
- Compute: Scale out rating stateless workers; keep allowance updates atomic (optimistic concurrency with retry).
- Cost: Archive old CDRs to cold storage; keep hot window small for fast rerates.
Common edge cases you’ll definitely meet
- Partial events: dropped sessions; apply minimum charge rules and cap with session controls.
- Zero-priced usage: still billable if it consumes allowance; ensure proper GL posting for usage value even if £0.
- Number portability: rating rules may depend on current carrier of destination; keep up-to-date portability databases.
- Roaming taps and taxation: ensure correct tax jurisdiction with place-of-supply rules; use visited network indicators and subscriber’s tax profile together.
A compact checklist for your first implementation
- Mediation:
- Define normalized schema with mandatory fields (event_id, timestamps, subscriber, usage metrics).
- Build dedupe with composite keys and persistent memory (e.g., a small TTL cache + durable store).
- Rating:
- Price rule engine with versioning; start with simple ladders and allowance decrement.
- Idempotent rated_event_id derived from event_id + rule_version.
- Billing:
- Clear bill cycle boundaries; support backdating and prorations from day one.
- Tax calculation modularized and versioned; store basis and result.
- Doc and Delivery:
- Templates with localization switches; keep legal text as content, not code.
- Email deliverability (SPF/DKIM/DMARC) and secure portal access in place before go-live.
- Controls:
- End-to-end reconciliation reports; alert when counts/values deviate.
- Shadow invoices in pre-production to calibrate before first real bill.
Summary: what to remember
- The billing engine is a disciplined production line. Each station—mediation, rating, billing, document generation, delivery—has clear inputs, outputs, and quality gates.
- Accuracy and auditability are as important as speed. Idempotency, rerating strategy, and clean time handling pay off forever.
- The “shape” of the solution is similar across the industry, regardless of vendor. If you understand the patterns, you can navigate any product.
- Start simple, instrument everything, and keep business rules versioned and testable.
Comments
Post a Comment