Why Intelligence Cannot Be Hard-Coded

Learning Series: Foundations of Smart Surveillance

Previous: https://varsity.thopps.com/what-looks-normal-until-it-isnt

How surveillance systems decide what actually matters


At some point, every intelligent system faces the same question:

“I see something unusual… now what?”

Detection can identify objects.
Tracking can follow movement.
Behaviour analysis can spot deviation.

But none of these can decide whether action is required.

That decision happens somewhere else.

And most people don’t realize where.

The moment intelligence actually begins

Imagine this situation:

A person enters a restricted corridor.

The system detects them instantly.

But should it alert?

What if:

  • they’re authorized?
  • they’re just passing through?
  • they exit within five seconds?

Triggering an alert immediately would be wrong.

Waiting too long would also be wrong.

This tension is where intelligence truly begins.

Rules: the first layer of decision-making

Early surveillance systems relied entirely on rules.

Simple logic such as:

  • if presence > 10 minutes
  • if entry occurs outside working hours
  • if movement crosses restricted line

These rules still power many real deployments today.

For example:

  • factories use rule-based zone violations
  • offices use time-based access rules
  • warehouses use line-crossing for safety

Rules work because they are:

  • predictable
  • explainable
  • easy to audit

But rules alone are fragile.

When rules start to fail…

Now imagine this:

A delivery person enters a loading area.

Stays for 12 minutes.

Leaves.

The rule says “alert after 10 minutes.”

But the behaviour was normal.

The system followed the rule —
but failed the situation.

This is where systems must evolve.

Introducing reasoning through state

Instead of asking only what happened,
systems begin tracking what stage something is in.

This is called state.

For example:

  • entered
  • waiting
  • unloading
  • exiting
  • resolved

Now the system understands sequence.

Not just duration — but progression.

This is how modern systems avoid premature alerts.

Reasoning emerges from relationships

Intelligence doesn’t come from one signal.

It comes from combining many weak signals.

Examples:

  • time + location
  • movement + direction
  • repetition + history

Individually, each signal is harmless.

Together, they tell a story.

This is how systems detect:

  • tailgating in secure doors
  • repeated access attempts
  • loitering near sensitive zones

None of these rely on a single detection.

They rely on relationships over time.

How this works in real systems today

In real-world deployments, reasoning layers are often built using:

  • rule engines written in Python
  • state machines tracking object lifecycle
  • Redis for short-term memory
  • event pipelines using Kafka or Redis Streams

Detection feeds data.

Reasoning interprets it.

This separation is intentional — and critical.

Why reasoning must stay outside the model

You could try to train a model to decide everything.

But then:

  • rules become invisible
  • explanations disappear
  • policies become unchangeable

That’s dangerous.

Which is why enterprise systems keep reasoning explicit.

When an alert fires, operators must know why.

Transparency is part of intelligence.

Why smart systems hesitate…

Good systems do something surprising.

They hesitate.

They wait for confirmation.
They observe persistence.
They validate sequence.

This hesitation prevents noise.

It’s why mature systems feel calm —
and immature ones feel chaotic.

Final Reflection

Rules define boundaries.

Reasoning understands situations.

True intelligence does not react instantly — it evaluates context before acting.

This is the difference between automation
and decision-making.

Even with good reasoning, many systems still fail.

Not because they detect incorrectly — but because they alert too much.

In the next article, we’ll explore why alerts fail in the real world, how false positives erode trust, and how mature systems reduce noise before acting.

Next in series: The Cost of Too Many Alerts

Hridya Syju
Hridya Syju