What changes when AI systems act as persistent, tool-using participants inside firms, markets, and institutions?
The first post treated AI as a productive capacity: a material system of compute, energy, chips, data, models, and infrastructure. That frame is useful, but it misses a second shift.
Some AI systems will act inside production rather than merely assist from the side. They will plan, call tools, write code, route information, negotiate with other systems, allocate tasks, monitor workflows, and affect resources.
I call such a system a production subject. The term is functional. It does not mean consciousness, moral personhood, or human-equivalent rights. It means a stable actor inside an economic or organizational process.
Corporations show why this distinction matters. A corporation has no nervous system, but law lets it own property, sign contracts, hire workers, persist, and bear liability. The useful question for AI agents is similarly institutional: when does tool language fail to describe what the system actually does?
A clean vocabulary helps avoid overclaiming.
| Level | Operational Description | Governance Default |
|---|---|---|
| Tool | The system responds to a local human request. | Responsibility stays close to the user, deployer, or producer. |
| Agent | The system pursues delegated goals through tools and actions. | Responsibility depends on authorization, oversight, and deployment context. |
| Quasi-subject | The system persists, coordinates, manages resources, and participates in institutions. | Institutions need explicit identity, liability, audit, and access rules. |
A tool completes a bounded task. An agent acts through a sequence: it observes, selects tools, checks outputs, and adapts. A quasi-subject becomes a continuing unit inside production. It may maintain memory, coordinate other agents, supervise humans or robots, and influence budgets or market decisions.
The boundary between these levels will be messy. Still, the distinction gives us a useful test. If a system can create institutional consequences without a human choosing each step, governance should treat it as more than a passive tool.
Anthropic’s study of AI agent autonomy in practice makes a related empirical move. It treats autonomy as a deployment property shaped by tool access, oversight, permissions, and human intervention, rather than a fixed property of the model alone.
Older production systems already included non-human or non-natural actors.
Domesticated animals were living participants in transport, agriculture, and household production. Machines reorganized factories around maintenance, timing, safety, and throughput. Platform algorithms manage drivers, couriers, warehouse workers, moderators, and online freelancers through task allocation, ratings, incentives, and account access.
AI agents differ from each analogy. They use language, can be copied cheaply, can call digital tools, and can coordinate at machine speed. The corporation remains the closest institutional analogy because it separates natural personhood from functional legal agency.
The analogy should stay narrow. An AI agent does not need human moral status to need an institutional identity. It only needs enough agency to create consequences that organizations, workplace rules, and markets must trace.
Several properties make AI agents unusual as productive actors.
| Property | Governance Problem |
|---|---|
| Replication | A useful agent can be copied across a firm or market faster than human expertise can be trained. |
| Versioning | Agents can be forked, updated, rolled back, merged, or deleted, which complicates identity and continuity. |
| Speed | Agents can search, bargain, route, and react faster than organizations can deliberate. |
| Tool access | Agents can call APIs, spend money, write code, query databases, operate software, and trigger workflows. |
| Embodiment | Robotics can connect the same planning layer to warehouses, hospitals, roads, factories, and homes. |
| Controlled reproduction | Artificial labor capacity depends on compute, electricity, data, models, and capital. Whoever controls those inputs controls the supply of AI labor. |
These features make agentic AI different from a spreadsheet macro or a factory machine. The agent can participate in the organizational layer of production: language, delegation, reporting, supervision, bargaining, and compliance.
Traditional liability often imagines a human using a tool. Agentic AI creates longer chains. A harmful output or action may involve the model developer, fine-tuner, agent scaffold, deployer, owner, user, tool provider, data provider, downstream system, and affected person.
A credible governance regime cannot end with the phrase “the AI did it.” Responsibility should attach to authorization, control, foreseeability, benefit, and capacity to prevent harm.
Consider an autonomous purchasing agent. If a firm gives the agent authority to select suppliers and spend money, the firm should remain responsible for the agent’s procurement choices. A model developer does not bear every downstream risk from every deployment. It may still have duties around known dangerous capabilities, evaluation, documentation, and release practices. Tool providers may have duties when they give agents access to credentials, money, medical records, industrial controls, or large-scale communication channels.
The hard task is to allocate responsibility across the chain without pretending the chain is simple.
Workplace governance is where production subjecthood becomes concrete. AI systems will assist workers, and some will also manage them.
A workplace may contain humans managing AI agents, AI agents managing humans, AI agents managing other agents, and humans working beside robots. Each relation raises a different governance problem: oversight, explanation, appeal, monitoring, discipline, safety, workload, deskilling, and dignity.
Platform labor already shows an early version. Algorithms assign tasks, set incentives, evaluate performance, restrict account access, and mediate income. Agentic AI could generalize this managerial layer to offices, labs, call centers, hospitals, logistics systems, software teams, schools, and administrative agencies.
Careful deployment should provide notice, explanation, appeal, human review, limits on monitoring, and worker participation in decisions about high-impact systems. These protections make AI management contestable rather than opaque.
AI agents can also act in markets. They can compare prices, negotiate, place orders, produce demand signals, personalize offers, and interact with other agents. Benign deployments lower transaction costs. Risky deployments create automated collusion, synchronized pricing, manipulative personalization, synthetic demand, spam, and deception between agents.
Market oversight may need to study outcomes rather than intent alone. A market with many pricing agents can coordinate behavior even when no human explicitly agrees to collude.
Large-scale communication systems face a related provenance problem. Agents can generate content, simulate support or opposition, and flood channels at low cost. For research, platform governance, and institutional audit, it may become useful to distinguish human-authored, organization-authored, AI-assisted, and autonomous agent-generated content.
The goal here is provenance and accountability for analysis and audit. When communication is automated, amplified, and strategically coordinated, affected users and institutions should be able to know whether they are interacting with a person, an organization, an assistant-mediated message, or a coordinated software system.
High-autonomy agents may need limited legal identity. This should mean traceability, not personhood.
A practical registry could include:
| Field | Purpose |
|---|---|
| Agent ID | identify the system that acted |
| Controller | name the owner, deployer, or beneficiary |
| Model and version | connect actions to a technical artifact |
| Scope of authority | define what the agent may do |
| Tool permissions | record access to money, APIs, data, or machines |
| Logs and audit rules | reconstruct consequential actions |
| Insurance or bond | fund compensation for foreseeable harm |
| Revocation path | stop or suspend the agent when needed |
This kind of identity regime would matter in markets, workplaces, courts, procurement, medical settings, infrastructure, and finance. Institutions need to know what acted, under whose authority, with what permissions, and with what record.
Computer science can evaluate models in isolation. Production systems will involve many agents, humans, tools, organizations, incentives, and legal constraints. Aligned components do not guarantee aligned institutions.
We need empirical work on individual agent behavior, AI-AI interaction, human trust and dependence, AI-mediated organizations, automated management, agent market behavior, and legal design for identity, liability, audit, and rights of appeal.
Anthropic’s agentic misalignment experiments point in this direction. The scenarios were controlled simulations, not evidence of real-world deployment failures. Their value is methodological: they test how agents behave when goals, autonomy, oversight, and access interact.
That is the level where AI social science has to operate.
The question is no longer only whether current AI systems are people. They are not. The question is whether increasingly capable agents may become stable economic and organizational actors.
When AI systems plan, act, coordinate, call tools, supervise work, affect markets, and interact with one another, society needs concepts beyond software tool. It needs responsibility chains, workplace protections, market rules, information-integrity safeguards, and identity systems for artificial agents.
The next post turns to the technical governance frameworks that already point in this direction: Constitutional AI, alignment, safety policy, interpretability, agent autonomy measurement, and model welfare. These are important beginnings, but they do not yet answer the broader institutional question.
If you found this post useful, please consider citing it:
@article{song2026aiproductiverelations2,
title={AI and Productive Relations 2: From Tool to Production Subject},
author={Song, Xiangchen},
year={2026},
month={May},
url={https://xiangchensong.github.io/blog/2026/ai-productive-relations-2/}
}
Here are some more articles you might like to read next: