How Much Power Should We Give Our AI Partners?

‍

A discussion article. Part of The Thinking Upgrade.

We each spend every moment of our lives with ourselves. Other people experience fragments of us: a conversation, a meeting, a meal, a disagreement, a shared achievement. We live with the continuous stream of thoughts, feelings, emotions, sensations, hopes and doubts behind those moments.

Even those closest to us cannot experience our inner worlds as we do. They may never know how much effort something took, what triggered our reaction, what we were afraid to say, or which competing responsibilities were occupying our minds.

And we are all human. No one is perfect. No one knows everything. We have limited time and energy, different capabilities, our own agendas, beliefs and values, and habits that can help us or hold us back. We can be thoughtful in one moment and careless in the next. We can sincerely want to improve while repeatedly doing things that undermine that intention.

‍

What if we could have AI Partners that helped us make more of our lives – with the awareness, patience and availability to support us when we needed it? And how much power would we be willing to give them?

‍

Imagine a future in which wearables could augment our sight, hearing, smell, taste and touch, alongside sensors on and within our bodies. Connected to advanced AI Partners, they could help us notice dangers, understand patterns, prepare for challenges and make more informed choices. With our permission, that support might be available around the clock.

This is a possibility to explore, with capabilities and reliability still to be established. Even an extensively equipped system would have an incomplete view of us. Sensing a change in our body would not establish what we were feeling or why. More information would still require careful interpretation.

Yet the potential is considerable. We could receive help across the many roles, tasks and activities that make up our lives: caring for others, leading a team, learning a skill, buying a service, managing our energy or deciding what to do next. Our AI Partners could help us examine the effects of our thinking, conversations, decisions, actions and inactions on ourselves, other people and the world around us.

Consider the gap between what we intend and what we actually do. ‍

We want to listen patiently, then interrupt. We want to look after ourselves, then overlook our own needs. We want to use our time well, then spend hours on something that matters very little. Familiar routines can carry us through a day without our questioning whether they still serve us.

An AI Partner might quietly ask: “You have said this relationship matters to you. Would you like a moment before you reply?” Or: “You have been refining this task for an hour. Is it now good enough for its purpose?” It might notice the conversation we keep avoiding, the concern we never raise or the opportunity we repeatedly postpone.

Such prompts could help us act on our intentions. They could also irritate us. Few of us enjoy being told what to do, particularly when we are tired, defensive or convinced we are right.

Would we give our AI Partners permission to challenge us when their advice was something we would rather not hear?

We might welcome help managing time and energy, but who would define a worthwhile use of either? Rest, play, a long conversation and an afternoon without a plan all have a place in a full life. An AI Partner could just as usefully ask, “When did you last allow yourself to stop?” as remind us of an unfinished task. Its understanding of 'best possible' would need to reflect the whole person, including our circumstances, preferences and responsibilities.

The same care would be needed around what we put into and onto our bodies. Imagine support that helped us understand ingredients, compare options, reflect on sleep and movement, and recognise patterns worth discussing with a qualified professional. It could bring our personal goals into view alongside affordability, culture, convenience and environmental impact. It would also need to distinguish a sensor reading, an uncertain interpretation and an established clinical finding. Would we want a helpful prompt at an agreed moment, or commentary on every meal?

For learning, the possibilities are equally compelling. We could prepare for a difficult meeting, rehearse a presentation, practise a language or receive guidance while developing a practical skill. Our AI Partners could help us identify what we already do well, where a familiar practice has become outdated and where expert support would make a difference.

Over time, good support could help us become more proficient and independent. Sometimes we would want a task completed for us; sometimes we would want to learn how to do it ourselves. Could our AI Partners recognise that difference, gradually reduce assistance and leave space for us to concentrate, experiment and discover?

‍

Their contribution to our thinking could be particularly valuable.

Before an important decision, they could help us examine evidence, separate observations from assumptions, consider alternatives and explore consequences. “What would change your mind?” “Whose experience have you not considered?” “What happens if you do nothing?” These questions could improve a conversation at home as much as a decision in a boardroom.

To do this well, our AI Partners would need to help us examine the inputs behind our thinking. News, social media, podcasts, influencers, thought leaders, colleagues, family members and friends all contribute to what we notice, believe and repeat. How much do we understand about the quality, breadth and influence of that stream?

Imagine being able to ask: “What supports this claim?” “Is this an observation, an opinion, a prediction or a sales pitch?” “Has something important been left out?” An AI Partner could help trace original sources, check dates and quotations, compare accounts and identify relevant expertise or commercial interests. It could flag when several apparently independent reports all relied on the same unverified source.

It could also help us reflect on our information habits. Are we hearing a diversity of informed perspectives? Are we repeatedly consuming material that provokes us without helping us understand? Which voices and experiences are missing? A familiar voice or impressive title should not exempt a claim from examination, and disagreement should not automatically be treated as misinformation.

That scrutiny would need to extend to what we say ourselves. Imagine an AI Partner listening, with appropriate permission from those involved, and offering a discreet correction through an earpiece or display: “You said fifteen million; the source says fifteen thousand.” “That quotation was attributed to someone else.” “They proposed the idea; they did not agree to carry it out.”

Where reliable evidence was available quickly enough, we could correct an error before building an argument around it or passing it on. “Let me correct that figure.” “I have misquoted you.” “I may have misunderstood – could you clarify?” Could this make acknowledging mistakes a more ordinary, constructive part of conversation?

Meaning would require particular sensitivity. An AI Partner might suggest that we check whether we had interpreted someone’s words correctly. It should not pretend to know their intentions. We could choose immediate alerts for consequential errors, gentle prompts for uncertainty and a private review afterwards for minor slips. Humour, imagination and the natural flow of conversation would need room to breathe.

And we would have to apply the same standards to the AI itself. High-quality thinking cannot simply mean agreement with a machine. Would it show the evidence, explain uncertainty and welcome correction? Would we still want its challenge when we were winning an argument or speaking with authority?

These questions become especially practical when we spend money. Before a purchase, our AI Partners could help us understand what we were buying and what our money would support: ingredients and materials, sourcing, supply chain impacts, working conditions, environmental practices, ownership and who ultimately benefits. They could examine ethical claims, compare alternatives and highlight unanswered questions.

They could also help us recognise when marketing was using urgency, fear, status or belonging to discourage reflection. Do we need this? What are the total costs? How long might it last? Could it be repaired? How does it fit our priorities and values?

What if our AI Partners helped us become more thoughtful customers, more capable learners and more constructive contributors in every role?

‍

The depth of that support would depend partly on what they knew about us. This brings us to the possibility of much richer Personal Data Records.

Imagine being able to ask every organisation we interact with for the information and insights it holds about us, in a form we could understand, download and use. Employers, recruitment companies, schools, universities, doctors, retailers, banks and digital platforms each hold different pieces of our story. What could we learn by bringing those perspectives together?

Our records might include qualifications, learning histories, performance reviews, interview feedback, health information, preferences and assessments that influenced decisions about us. We could ask what had been observed, what had been inferred, what was missing and what needed correcting. This is a vision of greater transparency and usable access; its design would need to address confidentiality, other people’s privacy and legitimate limits on disclosure.

AI Partners could help us prepare requests, organise responses, explain terminology and compare records. Each item could retain its source, date and purpose, with clear distinctions between facts, opinions and automated inferences. Where accounts differed, we could investigate rather than assume that combining them produced the truth.

Such a record could strengthen our self-awareness, self-knowledge, self-understanding and self-management. It could help us prepare for appointments, explain our experience and identify opportunities to develop. It could also reveal where the external picture of us differed from our own. A recruitment assessment for one role, or a teacher’s observation from years ago, should remain understandable in its context rather than become a permanent verdict on our potential.

Control would be essential. We should be able to choose which AI Partner or person sees which information, for what purpose and for how long. Sharing qualifications should not require exposing private health information or family conversations. We should be able to challenge errors, inspect access, withdraw permissions and move between providers. A fuller record should leave room for change – and for aspects of life we choose not to record at all.

‍

That choice becomes more complicated when our records include other people.

Should we record our interactions through audio and video? A recording could help us remember an agreement or revisit an explanation. It could also capture someone’s face, voice, home, relationships and vulnerabilities in ways they never intended. The usefulness to us would not settle whether it was acceptable to them.

What privacy and rights should people be able to expect when someone else’s AI Partner can see and hear them?

As principles for the future we want, should everyone have a meaningful say over being recorded, identified or analysed? Should they know what is captured, why, who can access it and how long it will remain? Should they be able to ask for recording to stop, challenge an interpretation or request deletion?

Imagine a friend saying, “I’m happy to talk, but I don’t want this recorded or analysed.” Would switching off become an ordinary act of respect? In workplaces, classrooms and services, how would we ensure that people could genuinely refuse without losing opportunities or support? A small recording indicator would not, by itself, answer those questions.

Permission would also need to be specific. Agreeing to a meeting transcript should not be treated as ethical permission to assess someone’s personality, infer their emotions, publish extracts or train a commercial system. Sensing, recording, retaining, analysing and sharing are different activities. Could agreed assistance work with minimal information, processed on the device and promptly discarded?

Public spaces would bring further difficulties. Being seen by a passer-by is different from appearing in a lasting, searchable record. How would devices avoid identifying bystanders or collecting their conversations? What extra care would be needed around children, people in distress and intimate settings? And how should we balance privacy with situations such as documenting abuse or using technology essential to someone’s disability support?

These are questions about the standards we should build, beyond any single legal rule. They reach into the quality of our relationships. Would we speak as openly, admit uncertainty or explore an unfinished idea if every interaction might be stored and evaluated? Our freedom to augment ourselves should carry responsibility towards those who have not chosen to become part of our digital record.

‍

Even with carefully agreed boundaries, we would face another challenge: security.

What if someone hacked into our AI Partner? They might gain access to intimate conversations, routines, concerns and relationships. If the system also had authority to send messages, make purchases, access accounts or control devices, the consequences could extend into our daily lives.

Someone could also try to manipulate what the AI remembered or the advice it gave. A compromised partner might appear reassuring while steering us towards a fraudulent supplier, suppressing a warning or sharing private information. Could a malicious message or document induce it to treat outside content as an instruction? How would we recognise and stop interference with a system we had come to trust?

The more power we give our AI Partners, the more important it becomes to consider what someone else could do with that power.

We would need strong protection of access and information, limited permissions, additional confirmation for consequential actions and a clear record of what had happened. We would need ways to disconnect a compromised partner, revoke access and recover without depending on that same system. Who would alert us, help us recover and take responsibility for failures?

Our Personal Data Records would raise the stakes further. Would everything need to sit in one enormous store? Could we gain a coherent view through carefully controlled connections between separate records? What should remain separate so that one breach could not expose our whole lives?

Security also brings us back to ownership and incentives. Who would control our AI Partners and pay for them? Would their recommendations serve our interests, an employer’s priorities, a provider’s commercial goals or a sponsor’s products? If we asked them to protect us from manipulation, we would need to examine how they influenced us too.

Ultimately, “giving power” would need to become a set of clear choices. Permission to observe would not automatically include permission to remember. Permission to advise would not include permission to send a message, spend money or share a record. We might authorise routine tasks within agreed limits, approve consequential actions individually and define a narrow emergency response in advance.

Those choices should be understandable, reviewable and easy to withdraw. Trust could grow with demonstrated reliability, but it should never require us to surrender the ability to question, pause or leave. Support accepted during a difficult period should not become permanent authority over our lives.

‍

Access and choice would matter too.

Would excellent AI support be available to people with different incomes, disabilities, languages and ways of thinking? Would people remain free to decline it without losing opportunities? Would workplaces come to expect constant monitoring simply because it had become possible?

We would also need to ask whether sensing everything, all the time, justified its energy, material and financial costs. Sometimes a simpler tool, a trusted human Thinking Partner or a quiet moment might meet the need.

The potential remains extraordinary. We could become better prepared, more aware of our impact and more capable of turning intentions into thoughtful action. We could notice dangers earlier, question poor practices, use resources more carefully and learn throughout our lives.

Our AI Partners could help us examine the consequences of inaction as well as action: the conversation we keep avoiding, the concern we never raise, the opportunity we repeatedly postpone.

But responsibility could not simply disappear into the technology. We would need to consider our own choices alongside the responsibilities of those who design, provide and deploy systems that influence them.

‍

Perhaps a useful starting point is to ask ourselves three questions.

Where in my life would I welcome more support than I currently receive?

What would I permit an AI Partner to do when its advice conflicts with what I want in the moment?

What parts of my life, judgement and freedom would I insist on keeping under my own control?

Our answers may change as we learn, as technology develops and as trust is earned or lost.

How much power should we give our AI Partners?

Enough to help us see more clearly, think more deeply and act more thoughtfully would be a meaningful ambition. Deciding what that means would require an ongoing conversation about the lives we want to lead.

And perhaps the most revealing test would be this: as our AI Partners become more capable, are we becoming more capable of leading our own lives?

‍

Thanks for reading.

Best,

Rob

‍

Developed by Rob Frost from his thinking and ideas, with drafting and refinement support from his AI Partner.

‍

‍