Skip to content
Alexandre Trimborn.
AI articlesMy harness, piece by piece

In depth · Lessons from practice

Discover my AI assistant setup.

A “harness” is the working environment around an AI: its files, its memory, its access. Here is how I built mine so that several assistants can take turns on my projects — and so that I can switch without explaining everything again.

A man in a suit and glasses shakes hands with a humanoid robot in an office.
AI-generated illustration.
01 / Why it matters

This memory is worth keeping.

I want to switch AI assistants without starting over.

When I work with an assistant, I spend time explaining my projects, my constraints, my preferences and the reasons behind my decisions. Through exchanges and corrections, it comes to understand how I work.

In practice, this context is everything I no longer need to repeat: a project’s vocabulary, the options already ruled out and why, the level of detail I expect, the checks I insist on. An assistant that knows it starts from the right place. One that does not makes me explain everything from the beginning.

This context represents hours of work. If it stays locked in with one provider, I lose part of my freedom. The price may go up, another AI may become more relevant, the service may stop suiting me — and yet I will hesitate to leave, because everything would have to be explained again.

Some assistants already offer built-in memory. It is convenient, but it lives with their vendor: another assistant cannot reach it, and I am not the one who controls how it is kept.

My goal: for this memory to follow me. I want to be able to move it, reuse it with another assistant and back it up myself.

Without a harness

One memory per assistantThree assistants, each with its own memory kept by its vendor. None is shared.Assistant AIts memoryat vendor AAssistant BIts memoryat vendor BAssistant CIts memoryat vendor C

With a harness

A shared foundationThe three assistants rely on the same harness, which brings together files, memory and access.Assistant AAssistant BAssistant CMy harnessfiles · memory · access
On the left, each assistant keeps its memory with its vendor: switching means explaining everything again. On the right, the context lives in a shared foundation: the assistant changes, the context stays.
Switching tools should remain a choice. It should not mean starting over.

In short: three building blocks.

I call this whole setup my harness (as in a harness you strap on): the working environment that connects my assistants to my files, my memory and my applications. It rests on three building blocks.

  1. Files. My documents and their successive versions are tracked with Git and hosted on GitHub.
  2. Memory. Decisions, their reasons and project progress are kept in Hindsight, a memory service I host myself.
  3. Access. Connections to my applications (email, documents) go through Activepieces, a gateway I also host.
Overview of the harnessMy assistants use three building blocks: files with Git, hosted at GitHub; Hindsight memory and Activepieces access, hosted on my server. Activepieces connects the assistants to my applications.My assistantsChatGPT, Claude…FilesGitMemoryHindsightAccessActivepiecesat GitHubon my serverMy applicationsemail, documents
Self-hosted componentAssistant or third-party service
Overview. The assistants keep nothing to themselves: they read from and write to the three building blocks.

How to read this feature. Chapter 2 shows the handoff in practice. Chapters 3 to 5 detail each building block. Chapter 6 follows the path of a piece of information, and chapter 7 takes stock. A glossary at the end covers the technical terms.

What I host myself, and why.

I host two building blocks on a server I administer: the memory, which holds my context, and the gateway, which holds the access to my applications. Entrusting the only copy of one, or all the access of the other, to one more online service (a “SaaS”) would create a dependency I prefer to limit.

This is not a judgment on the security of those services: an online service can be well secured. My choice is about what I want to be able to decide myself: where information is kept, who is involved in processing it, how to retrieve my data and how to withdraw access. In return, running these two building blocks is my job, with the responsibilities that come with it.

Not everything is on my side, though. My files are on GitHub — a deliberate exception, which I explain in chapter 3. I also use assistants hosted by their vendors, Claude through its API (called directly by a program, without going through a conversation) and my cloud applications. I am not trying to make these dependencies invisible: I am trying to identify them and to be able to leave them.

02 / How I work

ChatGPT starts. Claude takes over.

The most common case: I start a task with one assistant and continue it with another. This handoff already works for me, and I am the one who triggers it.

One project, several assistantsA handover I initiate
1Start

ChatGPT

Works in a Git branch: a separate line of work that leaves the main version untouched. It records its changes there, then writes a handoff note in the shared memory.

2Hand over

The context stays available

In GitThe changed files and their versions
In HindsightThe goal, the choices, the checks, what comes next
3Continue

Claude

Reads the note, opens the branch and compares: does what the note says match the files actually recorded? Only then does it continue.

Less repetition.Less context lost.Verification still matters.

What does a handoff note look like? A fictional example, for illustration: “Project: website. Branch: contact-page. Done: new form written and tested on mobile. Decision: the phone field stays optional. Not checked: the tablet layout. Next: proofread the copy, then ask for my approval before publishing.”

Why two places?

Files and memory do not say the same thing. Files show what was done : the text, the code, the successive versions. Memory explains why : the original request, the options ruled out, what remains to be checked.

With files alone, the next assistant sees the result without knowing the reasons. With memory alone, it reads an account it cannot check. Comparing the two is what makes the handoff reliable.

Why switch along the way?

To get a second look at a piece of work, or because another assistant suits the next step better. Without shared memory, that switch would cost me a long re-explanation; here, it costs one handoff note.

What the handoff does not do.

It is not automatic: I decide when one assistant stops and when the other takes over. The two do not work on the same task at the same time. And the handoff note is no substitute for checking: it may be incomplete, or outdated if something has changed since.

03 — 05 / The three building blocks

Three components, each with its own role.

The three building blocks read one after the other. Each is presented the same way: what it is for, how it works, and its limits.

03 / Files

Files: why I accept GitHub.

Two similar names, two different things. Git is the software that records every version of a file: I can see what changed, when, and go back. GitHub is an online service that hosts those files and lets several contributors — here, my assistants — work in the same place.

Without Git, you often end up with files named “report_v2”, “report_v2_final”, “report_v2_final_fixed”. Git replaces those copies with a history: each record (a “commit”) notes what changed, when and by whom.

Git is said to be “distributed”: a complete copy of the repository (the project folder tracked by Git) can hold the files and their whole history. My work is therefore not captive to GitHub: I can keep its content elsewhere. [1]

This way out is what makes me accept GitHub. I gain a meeting point for my assistants and a shared history, while still being able to move the content to another host. GitHub itself documents this backup through a “mirror” copy of the repository. [2]

A branch makes the handover verifiable.

A branch is a separate line of work: the assistant prepares its changes there without touching the main version. When ChatGPT prepares a task, the branch shows what was actually done; the memory says why, and what remains to be done. Claude can then compare that account with the recorded files.

A working branchThe branch leaves the main version. ChatGPT records two steps on it, then Claude takes over and adds one. The branch rejoins the main version after approval.Working branchChatGPTClaudehandoffMain versionmerged once approved
Each dot is a record. Work moves forward on the branch; the main version only changes at the end, once the result is approved.

Only one assistant publishes at a time. A shared branch does not replace coordination: before resuming, the assistant checks whether other changes have been made since the last handoff.

The limits. GitHub remains a trusted third party, with its own availability and confidentiality risks. A copy of the repository does not contain everything: large files stored separately (Git LFS), some discussions and some platform-specific settings are backed up separately. And a Git repository backs up neither my Hindsight memory nor the Activepieces connections. [2]

04 / Memory

Memory: separate from the assistant that uses it.

Hindsight is memory software for AI assistants, which I host on my server. Its value: the memory no longer belongs to one particular assistant. I keep decisions, their reasons and the useful context there; I can administer it, back it up and let every assistant I authorize consult it.

Why not put everything in the files? Because a file says what is, not how we got there. A conversation, conversely, contains everything — including hesitations and dead ends. Memory keeps the essentials in between: what was decided, why, and what remains open.

It is not an automatic recorder. Hindsight only retains what it is given : it does not memorize all my exchanges, nor reasoning I never put into words. From a text, it builds structured memories — facts, names, dates, the links between them — and keeps track of where they came from. [3]

How a memory is recorded, then found again.

  1. Record. A language model reads the submitted text and extracts the useful elements. [4] I attach the project, the source, the date and the status: approved decision, hypothesis, observed fact or test result. Everything is kept in a database, PostgreSQL. [5]
  2. Search. When an assistant asks a question, Hindsight searches four ways at once: by meaning, by exact words, by the links between elements and by dates. By meaning, a question about “the contact form” can find a memory that talks about “the page for writing to us”. By exact words, it finds a file name or a precise reference. Links connect what relates to the same project; dates answer “last week” or “before going live”. [6]
  3. Rank. The four searches bring back candidates. A second, specialized model (the reranker) then compares the question with each one and puts the most relevant first. [4]

Record

Save a memoryA text submitted by an assistant is analyzed by Claude Sonnet through an external API. The resulting structured memories are kept in a database on my server.A submitted textby an assistantClaude SonnetExternal API · extracts the elementsStructured memoriesfact · date · source · statusDatabaseon my server

Find

Finding a memoryAn assistant’s question triggers four searches on my server: by meaning, by words, by links and by dates. The results are ranked, then the memories and their sources are returned to the assistant.Questionasked by an assistantmeaningwordslinksdatesRankingMemories and their sources
In purple, what runs on my server. Recording calls an external model; searching and ranking stay on my side.

A fictional example. I submit: “After testing, we keep the form short; the phone field stays optional.” The stored memory looks like this: approved decision — the phone field is optional; project — website; date — that of the exchange; source — the assistant that submitted it. Three weeks later, the question “what did we decide for the form?” brings it back up.

Who does the computing: my server or an external service?

Recording is the most delicate step: poorly extracted information can be reused in many pieces of work. So I entrust it to Claude Sonnet, called through its API via the OpenRouter service. This is a different role from Claude’s when I talk to it as an assistant. The choice aims at extraction quality; it does not prove Claude would be superior to every other model for every piece of content.

Search by meaning and ranking, for their part, are computed on my server. Hindsight lets you choose the model for each step separately. [4]

I host the storage. Some processing happens elsewhere. At each recording, the text to analyze is sent to Claude through its API. Self-hosting therefore does not mean all data stays on my server: I take that into account when choosing what I record.

A high score does not prove a memory is true.

Each memory found comes with scores: by meaning, by words, after ranking, and a final score. They indicate how well the memory matches the question, not whether it is accurate, and their scale depends on the search: a score of 0.9 does not mean “90% true”. A highly relevant memory may be old, be only a hypothesis or be contradicted by a stronger source. So I also check its origin, its date and its status. [7]

For the curious: my technical configuration
Extraction
Claude Sonnet via the OpenRouter API, with Google Vertex configured as the route.
Search by meaning
Embeddings computed locally.
Reranking
Local MiniLM cross-encoder, up to 100 candidates, batched by length.
Building memory
Explicit saves, with automatic consolidation disabled.
Traceability
Project, dates, sources and status kept with useful information.

Switch assistants and keep the option to move servers.

Switching assistants requires no copy: the next one queries the same memory and receives the useful context. Moving the memory itself to another server is a different operation.

Hindsight’s current documentation describes backing up, restoring and transferring a “bank” — the set of memories in one space — from one installation to another; search indexes are rebuilt on arrival. Whether these features are available must be checked in the version in use. It is not a universal format : another AI would not import it without adaptation. [8]

05 / Access

Access: do not scatter the keys.

For an assistant to read my email or my documents, I have to open access for it. This usually goes through OAuth, the mechanism behind the “Allow this application to access your account” screens. The tool then receives a token: a digital key that lets it view data or act, according to the rights granted. These tokens must be protected, and the rights limited to what is needed: this is a cybersecurity decision. [9]

The problem: access that piles up.

By testing assistants and connectors, I could multiply connections to the same account. The risk: no longer knowing which tool still holds which access, and leaving authorizations active after I have stopped using them.

My answer: a single gateway.

Activepieces is automation software that I host. It keeps the connections I entrust to it; the assistant itself does not receive the key. It requests an action — “read this document” — and Activepieces carries it out with the matching connection. The documentation for its MCP server (the protocol that connects an assistant to tools) states that connection secrets are not returned and that operations stay within the scope of the authenticated project. The result of the action, however, is indeed passed to the assistant. [10]

An image helps: rather than handing a copy of my keys to every contributor, I entrust them to a single keeper. The contributor asks the keeper to open a door; they do not leave with the key.

Without a gateway

Scattered accessEach assistant holds its own connection to each application: nine separate access grants to track.Assistant AOutlookAssistant BOneDriveAssistant CGitHub

With a gateway

A single gatewayThe assistants request an action from Activepieces, which holds the connections and calls the applications.Assistant AOutlookAssistant BOneDriveAssistant CGitHubActivepiecesholds the connections
On the left, each assistant holds its own access: hard to know who can do what. On the right, the connections are in one place; the assistant asks, the gateway acts.

I gain an overview: which applications are connected, with which rights, and how to withdraw access. This matches the principles of centralized management, rotation and revocation described by OWASP, a reference body in application security. [11]

For this gateway, I consider relying on one more online service too risky given the control I want to keep. I prefer to host it and reduce the number of intermediaries I entrust with this access. That is my trade-off for this use, not proof that all online services are less secure.

The limits.

Grouping everything also concentrates the risk. A compromise of the gateway can affect several applications. It therefore requires protected administration, controlled network exposure, updates and suitable backups.

An instruction is not a technical restriction. Telling an AI “read only” does not reduce the token’s rights. What the tool can do must be distinguished from what I authorize in a given task.

Connections created elsewhere stay in place. Removing them, then checking their revocation with the provider, remain operations in their own right.

Self-hosted components still have external dependencies. My Microsoft connection uses Activepieces’ standard OAuth application and its token-refresh service — a use its documentation explicitly provides for. The accounts remain hosted at Microsoft. I host the gateway, while keeping these dependencies identified. [12]

Finally, hosting makes me responsible for backups. It is up to me to protect the encrypted connections, their encryption key and the database. Copying the installation files alone is not enough to restore the service. [13]

06 / Interactive diagram

Who receives what, and why?

Choose a path and follow the information step by step. Each step shows where processing happens: on my server, or at a third party.

Build reusable memory

The assistant submits useful information. Hindsight has the configured model structure it and then stores the details it extracts.

  1. 1 · Authorised clientAssistantSubmits a decision, its source and context.
  2. 2 · Self-hostedHindsightReceives the material selected for memory.
  3. 3 · External APIClaude SonnetStructures the content through the configured OpenRouter route.
  4. 4 · Self-hostedMemory bankStores facts, provenance and indexes.
What I control

Where the bank is stored and what information I ask it to keep.

What goes outside

The material needed for extraction goes to the model over an API. It does not stay exclusively on my server.

Self-hosted componentAssistant or third-party service

Simplified diagram of the relevant data flows. It leaves out detailed network rules and some authentication services. Assistants query Hindsight memory directly, without having to go through Activepieces.

07 / What to measure

Time saved and responsibilities I still have.

This way of working saves me time, but it takes attention. Hosting the memory and the gateway means administering a server: updates, backups, access monitoring. It is also a discipline: writing a handoff note, keeping files up to date, checking before resuming. Without it, the memory fills up with outdated information.

Full cost

I count everything: subscriptions, API calls, server, maintenance — and, on the other side, the time I no longer spend rebuilding context. Finer search also consumes computing. No quantified saving is established here.

Handover reliability

Less repetition, less lost context. But an outdated or poorly extracted memory can spread an error: sources, dates and the actual state of the files still have to be checked at every resumption.

Recoverable memory

Having a copy is not enough. It must also be intact, the database must restore, and then the service must fully resume. Succeeding at one step does not prove the next ones will work.

Glossary — 13 terms explained
API
A way for one program to call another, without going through a conversation.
Self-hosted
Installed on a server I administer, rather than at the software vendor’s.
Bank
In Hindsight, the set of memories in one space.
Branch
A separate line of work in Git that leaves the main version untouched.
Commit
A record in Git: what changed, when and by whom.
Repository
A project folder tracked by Git, with its whole history.
Embedding
A numerical representation of a text’s meaning; it makes it possible to search by meaning rather than by words.
Harness
The working environment around an AI: files, memory, access.
Token
A digital key given to a tool to access an account, according to the rights granted.
MCP
A protocol that connects an AI assistant to tools.
OAuth
The authorization mechanism behind the “Allow this application” screens.
Reranker
A model that reorders search results from most to least relevant for a question.
SaaS
Software used online, hosted by its vendor.
Sources and background — 14 references

Primary sources checked on 7 November 2026. The configuration also draws on my maintenance records. Current documentation may describe features that differ from those of the version I installed. My reasons and experience are distinguished from the documented mechanisms.

  1. Git — About Version ControlDistributed copies and history.
  2. GitHub — Backing up a repositoryGit mirrors, Git LFS and backup limits.
  3. Hindsight — RetainExtracting and structuring memories.
  4. Hindsight — ModelsSeparate roles of the language model, embeddings and reranker.
  5. Hindsight — StoragePersistence in PostgreSQL.
  6. Hindsight — Recall architectureSemantic, lexical, relational and temporal search.
  7. Hindsight — Recall responseRelative scores, provenance and dates of results.
  8. Hindsight — Admin CLIBackup, restoration and transfers between banks.
  9. IETF — RFC 9700, OAuth SecurityPermissions, protection and token lifecycle.
  10. Activepieces — MCP ServerAuthentication, secrets and tool scope.
  11. OWASP — Secrets ManagementCentralisation, least privilege, rotation and revocation.
  12. Activepieces — Environment VariablesEncrypted connections and hosted OAuth apps.
  13. Activepieces — Self HostOperations, persistent data and backups.
  14. Activepieces — LicenseThe difference between the open-source core and commercially licensed features.

Can your memory move with you?