How to design research that captures what people actually do with LLMs at work?

The apparent problem was "how to document professional uses of LLMs?". The real problem was how to transform heterogeneous, sensitive, multimodal field material into a durable research infrastructure that researchers could use months later, across cohorts, without losing context or leaking data. Therefore I designed fieldwork tools (paper vademecum, capture setup), a local-only data processing pipeline (ASR, OCR, Obsidian vault), and an agent harness for cross-source analysis.

The Artificial Inquiries field guide used structured paper exercises to externalize how people use LLMs - mapping tasks, prompts, obstacles, judgments. The capture setup paired an overhead camera, ambient microphone, and close-up camera, turning temporary workshops into durable evidence environments where gestures, prompts, and hesitations remain accessible months later.

The Artificial Inquiries vademecum is a 19-exercise paper sequence that externalises how people use LLMs - mapping tasks, prompts, obstacles, judgments, and trajectories. It does not ask only for opinions about AI. It asks participants to reconstruct their actual workflow: what they copy into the model, when they hesitate, how they judge outputs, what they refuse to delegate. The paper format makes capture non-surveilling and creates physical traces that can be scanned, OCR'd, and linked to audio and video evidence.

The team needed AI for transcription, OCR, navigation, clustering, and synthesis, but sensitive data could not go online. This made local models and local-first interfaces necessary, producing the Open Research Toolkit for researchers processing sensitive data without cloud AI. The same logic led toward Spinosa, enabling agent-based reasoning across the archive while preserving traceability and source inspection.

Fieldwork material - recordings, transcripts, scans, participant documents - was processed entirely locally: on-device speech recognition, local OCR, strict naming conventions encoding cohort, session, and participant IDs. The structured archive became an Obsidian vault with cross-linked, queryable artifacts — a working analysis environment rather than storage.

ORT answers the interface problem (non-technical researchers processing sensitive data locally); Spinosa answers the reasoning problem (source-linked AI analysis across structured archives). The vault agent system bridged them: Navigator located files, Reader extracted content, Synthesizer composed answers, Citation traced every claim to source. The principle: answers must be inspectable, source-connected, and framed as draft findings rather than final truth.

The same processing logic that produced ORT led toward Spinosa. Once the archive is structured and navigable, the next problem is how to let researchers reason across it with agents while preserving traceability, methodological control, and source inspection. Spinosa is the answer to that problem - a CLI-based research product for transforming complex archives into operable corpora for AI-assisted evidence finding.

The visual identity was not the core of the project, but it enabled us to communicate the research effectively by blending into the current landscape of AI projects. The branding, website, and publication materials give the project a coherent public presence - making it feel like part of that landscape rather than a traditional academic research process.

Ecologies turned the messy work practices of 50 participants into a structured, source linked research environment. The project produced the Artificial Inquiries field guide, fieldwork capture setups, local processing workflows, a multimodal research archive, and the technical foundation for ORT and Spinosa.

My role was product owner of the research infrastructure. I designed the fieldwork setup and tools, including the paper vademecum and capture system, architected the digital data processing pipeline, and built the vault agent harness. I worked with Donato, anthropologist, and Gabriel, design researcher and sociologist, on fieldwork sessions, tool iteration, and analysis workflows. The theoretical framework and research questions were led by them.

The infrastructure was shaped by several methodological and technical decisions. We chose a paper vademecum instead of fully digital capture because paper made workshops feel less surveilled, allowed participants to draw and write freely, and created physical traces that could later be scanned and connected to digital data. This created a new problem: researchers needed a way to reconnect fragmented paper traces to specific audio, video, and transcript moments.

We used audio and video capture because observation notes alone compressed too much of the actual behaviour. Gestures, hesitations, embodied routines, and embedded actions often carried the most important evidence. The tradeoff was a richer research record at the cost of heavier GDPR constraints and more complex post session processing.

GDPR also made local processing necessary. On device ASR, local OCR, and local model workflows were not just compliance choices, but methodological ones: sensitive participant data should not be sent to cloud APIs, and interpretation should not be outsourced to opaque external systems. The main bottleneck was not data collection, but processing the material responsibly and giving researchers usable retrieval tools afterward.

The system is designed around calibrated trust. OCR can misread handwriting. Vision models can reconstruct document structure while inventing detail in ambiguous scans. Local models reduce privacy exposure but may underperform cloud endpoints. Audio, video, paper, and transcript streams can fragment context and create false coherence. For this reason, every automated layer preserves researcher judgment as the final gate. Agent reports cite file paths, summaries link back to source material, and every transformation remains reversible.

The research infrastructure is complete and operational. It has supported 12 sessions across 3 cohorts. The fieldwork tools, data pipeline, Obsidian vault, and agent harness are finished, and the research team can now run new sessions, process data, and navigate the archive independently. Research outputs are still being produced, but the infrastructure itself is designed and implemented.