933 Commits
Author SHA1 Message Date
evgeny d8cbe2e589 Add proxy bug list and lightsout game (desktop + Android) 2026-09-29 12:19:23 +03:00
evgeny e1946b208d Preserve proxy TCP streams across connect, backpressure and FIN 2026-09-29 04:00:34 +03:00
evgeny 09bc8c35c5 Release TUN proxy connections when lwIP aborts their PCB 2026-09-29 03:59:30 +03:00
evgeny f19ff11b36 Scope TCP proxy streams and restart messages to their peer 2026-09-29 03:58:53 +03:00
evgeny 268243feeb Isolate datagram proxy state and replies between instances and peers 2026-09-29 03:58:14 +03:00
evgeny 4be1e62251 Parse IPv4 options and reject malformed TUN proxy packets 2026-09-29 03:57:37 +03:00
evgeny 10b66e2ad0 Validate SOCKS5 authentication and destination addresses 2026-09-29 03:57:21 +03:00
evgeny 7a41fcf512 Notify TCP writers when the final synchronous send drains 2026-09-29 03:57:15 +03:00
evgeny 518df33b0e Fix UDP and ICMP packet construction at the TUN boundary 2026-09-29 03:56:55 +03:00
evgeny fdec5b65c5 Add independent radio compressor and Android audio controls 2026-09-29 03:37:47 +03:00
evgeny ce1f23fe46 Radio: stereo capture/playback with mono fallback; CAMCORDER source; bump RADIO_MAX_OPUS
- stereo capture: Android AudioRecord stereo (CAMCORDER source for stereo,
  voice sources fall back to CAMCORDER) + desktop separate miniaudio
  capture/playback devices (no asymmetric duplex)
- Opus stereo encode/decode, voice_jitter stereo (per-channel frame_size,
  slot-based SoundTouch)
- radio_audio_start takes explicit capture_channels (stereo->mono fallback)
- RADIO_MAX_OPUS 256->512 (128kbps stereo frames up to 320B)
- radio_capture_stereo setting + desktop radio settings UI (opus mode/bitrate,
  capture stereo)
- radio_send_to send stats logging (etcp_send ok/fail per second)
2026-09-29 02:31:41 +03:00
evgeny 331970a121 Radio: Opus mode/bitrate + mic source + PTT release tail; drop online status from member JSON 2026-09-29 00:40:10 +03:00
evgeny e3caf15f32 Guard routing loops and reconcile blocked transit against current paths
Track visited nodes outside the signed body for all routed packet types. Recheck queued transit routes without losing FIFO priority, reject unknown-group transit, and suppress BGP announcements caused only by RTT jitter.

Document routing rules and wire format. Validate graph churn, REINIT, encrypted transit, malformed paths, loop drops and queue cleanup; full suite: 106 passed, 1 skipped, plus proxy integration and sanitizer checks.
2026-09-28 15:36:47 +03:00
evgeny fa9e102271 Preserve config string terminator when preparing key updates 2026-09-28 15:16:57 +03:00
evgeny 39ce249361 Prune unconfirmed BGP paths and restart stalled exchanges 2026-09-28 14:58:13 +03:00
evgeny b76e6553bc Reconcile BGP paths and advertise selected routes per neighbor 2026-09-28 14:37:44 +03:00
evgeny 546e0c185d Fix WITHDRAW removing nodes with surviving paths 2026-09-28 14:20:55 +03:00
evgeny 4503bdfa7e diagnostics: explain and rate-limit rejected ETCP session data 2026-09-28 13:16:39 +03:00
evgeny c63e93715f fix: accept invite public keys with a leading zero byte 2026-09-28 12:26:13 +03:00
evgeny 904a9fbda1 docs: refresh development guides for current architecture 2026-09-28 12:10:39 +03:00
evgeny 01fcefdec5 docs: clarify core, group and chat header contracts 2026-09-28 12:01:08 +03:00
evgeny 72fee9e245 Separate core, UTUN and chat lifecycles with owned asynchronous work 2026-09-28 11:39:08 +03:00
evgeny 5838e63cde Use shared group requests for config and chat connection planners 2026-09-28 11:07:25 +03:00
evgeny 0756a6e020 Serialize group topology lazily through a cancellable backpressure sender 2026-09-28 02:45:00 +03:00
evgeny 792776a1f6 Negotiate group sessions and scope topology messages to peer epochs 2026-09-28 02:15:54 +03:00
evgeny 52ef7a4b3c Own recovery attempts in group sessions and require restored routes 2026-09-27 23:06:48 +03:00
evgeny 66c59501fc Track topology exchange readiness per group and peer 2026-09-27 22:14:03 +03:00
evgeny bce17d9cfc Route UTUN configuration and group ownership through NCD 2026-09-27 21:59:32 +03:00
evgeny 9745a987ac Define join contract and require committed CHAT membership for BGP 2026-09-27 20:34:24 +03:00
evgeny 1a8072e9b2 Release group NCD ownership when destroying peers 2026-09-27 20:08:17 +03:00
evgeny 26e6c05313 Apply fresh BGP addresses to shared NCD connections 2026-09-27 19:18:40 +03:00
evgeny ec51b2d61f Version signed node snapshots with persistent timestamps 2026-09-27 19:11:11 +03:00
evgeny 7374d1648c Support 8 KiB member owner tags without signature or serialization truncation 2026-09-27 17:12:13 +03:00
evgeny de714f5023 Add bounded Merkle pull lifecycle and terminal error handoff 2026-09-27 16:17:04 +03:00
evgeny 3e25bcef02 Refactor Merkle sync into bounded bidirectional rounds with atomic member updates 2026-09-27 14:44:39 +03:00
evgeny 1b92b4885b Restrict dynamic auto socket test to Linux 2026-09-27 12:48:40 +03:00
evgeny 10d3ecfda4 Record final root transport and full suite validation 2026-09-27 12:19:52 +03:00
evgeny 03f71fd94e Bound ETCP receive window and fix timestamp and allocator UB 2026-09-27 12:16:15 +03:00
evgeny 59f4802904 Fix autosocket ownership and exercise interface churn under load 2026-09-27 11:57:29 +03:00
evgeny 3131dcbb7c Fix ETCP session recovery and router lifecycle; confirm chat invites before publishing 2026-09-27 02:51:47 +03:00
evgeny 018c8e795d android: фикс UAF AudioTrack при переключении группы рации (start → releaseLocked) 2026-09-26 19:18:42 +03:00
evgeny 0209bcf5d3 android: рация — единый режим per-канал, не гаснет при выходе/заходе в группу 2026-09-26 19:12:07 +03:00
evgeny 44d4265084 ll_queue: queue_waiter_cancel отменяет отложенный call_soon при internal==NULL (фикс UAF waiter_defer) 2026-09-26 18:58:27 +03:00
evgeny f6e6e2c73f stcp: write-error закрывает conn отложенно (фикс UAF keepalive/etcp_tcp_send)
Синхронный send-путь (stcp_link_send → queue_data_put → tx_queue_cb →
flush_pending → try_send) при ошибке сокета вызывал stcp_conn_do_close →
on_close → tcp_link_close_cb → etcp_link_close внутри стека вызывающего,
освобождая ETCP_LINK под keepalive_timer_cb/etcp_tcp_send (use-after-free,
SIGSEGV в keepalive_timer_cb:210).

Теперь write-error планирует отложенный close (uasync_call_soon) с отменой
handle в stcp_conn_do_close; первый err сохраняется; flush_pending не работает
вне STCP_STATE_DATA. Мёртвый hook on_write_error удалён.
2026-09-26 16:42:51 +03:00
evgeny 8aaac45020 ll_queue: отложенный первый запуск автозабора (callback_defer) + тест
queue_set_callback_defer(q, 1) переносит первый запуск callback из
queue_data_put* в uasync_call_soon (очистка стека производителя).
Повторные запуски (queue_resume_callback) и так всегда deferred.
2026-09-26 16:42:46 +03:00
evgeny cc2d2b5f78 etcp: фикс UAF teardown (двойное закрытие линка) + вытеснение down-линка при коллизии адреса
- tcp_link_close_cb (серверная ветка): убрать преждевременный огонь link_status/on_link_down,
  etcp_link_close сам снимает линк из списка и стреляет коллбэки в безопасном порядке
  (устраняет двойное закрытие линка — UAF из callring 45502)
- etcp_connection_close: при callbacks_running откладывать закрытие через uasync_call_soon
  вместо *(int*)0=0
- callbacks_running: uint8_t-флаг -> int-счётчик вложенности (++/--)
- etcp_fire_conn_status / etcp_fire_link_status_cbk: поднимать callbacks_running вокруг огня
- insert_link_queue: при коллизии адреса с чужим conn — ERROR при одинаковом pubkey,
  отказ при up-линке, вытеснение (etcp_link_close) при down-линке

Попутно: доки/комментарии в etcp_router, ACK-интервал 100мс->10мс (ROUTER_ACK_INTERVAL_TB).
2026-09-26 13:40:33 +03:00
evgeny 265421eb96 tun proxy: фикс TCP-суммы (нечётный pbuf), FIN до дренажа to_lwip, tcp_bind/rexmit_fast/active_pcbs_changed + нагрузочный burst-тест 2026-09-25 23:36:45 +03:00
evgeny a20d73d2be proxy/tun: добавить DEBUG-логи диагностики застревания keep-alive (lwIP IN/OUT/RECVED/RTO/PERSIST + per-conn DIAG) 2026-09-25 22:12:47 +03:00
evgeny 4353daf913 freebsd: уникальный dst-адрес per-interface (tunN -> 192.0.2.(N+1)) 2026-09-25 20:16:39 +03:00
evgeny 3a4daf6e77 etcp: строгая последовательность коллбэков при закрытии (NEW→UP↔DOWN→DELETE)
- close: финальный per-connection DOWN только если был UP, до state=2
- etcp_cbk_fire / etcp_fire_conn_status / etcp_link_ready_callback / etcp_on_up: guard state==2
- документация контракта в etcp_api.h / etcp.h
2026-09-25 19:13:38 +03:00
evgeny ab5d1c5f91 topo_group: диагностика senders_list (new_conn/remove_conn) для локализации UAF
- new_conn: DEBUG_INFO ADD с conn/peer/state/ch
- remove_conn: state в enter-логе, детали в !found_in_list WARN, REMOVED-лог
2026-09-25 18:18:39 +03:00
evgeny 08e8b2f1cc route_connectivity: фикс UAF при удалении узла с влётными пробами; отмена TCP-пингов при teardown
- topo_nodeq_remove_node: единый хелпер удаления nq (cancel проб + free paths/subnets + remove + free), заменяет все free-сайты
- route_connectivity_cancel_all: итерация по всем группам + зануление ctx->nq + отмена standby-defer
- etcp_send_tcp_ping: out_cli — хендл stcp_client для досрочной отмены; conn_probe_ctx.tcp_cli
- conn_mgr: cand_tcp_pings[] — отмена влётных candidate TCP-пингов в conn_mgr_destroy
2026-09-25 16:01:19 +03:00
evgeny 3f7b4351cd stcp: фикс UAF log_dump, дренаж очередей при закрытии, защита pre-closed серверной ветки, логирование тихих сбросов rx 2026-09-25 13:49:43 +03:00
evgeny ceaf466b1c chatgui-android: входящий звонок поверх блокировки (inline-оверлей) + вибрация всегда кроме silent 2026-09-24 12:08:49 +03:00
evgeny fb4343fca6 transport: reality-релей idle-timeout (конфиг, дефолт 5с) + warn при >64 релеях; убрать лимит FD_SETSIZE; проверка socket_id в stcp_server 2026-09-24 12:08:45 +03:00
evgeny cc269e810c Звонки: надёжность отмены (INVITE-кодограмма + watchdog) и вибрация входящего по ringer-режиму 2026-09-24 10:15:19 +03:00
evgeny b90dc752d2 call: две фазы исходящего вызова — индикация + разные гудки (desktop/Android) 2026-09-23 19:16:53 +03:00
evgeny 7198154106 chatgui: клик по медиа с отсутствующим файлом — сброс пометки и перекачка 2026-09-22 21:59:59 +03:00
evgeny d237dfec7c diag: кольцевой трейс вызовов teardown + дамп при SIGSEGV (размотка UAF link close) 2026-09-22 20:52:02 +03:00
evgeny 4f29839de5 chat: инвайтер шлёт CONNECT_RESULT(0) после приёма джойнера
- chat_join_process_request: постить CHAT_EVT_CONNECT_RESULT(result=0) в GUI инвайтера
  (иначе диалог «Пригласить в канал» висел на «ожидание ответа»)
- cs_handle_join_request: снять pending_invite_* и погасить invite_send_timer после join
2026-09-22 18:26:44 +03:00
evgeny b46fd599fa chat: push-invite join key + запрет флагов на плейсхолдере
- CHANNEL_INVITE теперь несёт join_key (регистрируется инвайтером локально),
  джойнер шлёт JOIN_INFO_REQ с валидным ключом — чинит автоjoin chatserver-узлов
- ask-путь сохраняет join_key для chat_sync_accept_invite
- chat_member_tags_commit отвергает установку флагов на source!=0 (плейсхолдер)
- member_sync_apply_record: defensive ERROR при adm_tags на плейсхолдере
2026-09-22 17:58:50 +03:00
evgeny 0e625da109 call: ретрансмит INVITE каждые 2с + fail-fast при отсутствии маршрута (6с) 2026-09-22 16:06:52 +03:00
evgeny 93becd36e6 keepalive: TCP-линки без ретраев/ACK; не пинговать спящих пиров, bg-ping цикл ≤ 1/мин 2026-09-22 03:47:50 +03:00
evgeny a3e890b711 keepalive: раздельный sleep-код (announce+ACK-ретраи в окне, dead-link) + без flagless в спячке 2026-09-22 03:29:42 +03:00
evgeny a3df1720b4 keepalive: симметричный sleep-протокол (без is_server) + отдельный ACK-бит 2026-09-22 02:41:15 +03:00
evgeny 331c7849f0 keepalive: убрать периодический keepalive в standby (обе стороны молчат, только фазовые анонсы + один ack) 2026-09-22 02:19:28 +03:00
evgeny 945c6a82b0 keepalive: гвард нулевого тайминга фазы (busy-loop) + разделение echo/announce на сервере 2026-09-22 01:35:25 +03:00
evgeny cc0dcb071e chat: UI настройки звука канала (Android toggle + JNI, desktop через C API) + trampoline 2026-09-22 01:25:55 +03:00
evgeny 3d9c79e752 db_sync: троттлинг push к спящему пиру в тихом чате 2026-09-22 01:21:51 +03:00
evgeny b39b85db13 chat: троттлинг по фазе спячки (member_sync send-only, db_sync sound-aware) + настройка звука 2026-09-22 01:20:32 +03:00
evgeny 9f89fd0f7a transport: keepalive duty-cycle тайминг (active_remaining+sleep_duration) + серверная фаза троттлинга 2026-09-22 01:20:26 +03:00
evgeny 0a1ba79d6f chat: member_sync per-peer throttling on peer sleep (cancel_peer + recv/start gating) 2026-09-22 00:42:24 +03:00
evgeny e4e9fca243 transport: per-peer sleep callback (peer_sleep_cbk) fired from keepalive protocol 2026-09-22 00:40:32 +03:00
evgeny 5628138d16 etcp: keepalive sleep-протокол на флагах (KA_FLAG_SLEEP + sleep_time, announce-retry, ack/echo) 2026-09-21 23:30:39 +03:00
evgeny 8a9d8d0097 fix: утечки памяти в router-сервисах и при shutdown
- conn_mgr_router_recv_handler / md_etcp_recv не освобождали entry (рантайм-утечка на каждое сообщение)
- reality_relay: registry + shutdown hook (отложенный free не отрабатывал при остановке)
- etcp_router: uasync_drain_immediate после destroy (router_close_finalize)
- мелкие callback-списки: activity_cbks, node_cbks, props_cbks/apply_cbks
2026-09-21 21:55:11 +03:00
evgeny 2f3001458f android: standby keepalive sync + rx logging during sleep 2026-09-21 21:17:35 +03:00
evgeny 845af9200d Stop call ring sound immediately on accept 2026-09-21 19:41:21 +03:00
evgeny 328d36376c chat: пишем local_attrs атомарно при вставке своего медиа (гонка с MSG_RECEIVED) 2026-09-21 19:36:07 +03:00
evgeny 6e2a2f0011 media_download: сброс таймера простоя при CONN→REQ + правки диагностики
- md_dl_conn_cb: обнулять last_progress_tb при переходе CONN→REQ, иначе время
  установки indirect-соединения (~2с) засчитывается как простой блока и
  вызывал преждевременный фейловер с держателя, реально имеющего блок
- STALL-лог: печатать миллисекунды корректно (значение в единицах 0.1мс /10)
- media_index: INSERT OR IGNORE — повторная регистрация блока как servable
  больше не шумит как ERROR (UNIQUE constraint)
2026-09-21 19:16:27 +03:00
evgeny 9793733cd5 chatgui: диагностика воспроизведения голосовых сообщений (Windows)
- sound_manager: ma_engine_init/playRawPcm логируют через DEBUG_* (qWarning/qDebug терялись в GUI-приложении без консоли); детальный результат старта PCM (rc/is_playing/cursor)
- main: ранняя инициализация лога в GUI-потоке до SoundManager::init, чтобы диагностика инициализации попала в файл
- messagedelegate: лог клика по Play/timeline (filePath/dlState/существует ли файл)
- voiceplayback: лог заголовка opus (sr/ch/frameSamples)

Попутные правки:
- u_async: фикс двойного close wakeup_pipe (eventfd/Windows)
- chat_core: NULL-guard в get_db/is_initialized
- android: sqlite открывается в utun_instance_init, cleanup stale-потока без утечки fd
- android SettingsScreen: компактная раскладка имени интерфейса
2026-09-21 18:59:04 +03:00
evgeny 19dc076528 chatgui: инициализация call/dm/media backfill при старте узла (fix звонков) 2026-09-21 17:20:41 +03:00
evgeny 6a0373ee73 call: диагностика ставок TX/RX и детект сталла аудио-девайса с хард-ресинком 2026-09-21 16:54:14 +03:00
evgeny 570b421ba0 conn_mgr: parallel reverse+indirect, retransmit, and call teardown fix
- call_on_conn_status: only end call when its own data conn drops
  (INDIRECT call was killed by unrelated topo_group_connect conn DELETE)
- conn_mgr: run REVERSE and INDIRECT in parallel after DIRECT fails;
  reverse is preferred (indirect defers via indirect_ready until reverse
  resolves). REVERSE timeout 15s -> 3s.
- retransmit DIRECT_REQ / EXCHANGE_REQ every 1s (idempotent, same request_id)
  to survive packet loss (router RST dropping stale-channel requests)
- replace main.phase with reverse_active/indirect_active/indirect_ready flags
- add diagnostics to trace the parallel flow
2026-09-21 15:22:16 +03:00
evgeny ee54bd52b3 conn_mgr: fix reverse-phase timer leak causing use-after-free crash
cm_ncd_callback re-entered cm_start_phase_reverse on every NCD timeout
during REVERSE phase (shared NCD revived by other modules), leaking reverse
timers: only the last one was tracked in entry->main.timer, so cm_entry_cleanup
cancelled only that one and the earlier timers fired on the freed entry.

- ignore repeated NCD timeouts while already in REVERSE (phase==2)
- cancel entry->main.timer before overwriting in reverse/indirect phases
- remove entry from reverse_pending in cm_entry_cleanup
2026-09-21 13:57:32 +03:00
evgeny e885fe30f5 chatgui: remove universal black background from video window (fix black boxes around controls) 2026-09-21 12:50:06 +03:00
evgeny cd0757e272 chatgui: declare metatype for shared_ptr<vector<int16_t>> (Qt5 build fix) 2026-09-21 12:42:21 +03:00
evgeny f5fece4959 chatgui: fix video playback on Windows (metatype registration + swscale YUVJ range) 2026-09-21 12:39:18 +03:00
evgeny 01987a75d0 chatgui: verify FFmpeg DLL deployment + fix dependency resolution order 2026-09-21 11:52:21 +03:00
evgeny 9d4538935d chatgui: restore FFmpeg build script (build_ffmpeg.sh + .gitignore)
Скрипт был случайно удалён в d773f172 вместе с .gitignore/.opencode.
Без него на Windows не собирается ffmpeg/ffmpeg_build/install -> HAVE_FFMPEG
не определяется -> воспроизведение и транскодирование видео отключены.
2026-09-21 11:29:47 +03:00
evgeny 6ee9bad17d chat: defer media auto-download to per-channel backfill after sync
- db_sync insert-cb gets initial_sync flag (SEND_DATA=1, PUSH/local=0)
- on_msg_inserted skips media download during initial sync
- new chat_media_autodownload_backfill_channel (newest-first) triggered
  on SYNC_DONE per channel (2s debounce)
2026-09-21 02:07:24 +03:00
evgeny a5047c29f7 chatgui: always deploy non-Qt DLLs (FFmpeg, mingw-runtime) next to vibechat.exe 2026-09-21 00:58:56 +03:00
evgeny 2f1545de38 chatgui: fix Qt5 build — QMouseEvent/QWheelEvent position() -> localPos()/posF() 2026-09-21 00:51:48 +03:00
evgeny de850db548 chat: remove join_password check from per-channel JOIN_INFO_REQ (server-link only) 2026-09-20 22:23:42 +03:00
evgeny 39a1da4c28 chat: fix use-after-free in invite_candidates trampoline + accept JOIN_INFO_RESP/READY/ERROR for unknown channels 2026-09-20 22:03:31 +03:00
evgeny d1432b7d19 tests: fix Windows build — SKIP fork-based e2e, cross-platform mkdtemp/setenv 2026-09-20 20:21:50 +03:00
evgeny a7a3666d85 radio: AGC release-rate (без кликов), jitter dead-zone, TX/RX-статистика, FGS-гонка и кэш настроек 2026-09-20 20:07:48 +03:00
evgeny ba1c2d85d3 reality_relay: async DNS вместо getaddrinfo + AGC-компрессор по умолчанию (-12 dBFS) 2026-09-20 15:28:46 +03:00
evgeny 738cfe7835 radio: адаптивный джиттер-буфер (pre-roll, симметричный темп, reorder) + AGC-компрессор микрофона
- voice_jitter: vj_create_cfg (target_ms/min_tempo/max_tempo/pre_roll), pre-roll
  с таймаутом, vj_end для дренажа остатка, глубина учитывает внутренний буфер SoundTouch
- radio_audio: reorder по seq (mesh, out-of-order через соседей), force-flush pending
  по таймауту и FIN, компрессор микрофона (AGC, конфиг [chatserver]), статистика буфера
- audio_compressor: clear_output для потокового слива без потери gain_smoothed
- chatgui messagelist: fromUtf8 для emoji-строк
- тесты: pre-roll/симметричный темп, reorder, late-drop
2026-09-20 14:39:25 +03:00
evgeny 27e66f1092 log: move socket diagnostics SYS->SOCKET and conn lifecycle ETCP->CONNECTION (avoid poll() flood) 2026-09-20 01:51:40 +03:00
evgeny 1bde2eca35 diag: log conn add/remove/free pointers to pinpoint BGP senders_list UAF 2026-09-20 01:42:05 +03:00
evgeny 16237b0df8 etcp_send: fix contract (caller owns entry on error) + fix 7 leak sites 2026-09-20 01:35:30 +03:00
evgeny fae0a7c069 uasync: cast function pointers in socket_add log to void* 2026-09-20 01:18:33 +03:00
evgeny b33caa56b8 uasync: pass socket name to all add_socket call sites 2026-09-20 01:13:38 +03:00
evgeny c36cebe357 uasync: add socket name diagnostics and HUP/ERR auto-unregister safety net 2026-09-20 01:11:30 +03:00
evgeny 90520a83ff topo_group_connect: не подключаться к самому себе в авто-цикле
Phase 1/2/3 и on_up/on_down не фильтровали self: узел попадал в список
connected-пиров (get_connected_peers не исключает self) и разворачивал
node_conn_direct_open на собственные адреса (12 линков на 5C31->5C31).
Ручной путь (connect_node_once) фильтр уже имел.

Добавлен skip self в Phase1 init/cycle_restart, Phase2, Phase3 и в
on_up/on_down (self не помечается connected и не считается в active_conn_count).
2026-09-19 23:40:06 +03:00
evgeny 6b92964794 radio/android: PTT FGS microphone permission + детект потери микрофона
- FOREGROUND_SERVICE_MICROPHONE запрос + foregroundServiceType microphone|specialUse (Android 14+)
- RadioOverlayService.start: не стартовать без mic-разрешения, ServiceCompat.startForeground с TYPE_MICROPHONE
- RadioAudioEngine: micDead-флаг и лог при обрыве captureLoop/read
- radio.c: лог первого кадра burst на транзитных узлах
- SettingsScreen: категория radio в списке debug-категорий
2026-09-19 23:13:04 +03:00
evgeny 688785fc6e etcp: фикс зависшего reinit_pending после реинита готового соединения
etcp_conn_queue_set_ready() делает ранний выход по state!=0, поэтому при
реините уже готового соединения (peer restart / duplicate INIT / collision)
reinit_pending оставался в 1 навсегда. Добавленные в 841b3042 guard'ы
etcp_conn_input/etcp_output_try_assembly дропали все входящие пакеты.

Снятие reinit_pending + REINIT перенесено в etcp_conn_ready.
2026-09-19 23:13:04 +03:00
evgeny 5e955168d2 chatgui-android: PTT-оверлей — talking flow, цвет idle/talking, drag-lock
- RadioAudioEngine: StateFlow talking для реактивной подсветки кнопки
- RadioOverlayService: градиент idle/talking, drag-lock, сброс isShown при destroy
- ChatScreen: скрывать PTT-кнопку при активном оверлее, подсветка по talking
2026-09-19 21:17:38 +03:00
evgeny 841b3042cf etcp: фикс double-free буферов паковщика и повторных free после неудачного queue_data_put
- pn_send_to_etcp/pn_unpacker_cb: занулять pn->data/pn->recvpart ДО queue_data_put
  (ре-энтрантный close/reinit в синхронной цепочке освобождал буфер повторно)
- etcp_output_try_assembly: гард state==2/reinit_pending + убрать re-put освобождённого entry
- etcp_conn_input: гард ре-энтрантного reinit/close в цикле секций, локальный etcp вместо pkt->link
- убраны повторные free после queue_data_put (он сам освобождает entry+dgram) в 6 местах
- диагностика: obj=%p в DOUBLE FREE, логи free dgram в pn_deinit/ack_recv
2026-09-19 21:17:33 +03:00
evgeny 4a785772be chatgui-android: PTT по volume-кнопкам (вверх/вниз) + синхронизация оверлея
- volume up/down обе держат PTT (holdCount), настройка radio_volume_ptt
- syncRadioOverlay при включении/выключении рации
2026-09-19 18:16:46 +03:00
evgeny 8a9184675a radio: дедупликация дублей в mesh + фикс OOB в hop_list
- radio_recv: кэш (src,stream,seq) с TTL 2с гасит кадры, пришедшие по второму пути (diamond/mesh)
- фикс записи за границу new_hops[hop_count] при hop_count==RADIO_MAX_HOPS
- счётчик c_dup_dropped + radio_dup_dropped() для диагностики
- тест test_radio_mesh (4 узла, diamond): C получает ровно N кадров, dup>0
2026-09-19 18:16:42 +03:00
evgeny b96ed8bd00 radio: PTT-радио в каналах (src/radio + chatgui + chatgui-android)
Новый модуль src/radio (radio/radio_audio/radio_headless) — push-to-talk
радио поверх topo-групп, доставка аудио в канал, headless-режим и тесты.
Интеграция в chatgui (ptt_hotkey_manager, radio_audio_engine, радионастройки,
вызовы) и chatgui-android (RadioAudioEngine, RadioOverlayService, UI).
2026-09-19 17:09:00 +03:00
evgeny 398078a28f headless/call: закрывать клиента и логировать при ошибке send() (EPIPE/ECONNRESET) 2026-09-19 10:43:04 +03:00
evgeny 6c42390fc2 utun: игнорировать SIGPIPE, чтобы send() в закрытый TCP-сокет не убивал демон 2026-09-19 10:32:57 +03:00
evgeny 1de1409699 conn_mgr: handle-based API (conn_mgr_open/close), фазы DIRECT/REVERSE/INDIRECT, тесты phases+handles 2026-09-19 09:31:08 +03:00
evgeny 734a414186 etcp: единый реестр svc_id + убрать коллизию CHAT/CONN_MGR (0x11)
Все ETCP_ID_*/ETCP_RT_ID_* сведены в etcp_api.h (единственное место
определения) с реестром и предупреждением про коллизии в пределах одного
пространства имён. Убраны локальные define из topo_group.h,
topo_group_invite.h, db_sync.h, chat_sync.h, chat_join.h; хардкод 0x31 в
member_sync.c заменён на ETCP_RT_ID_MEMBER_SYNC.

Коллизия: chatgui-десктоп биндил мёртвый ETCP_RT_ID_CHAT 0x11 поверх
ETCP_RT_ID_CONN_MGR 0x11 — затирался conn_mgr_router_recv_handler, из-за
чего INDIRECT-обмен conn_mgr не работал и голос звонка не устанавливался.
Удалён мёртвый код recvCallback/messageReceived/send в utun_node.{cpp,h}.
2026-09-19 00:18:39 +03:00
evgeny 6d3ad965ee topo: remember REQUEST_TABLE for not-yet-created groups, fulfill on create
При REQUEST_TABLE для ещё не созданной группы (CHAT-канал не загружен) пир
дропал запрос, и проситель навсегда не узнавал таблицу — односторонняя
видимость (звонок android→linux падал «peer offline»). Теперь запрос
запоминается в pending_table_reqs[32] и выполняется при создании группы
(topo_group_fulfill_table_reqs → handle_request_table).

Попутно: test_chat_join_e2e — критерий успеха через CHAT_EVT_CONNECT_RESULT
(JOIN_READY) вместо senders_list; doxygen-комментарии в etcp_api.h.
2026-09-18 23:36:45 +03:00
evgeny 8bfb456161 etcp: fix OOB/DoS in packet parsing and ping path
- FILLER: drop (uint16_t) truncation in fill_len bound (OOB pointer)
- ACK: require len >= ETCP_ACK_BASE_SIZE (8); wrap-aware clamp of till
  to next_tx_id + 16MB cumulative-ACK iteration limit (DoS)
- ping: bound len before sc_encrypt (stack overflow), ulen bound,
  allocate ETCP_DGRAM + PACKET_DATA_SIZE (heap overflow)

Signed-off-by: uTun <dev@utun>
2026-09-18 17:58:40 +03:00
evgeny 1340ab123b reality/stcp: fix OOB read in key_share parse, padding clamp, shbuf leak
- reality_parse_client_hello: bound shares_len by extension data length
  (len-2) to prevent stack-buffer-overflow from malformed ClientHello
- stcp handshake: reject padding_size > STCP_HS_PADDING_MAX (512)
- reality_ch_body_cb: free shbuf on stcp_try_send error (leak)
- tests: fuzz key_share parse; raw-socket handshake control/clamp/garbage

Signed-off-by: uTun <dev@utun>
2026-09-18 17:20:19 +03:00
evgeny e7fc69bb1d topo: комментарии к функциям topo_node/topo_group
conn_mgr: src node_id вместо conn->peer_node_id, препенд ETCP_RT_ID_CONN_MGR к dgram, bind CONN_MGR после etcp_router_init

call: guard CALL_ST_CONNECTING при accept
2026-09-17 17:30:05 +03:00
evgeny 1fce405204 debug: вычистить логи из категории DEBUG_CATEGORY_DEBUG (удалить дампы, перекатегоризовать ERROR-проверки в ETCP) 2026-09-17 00:17:32 +03:00
evgeny e6a4ffd418 reality: fix leak of ClientHello init buffer in reality_server_start_relay 2026-09-17 00:05:42 +03:00
evgeny 1f1df4f437 mem: детект аномально больших аллокаций (LARGE ALLOC log + calloc overflow guard) 2026-09-16 21:12:46 +03:00
evgeny 51a5c5bcbf router: fix UAF on rconn->recv_conn after ETCP_CONN destroy
recv_conn хранил заимствованный указатель на физический conn без очистки
при его освобождении. Добавлен etcp_router_conn_destroyed(), вызываемый из
etcp_connection_close фазы 1; RST теперь отправляется через живой conn,
доставивший пакет, а не через устаревший recv_conn.
2026-09-16 20:27:03 +03:00
evgeny 51d6d22a27 sock_match: классификация сокета по адресу, а не по type из конфига (fix Q8/Android→LAN)
ip_is_public() вынесен в lib/platform_compat (единый классификатор IPv4 с ntohl).
auto_socket_reconcile_config: тип сокета выставляется по фактическому адресу.
sock_collect_views: has_priv/has_inet/is_eim — по классификации адреса и NAT-детекции.
is_local_subnet упрощён до !ip_is_public(). Тест test_sock_match обновлён.
2026-09-16 20:26:55 +03:00
evgeny 21d9a4e5b7 dm: заполнять peer_name при автосоздании беседы (иначе в списке 0x...) 2026-09-16 20:26:55 +03:00
evgeny 5293fdd3a9 call(android): звать call_audio_init перед start; логировать feed_pcm без instance
Без call_audio_init(inst) g_inst оставался NULL и call_set_audio_ops не
регистрировался: TX (feed_pcm) падал на !ua, RX (audio_ops->on_media) не
вызывался — тишина и спам 'callAudioFeed returned false'. Зеркалим desktop:
utun_bridge_call_audio_start теперь делает call_audio_init + call_audio_start.
2026-09-16 20:26:55 +03:00
evgeny 2de554f8b9 call: единый голосовой стек (libutun_voice) + SoundTouch на Android
- voice_jitter (кодированные Opus-кадры + decode-on-pull + SoundTouch time-stretch)
  вместо PCM-буфера audio_jitter; излишек держится до декодирования
- call_audio.c/.h — единая реализация для desktop/Android/headless (C-интерфейс)
- libutun_voice (voice_sources.cmake): SoundTouch + voice_jitter + call_audio + call_tones
- Android: pull-модель (nativeCallAudioPull), убран Kotlin FIFO, ANDROID_STL=c++_static
- headless audio-сокет: PCM-протокол (decode-only, без тактирования), daemon остаётся C-only
- CALL_STATS расширен dropped/underruns; удалены audio_jitter.cpp и libutun_lite/call_audio.c
- тест test_voice_jitter (ёмкость/догон/стационар), test_call_headless переведён на PCM
2026-09-16 20:26:55 +03:00
evgeny de930b21a2 ncd: fix UAF on deferred UP delivery after handle close
ncd_deliver_up_cb (uasync_call_soon) держал сырой указатель handle h,
который мог быть освобождён node_conn_direct_close/force_close раньше
выполнения отложенного callback'а. Сохраняем токен call_soon в handle
и отменяем его при close через uasync_call_soon_cancel.
2026-09-16 20:26:55 +03:00
evgeny e4557b0473 dm: маршрутизация через общую группу (group_id), JSON-эскейпинг; звонки: иконки/UI в desktop и android 2026-09-16 15:05:48 +03:00
evgeny b483335c06 bgp/ncd: владение conn через NCD-handle в senders_list; call audio route persist + incoming call sound 2026-09-16 13:29:28 +03:00
evgeny b11e77e4c0 call: P2P аудиозвонки — сигналинг, релей медиа, аудио-контур, headless/desktop/android
- src/call/: сигналинг (call.c, call_proto.h, ETCP_RT_ID_CALL 0x36) + релей медиа
  + headless аудио-сокет (call_headless.c, call_audio_bind) + Opus/jitter/тоны
- chat: события CHAT_EVT_CALL_* (34–40), CHAT_EVT_DM_MESSAGES (41), DM-трамплины
- chatgui: call window, miniaudio I/O, audio_frame_ring, test_audio_jitter
- chatgui-android: CallScreen/IncomingCallDialog, аудио-роутинг, JNI, рингтон
- tests: test_call, test_call_headless
2026-09-16 10:17:57 +03:00
evgeny 0157f7f96b build: fix --asan to build lib/libopus dependency 2026-09-15 22:44:27 +03:00
evgeny 21ec740d82 socks_proxy: defer conn free to fix UAF from synchronous write_queue flush 2026-09-15 22:39:27 +03:00
evgeny b085715654 tun/nat: fix UAF при teardown и use-after-free в tun_read_packet 2026-09-15 22:07:55 +03:00
evgeny 6dc017337d chat: fix UAF в member list trampoline (req->inst после u_free) 2026-09-15 21:52:46 +03:00
evgeny 73fe99b33f dm: прямой обмен сообщениями (DM) + soundtouch + архитектура звонков
- src/dm/: dm_core/dm_crypto/dm_mailbox — прямой обмен сообщениями (DM)
- lib/soundtouch-master/: vendored SoundTouch (изменение темпа/высоты аудио)
- doc/dm_arch.md, doc/chat_call_arch.md: архитектура DM и звонков
- tests: test_dm.c, test_dm_e2e.c
- chat: рефакторинг chat_* модулей, member_sync, invite_build
- chatgui/chatgui-android: синхронизация с новым API чата
- lib/u_async, debug_config, utun_instance: сопутствующие изменения
2026-09-15 21:09:17 +03:00
evgeny 761f64cf76 chatgui-android: отображение REALITY-линков (TCP+REALITY) в статусе узлов и сокетов 2026-09-15 12:18:22 +03:00
evgeny 27b591daee reality: использовать NTP-время вместо time(NULL) в анти-реплей проверке timestamp
Часы colo2 отстают на ~52с (ntpd выключен), что ломало reality auth
(diff < -30с = time_window). Теперь клиент и сервер передают
ntp_time_get_seconds() в reality_client_hello_build_at /
reality_server_hello_build_at вместо сырого системного времени.
2026-09-15 05:39:28 +03:00
evgeny 481872d5db chat: invite-ссылки с несколькими адресами + ncd-адреса узлов (join) 2026-09-15 05:16:04 +03:00
evgeny d88dccfa19 etcp/stcp: обрыв etcp_link->tcp_link при закрытии stcp_link (fix UAF) 2026-09-15 00:25:42 +03:00
evgeny af382403ec tun_freebsd: reuse existing TUN interface on SIOCIFCREATE EEXIST 2026-09-14 23:17:59 +03:00
evgeny 952504d8eb etcp: убрать ложный reinit по gop (got_initial_pkt)
reinit должен происходить только при смене reset_id пира
(etcp_conn_apply_peer_reset_id). gop-проверка 'dirty vs clean' на догоняющем
TCP-линке давала ложный reinit и рвала соединение (JOIN_INFO_RESP терялся).
Удалён peer_got_initial_pkt из stcp_link (стал не нужен).
2026-09-14 22:46:41 +03:00
evgeny d58d4d39c4 chat join: запуск JOIN_INFO_REQ при уже поднятом соединении + добавление джойнера в группу
- cm_invite_trampoline: если уже подключены к connection-узлу — cs_on_conn_up
  повторно не сработает, запускаем join-хендшейк напрямую (cs_start_channel_join)
- chat_join_process_request: после верификации ключа сразу добавляем джойнера
  в CHAT-группу (topo_group_new_conn), чтобы BGP подключился
2026-09-14 22:46:41 +03:00
evgeny 910cfeac20 chatgui: настройка звука уведомления на канал (sound_on_message) 2026-09-14 21:12:53 +03:00
evgeny fc2262912a chat_join: верификация join-ключа на инвайтере (REQUEST_FWD)
Инвайтер добавлял мембера без проверки join-ключа — любой известный узел
мог самопригласиться в канал через REQUEST_FWD напрямую владельцу.
register_key теперь всегда хранит выданный ключ локально,
process_request отклоняет запрос с неизвестным/чужим ключом.
Тесты: unknown join_key rejected + регистрация ключа в t_process_request.
2026-09-14 21:12:53 +03:00
evgeny 75e39d2fee chat_join: модуль join-протокола, invite-ссылки v2, синхронизация мемберов 2026-09-14 19:57:27 +03:00
evgeny a01a4ddba4 fix: UAF in _on_member_sync_done (channel_cache freed by refresh)
cs_flush_sync passed &cs->channels[i] as member_sync cb_arg, but the 30s
cs_refresh_channels frees/reallocs cs->channels[], leaving the arg
dangling when _session_done later fires the callback. Re-find the channel
by the (stable) ns id instead of trusting arg.
2026-09-14 15:19:07 +03:00
evgeny 9ee61dce6c fix: ncd handle UAF in topo_recovery and topo_group_invite callbacks
topo_recovery_callback: detach cb on UP handoff and close handle on
DOWN/TIMEOUT so no handle with cb_arg==ctx outlives ctx (ASAN
heap-buffer-overflow in topo_recovery_callback).

tgi_to_channel_cb: force_close handle on NCD_EVENT_TIMEOUT so a late UP
cannot fire into the freed tgi_conn_ctx.

Also: android chatgui reverseLayout scroll + msg order/stable pagination.
2026-09-14 15:10:04 +03:00
evgeny fc3295ce12 nat_detection: гасить NAT-чек при закрытии линка через link-status коллбэк (UAF na->link) 2026-09-14 14:37:34 +03:00
evgeny e0f5afbff8 stcp: гасить on_close_cb/close_arg в stcp_link_close; keepalive: снапшот+ревалидация link до/после каскада (UAF) 2026-09-14 14:27:38 +03:00
evgeny da329d0479 merkle_sync: ms_outq_drain_cb — сохранять e->len до etcp_send (UAF); документировать ownership etcp_send 2026-09-14 13:49:12 +03:00
evgeny 1f8ccabdbf etcp: tcp_link_close_cb — сохранять link поля до каскада (UAF local_link_id) 2026-09-14 13:39:07 +03:00
evgeny 187bcd80b9 uasync: ре-валидация node перед последующими коллбэками (счётчик cb_called) вместо снапшота 2026-09-14 12:56:52 +03:00
evgeny 836c715299 uasync: снапшот полей socket_node в poll()/select() — защита от realloc во время коллбэков (UAF) 2026-09-14 12:47:35 +03:00
evgeny 369b253bdb chatgui-android: автоскролл чата вниз при открытии клавиатуры и своих сообщениях 2026-09-14 12:14:44 +03:00
evgeny d78d652bf5 shutdown: закрывать shared sqlite (topo_sqlite_db) в самом конце utun_instance_destroy (UAF при teardown) 2026-09-14 11:47:45 +03:00
evgeny 884f36c2d4 ncd: cancel connect/fin_wait таймеры в ncd_deferred_close (UAF fin_wait_timeout_cb) 2026-09-14 11:34:36 +03:00
evgeny f2f4a37344 routing: fix use-after-free в route_ping_cancel_for_conn (чтение request_id после u_free) + формат %08x 2026-09-14 11:29:31 +03:00
evgeny 11991e688c build: ASAN-сборка включает lib/libopus (зависимость src) 2026-09-14 11:20:12 +03:00
evgeny d730b06436 build: доработка build_bsd_full.sh (ASAN/hardened через clang, gmake fallback, -jN, FFmpeg) 2026-09-14 11:18:05 +03:00
evgeny d0dfa97a47 transport: SOCKS5-прокси для ETCP-линков (CONNECT + UDP ASSOCIATE + пинги) 2026-09-14 00:24:11 +03:00
evgeny 7c8d65db78 chat: локальное удаление канала (Android) + db_sync инстансы в куче 2026-09-13 22:08:33 +03:00
evgeny ddfffad1ed conn: сопоставление сокетов по ролям (direct/nat/local) + hairpin; NTP-детект оффлайна
- sock_match: единая классификация сокетов (DIRECT/NAT/LOCAL) и адресов (PUB/PRIV/UNREACH)
- eim приравнен к direct; на приватный адрес — только с приватного интерфейса;
  оба EIM за одним NAT (одинаковый публичный IP) — только через LAN
- topo_node: LOCAL-сокеты рекламируются (INTERFACE), private скрыт,
  NAT-адрес не рекламируется для strict и при оффлайне
- ntp: флаг reachable по циклам + пере-анонс nodeinfo при смене доступности
- cm_direct_add_links / ncd_create_links / conn_match_candidate_sockets на sock_match
- test_sock_match: unit-тесты классификации и сопоставления
2026-09-13 16:48:00 +03:00
evgeny ff53c735dd chat: превью медиасообщений в списке каналов (Android + desktop) 2026-09-13 15:55:33 +03:00
evgeny 07ef7d9a01 chat: живой счётчик медиакеша + сверка + DESC-проход докачки/вычистки 2026-09-13 14:49:57 +03:00
evgeny 1eb0566cb5 chat: enforce storage_total_size + media orphan cleanup; async_dns: fix use-after-free
- media cache: удаление сирот на старте (сверка ФС с сообщениями) + лимит
  storage_total_size (удаление самых старых файлов, только старше backfill_days)
- ma_dir_walk(): кроссплатформенный рекурсивный обход каталога (POSIX/Windows)
- async_dns: первый драйв slow-пути отложен через call_soon (исключён
  double-free при синхронном завершении запроса); cancel унифицирован
- chat: видимый WARN 'media too large' + честный счётчик started/skipped
2026-09-13 03:11:10 +03:00
evgeny 7bba39ddcb nodeinfo: 2 адреса на сокет (до/после NAT) + типы адресов local/public/nat/nat_eim/nat_strict
- topo_node: сокет рекламирует INTERFACE (до NAT) + NAT (после NAT, EIM);
  strict/direct отдельным адресом не передаются
- topo_node_sqlite: addr_type_classify различает NAT_VERIFIED_*; добавлен
  ADDR_TYPE_NAT (тип неизвестен), fallback больше не даёт DIRECT по IP
- chat_member: бейдж NAT-типа по приоритету (public>nat_eim>nat>nat_strict>local)
  вместо MIN(addr_type)
- GUI (desktop+android): адреса мембера с socket_id и ролью (local/public/nat/nat_eim/nat_strict)
2026-09-13 01:05:25 +03:00
evgeny a823898379 chat/media: докачка недостающего медиа после стартовой синхронизации + HAVE_BLOCK анонс локальных блоков 2026-09-12 21:28:08 +03:00
evgeny e636c3a420 docs: доступ на узел colo2 2026-09-12 21:24:04 +03:00
evgeny 5ee6bf53d8 conn_mgr: ping активных кандидатов с noop-колбэком вместо NULL (bad args) 2026-09-12 21:07:24 +03:00
evgeny c6118280bd conn_mgr: не удалять как stale кандидатов с активным соединением 2026-09-12 20:00:48 +03:00
evgeny 265008961a conn_mgr: кандидаты-посредники из topo_node-списка (активные соединения); гейт md_super_connect по is_supernode; откат ping→best_candidates 2026-09-12 19:38:48 +03:00
evgeny 2045d4a169 media: восстановить conn_mgr_open (DIR/REV/IND) + популяция best_candidates из RTT + watchdog не трогает CONN-блоки 2026-09-12 18:47:01 +03:00
evgeny 905fde5a27 media: BLOCK_REQ транзитом — убрать требование прямого соединения (NAT-узлы качают через роутер) 2026-09-12 18:27:23 +03:00
evgeny 0055c2c6fe media: фикс детекта суперузла (json_flat_get), backfill осиротевших медиа, fallback на автора при пустом QUERY_RESP 2026-09-12 18:19:47 +03:00
evgeny 5a83aaa786 opencode: подключить aiterm-mcp (интерактивный терминал для отладки) 2026-09-12 16:19:09 +03:00
evgeny ce12f058d9 tcp_io: fix error_cb busy-spin on graceful EPOLLHUP; socks_proxy: close local socket on both FINs
- error_cb (tcp_io): при graceful HUP (SO_ERROR==0) после уже прочитанного FIN
  сокет оставался в epoll и level-triggered EPOLLHUP крутил цикл на 100% CPU.
  Теперь сокет закрывается через tcp_conn_push_close.
- socks_proxy on_fin_cb: при обоих обменянных FIN локальный сокет закрывается
  (send_close + push_close), симметрично exit-стороне (tcp_proxy_server).
2026-09-12 16:08:14 +03:00
evgeny 56d0928632 etcp: wire-MTU кап 2048→1500 (согласовано с TUN_MAX_PACKET_SIZE), enc_buf привязан к MTU; тест multilink: per-link проверка трафика с ретраем через keepalive-тики 2026-09-12 13:53:55 +03:00
evgeny 13e2697f80 test_etcp_link_stress: переписан — 5 вариантов добавления линков (V1/V2, коллизии V3a/b/c), kill/failover и инвариант «нет реинита при живом линке»; фикс stack buffer overflow в etcp_encrypt_send (enc_buf 1600→2048) 2026-09-12 13:31:37 +03:00
evgeny 42a94aa916 BGP: фикс владения адресами в реестре (мульти-группа), проброс group_id в сервис (conn_mgr per-group), membership-фильтр create_group; тест media через chat-группу 2026-09-12 02:13:34 +03:00
evgeny 6c9a47b201 media_delivery: диагностика маршрутизации md_send/md_dl_send/router_deliver (conn, subcmd, no-route) + тест test_media_delivery_chat 2026-09-11 23:48:31 +03:00
evgeny d7bb58c6fb chat: автоподключение к узлам группы (вкл/выкл) + ручной connect к узлу (desktop+android); фикс reinit при QR-join 2026-09-11 19:46:02 +03:00
evgeny 9ef501a2fb etcp_connections: собрать forward-объявления в блок и добавить пояснения к функциям 2026-09-11 18:13:48 +03:00
evgeny d95048160d etcp: закрыть UAF в клиентской ветке tcp_link_close_cb
etcp_on_link_down может каскадно закрыть conn и освободить link;
использовать сохранённый etcp и проверять state!=2 перед reconnect
2026-09-11 17:46:38 +03:00
evgeny be1f08b43e etcp: защита от UAF в start_keepalive_timer/enter_ready_tcp + диагностика указателей
- start_keepalive_timer: guard !link->etcp->instance->ua (SIGSEGV на 80 строке)
- etcp_link_enter_ready_tcp: return при etcp->state==2 после reinit/apply_reset/conn_ready
- tcp_server_on_link/enter_ready_tcp: DEBUG_CATEGORY_DEBUG с адресами link/etcp/inst/ua
- topo_node: убрать лишние DEBUG_CATEGORY_DEBUG (chain_rtt)
2026-09-11 17:13:35 +03:00
evgeny 7de636be6e topo: убрать ALTER-миграции, backfill deleted из adm_tags 2026-09-11 16:48:15 +03:00
evgeny b7da6547c1 connection: логировать только connect/reconnect/disconnect; фиксы конфига чатгуи
- категория connection теперь только жизненный цикл соединения; INIT/PING/keepalive
  переклассифицированы в ETCP/SOCKET/KEEPALIVE/REALITY/ROUTING, удалены дубли
- settingsdialog: устранён дубль my_node_name в конфиге (двухпроходная запись + только при изменении)
- storagesettingspage: канонические ключи storage_total_size/storage_unit_size с суффиксом M/G
- audiodevicesettingspage: compressor_max_gain_db min 0→1
- etcp_connections: %s-баг в логе TCP link (addr → sockaddr_storage_to_str)
- docs/sample: [chatserver], актуализированы ключи хранилища
2026-09-11 15:24:56 +03:00
evgeny b7669eea36 Fix use-after-free of ETCP_LINK in keepalive_timer_cb 2026-09-11 13:21:35 +03:00
evgeny 70a18c8be8 etcp: чистка — убрать мёртвый peer_id/guard reset_id, поправить комментарий STCP_HS 2026-09-11 11:59:48 +03:00
evgeny f36ad8903a etcp: убрать session_id из INIT/STCP handshake, симметричная эпоха reset_id 2026-09-11 11:57:29 +03:00
evgeny ab6773a313 etcp: перепривязка/своп remote_link_id при INIT со сменённым адресом (NAT) 2026-09-10 21:20:26 +03:00
evgeny 311cee0ee1 Fix group node scan stall and UAF on server link teardown
- node_conn_direct_close: immediate conn cleanup when links are down
  (graceful fin_wait on a down conn left a lingering entry, so re-open
  in Phase3 stuck in 'pending' with no timer and scan never continued)
- tcp_link_close_cb: skip etcp_link_close if conn was already closed
  synchronously during etcp_on_link_down cascade (fixes use-after-free)
2026-09-10 19:01:21 +03:00
evgeny 5633903be0 stcp: закрывать принятые коннекты в stcp_server_destroy (UAF server_tlink->conn после tcp_socket_remove) 2026-09-10 17:53:14 +03:00
evgeny 6a674b3422 etcp: etcp_inflight_nullify_link читал entry->data вместо entry — last_link не обнулялся (UAF в bbr/retrans) 2026-09-10 17:53:14 +03:00
evgeny fdad786744 etcp: сбрасывать last_rr_link при закрытии линка в обеих ветках (use-after-free в loadbalancer) 2026-09-10 17:20:19 +03:00
evgeny ee8b2a4a45 chat: deleted-флаг выводить из adm_tags — member_block_put вместо деструктивного member_put, удалить member_put, фильтр deleted=0 в списках 2026-09-10 17:20:18 +03:00
evgeny 94136564ee platform_compat: выбирать глобальный IPv6 (2000::/3) вместо ULA при выборе адреса интерфейса 2026-09-10 17:20:17 +03:00
evgeny 327aac1c12 ncd: убрать временные DEBUG_CATEGORY_DEBUG логи (scan_conns, open/open_node dedup) 2026-09-10 16:01:57 +03:00
evgeny e75e3750aa ncd: синхронный close timed_out-conn чтобы не было коллизии линков при cycle_restart 2026-09-10 15:57:07 +03:00
evgeny d886a2ff84 Диагностика коллизии линков + детектор дубликата node_id + reality-конфиг autogen 2026-09-10 14:40:45 +03:00
evgeny c2882c533e reality: конфиг [client] для REALITY-клиентов (server_name/short_id/public_key/version/fingerprint) + TCP-линки через NCD 2026-09-10 12:33:16 +03:00
evgeny 898a0a4a66 reality: реклама reality-параметров сокетов через BGP и применение к исходящим TCP-линкам 2026-09-10 11:24:17 +03:00
evgeny edb3bb36e1 reality: TLS-камуфляж для STCP (ClientHello/ServerHello + релей на реальный сайт) 2026-09-10 01:11:26 +03:00
evgeny 0f308fdbd7 chatgui/chatgui-android: уведомления со звуком, флаги и прочие улучшения UI 2026-09-10 00:23:14 +03:00
evgeny 6c857ceab8 chat: обрабатывать CHANNEL_INVITE/INFO_REQ для ещё не созданного канала + GUI-результат invite-by-link
- chat_sync: не дропать CHANNEL_INVITE/CHANNEL_INFO_REQ при отсутствии CHAT-группы (group_id == channel_id, выводим ch_id напрямую) — приглашение по ссылке на сервер теперь доходит до autojoin
- inviteby: регистрация connect_result_cb, показ результата, автозакрытие при успехе
- joindialog: исправлен маппинг кодов результата (2=timeout, -1=not found, -7=not init)
2026-09-09 22:13:52 +03:00
evgeny 22da6bd832 chatgui: открывать SQLite-БД до auto_socket_init (переиспользование портов) 2026-09-09 21:29:27 +03:00
evgeny 738ac8928b tests: fix mkdtemp cross-platform in test_media_delivery_download 2026-09-09 20:32:48 +03:00
evgeny e3168d1e55 Удалить обмен метриками ETCP (ETCP_SECTION_METRICS)
Полностью убрана фича периодического обмена CSV-метриками с Ed25519-подписью:
- ETCP_SECTION_METRICS и парсинг/верификация секции
- struct etcp_metrics, etcp_metrics_add_*/start/stop и снапшот-таймер
- мёртвый API on_metrics_rcvd/metrics_user_ptr (никто не подписан)
- link->remote_ed25519_pubkey (заполнялся только на клиенте → верификация на сервере всегда падала)
- stats_dir в UTUN_INSTANCE
Общий sc_stream_sign_* (деривация Ed25519 из X25519) и INIT-передача обоих ключей сохранены.
2026-09-09 19:59:29 +03:00
evgeny 5b31a765c2 chatgui-android: auto_sockets=android — сокеты из конфига, лёгкое обновление без полного рестарта
- auto_sockets три-режимный: off/system/android; android берёт сокеты из [server] конфига
- auto_socket_reload_config: reconcile сокетов (remove→create) по имени+addr:port, сохранение портов
- etcp_create_server_socket вынесен из init_sockets; stcp_server_list_find_by_sock
- instance_lite_update_sockets + JNI nativeUpdateSockets + NativeLib.updateSockets
- NetworkChangeMonitor: updateSockets вместо restart; порты сохраняются при смене сети
2026-09-09 17:34:46 +03:00
evgeny c474c37284 chatgui/chatgui-android: просмотр изображений с зумом + инфо о медиа
- chatgui: просмотрщик изображений (ImageViewerWindow) с зумом/поворотом
- chatgui: отправка картинок как image с определением разрешения и типа
- chatgui: у видео-пузыря строка info (res·size·bitrate) и бейдж только длительность
- chatgui: контекстное меню медиа (сохранить в файл/скачать)
- chatgui-android: большие emoji-only сообщения, битрейт/разрешение у видео
- chatgui-android: сохранение неактивных интерфейсов как закомментированные секции
- chatgui-android: отправка картинок по расширению (jpg/png/gif/bmp/webp)
- photo_sender: content-type и расширение по типу файла
2026-09-09 16:18:26 +03:00
evgeny ca16b46404 chatgui-android: съёмка фото — JPEG, выбор разрешения, отправка аттачем, просмотр с зумом 2026-09-08 20:02:40 +03:00
evgeny c94acd892e chatgui: автозакрытие видеоплеера по завершении воспроизведения 2026-09-08 20:02:37 +03:00
evgeny 73e81032bc chatgui-android: видеосообщения — запись, выбор камеры, качество, подсветка
- C-слой: video_sender (content_type video/mp4, media_video=0 без FFmpeg)
- chat_msg_submit: duration/width/height для метаданных видео в теле сообщения
- JNI: nativeVideoSend + NativeLib/Repository/ViewModel
- VideoRecordScreen: CameraX VideoCapture, 4 пресета качества (разрешение+битрейт),
  выбор камеры (селфи/широкая/обычная/макро/теле по фокусу), тумблер подсветки,
  запись/пауза/отправить, лимит 60с
- InputBar: единая кнопка — короткий тап переключает voice/video, удержание — запись
- режим камеры/микрофона сохраняется в ViewModel
2026-09-08 18:38:18 +03:00
evgeny fe1726ebea chatgui-android: видеоплеер — зум, поворот, полноэкранный режим 2026-09-08 16:39:59 +03:00
evgeny aa166a2030 topo_group: RESYNC — пассивная сторона перезапускает BGP-обмен после флэпа
Сервер (пассивная сторона) на UP шлёт общий RESYNC, клиент по нему
заново анонсирует группы (JOIN_GROUP), которые ему положены по конфигу.
Чинит потерю маршрутов при одностороннем флэпе, когда клиент не замечает
разрыв и не перешлёт JOIN_GROUP повторно.
2026-09-08 16:39:55 +03:00
evgeny f28d3e91f7 media: рефакторинг загрузки блоков — конечный автомат, фейловер, watchdog простоя
- media_download: состояние блока (IDLE/CONN/REQ/RECV/WAIT/DONE/FAILED),
  кандидаты-держатели (MD_MAX_HOLDERS_PER_BLOCK), фейловер + ретраи (MEDIA_DL_MAX_ATTEMPTS),
  watchdog простоя (MEDIA_DL_STALL_TIMEOUT_TB) вместо фейкового времени
- media_delivery: настраиваемые dl_stall_timeout_tb / dl_max_attempts
- chat_msg: пометка ошибки скачивания в local_attrs ("st":"er") для UI
- tests: новый test_media_download_timeout (watchdog/фейловер), рефакторинг
  test_media_delivery_download/full
- chatgui-android: состояние 'error' у вложений/видео (иконка + tap to retry)
2026-09-08 06:53:48 +03:00
evgeny 089bbe5189 chatgui: запуск на XWayland (xcb) + ручной move/resize с пропорцией
- main.cpp: на Linux принудительно QT_QPA_PLATFORM=xcb (если не задан) — на Wayland
  композитор владеет позицией окна, на XWayland ручной move/resize работает полностью
- videoplayer_window: вернуть ручной move + ручной resize с пропорцией (один путь,
  без платформозависимого нативного startSystemMove/Resize)
2026-09-08 06:48:23 +03:00
evgeny 0dfb39cfcb chatgui: гибридный resize — нативный на Wayland (край тянется), ручной с пропорцией на X11/Windows 2026-09-08 02:47:59 +03:00
evgeny 3adf9871fb chatgui: вернуть ручной resize с пропорцией (двигает окно, край под мышью) + фикс перемотки
- resize: ручной setGeometry с живой пропорцией (якорь = противоположный край);
  drag — нативный startSystemMove
- перемотка: QProxyStyle SH_Slider_AbsoluteSetButtons=LeftButton вместо сломанного
  SeekSlider (sliderReleased не эмитился — seekMs не вызывался)
2026-09-08 02:39:06 +03:00
evgeny 645591bb64 chatgui: нативный move/resize (Wayland) + фикс замирания видео при перемотке
- окно: startSystemMove/startSystemResize вместо ручного move/setGeometry
  (Wayland игнорирует позицию окна) — драг и resize за левый/верхний край работают
- пропорции: снэп к videoAspect после нативного resize (дебаунс 150мс)
- перемотка: backpressure по m_buffered (декод не убегает вперёд и воркер живёт
  для seek), seek через av_seek_frame(-1, us), пропуск кадров с AV_NOPTS_VALUE
- SeekSlider: клик по шкале = прыжок к позиции
- крестик закрытия справа вверху; курсор контролов — стрелка
- test_video_engine: проверка seek
2026-09-08 02:04:18 +03:00
evgeny f2b34459c3 chatgui: баббл видео — размер + время m:ss (напр. 1.5MB · 0:45) 2026-09-08 01:47:32 +03:00
evgeny d09a6af52a chatgui: плеер — фикс drag, центр/fullscreen при открытии, resize за края с сохранением пропорций
- m_render WA_TransparentForMouseEvents — события мыши напрямую в окно (drag работает)
- открытие: видео влезает — натуральный размер по центру; больше экрана — fullscreen
- resize мышью за 4 края и 4 угла, пропорции сохраняются (якорь = противоположный край)
- нижний край доступен через eventFilter на панели контролов (пустые поля пропагируют)
- сброс fullscreen при повторном playFile
2026-09-08 01:31:12 +03:00
evgeny 76182d018f chatgui: QPointer-защита при генерации превью в фоне 2026-09-08 01:01:28 +03:00
evgeny 74b17685f7 chatgui: фикс синхронизации кадров (часы не убегают вперёд при быстром декоде) + smoke-тест движка
- presentTick: кадры всегда по мастер-часам, завершение только когда очередь пуста
- clockMs: по окончании аудио часы в конец (хвостовые кадры не теряются)
- fallback на системные часы, если аудио-устройство недоступно
- tests/test_video_engine: smoke-тест FFmpeg-декодера (opened/frames/seek)
2026-09-08 01:00:38 +03:00
evgeny 6542618927 chatgui: воспроизведение видео — FFmpeg-плеер + frameless окно
- videoplayer_engine: декод в worker-потоке (FFmpeg), A/V sync по аудио-курсору,
  seek с epoch-защитой от устаревших кадров
- videoplayer_window: окно без рамки, drag-перенос, dblclick/Alt+Enter fullscreen,
  Esc закрытие, timeline + play/pause/close, авто-скрытие контролов
- видео-баббл в списке: превью-кадр (jpg), play-оверлей, длительность, прогресс
- превью извлекается после скачивания (video_extract_thumbnail) в фоне
- mainwindow: синглтон-плеер, openVideoRequested → openVideo
- CMake: HAVE_FFMPEG для vibechat
2026-09-08 00:55:06 +03:00
evgeny e264b8aaa7 video: video_extract_thumbnail — превью первого кадра в JPEG (MJPEG) 2026-09-08 00:44:01 +03:00
evgeny 7f3b4f61f4 chat: dur/w/h в тело видео-сообщения (после транскода) 2026-09-08 00:41:43 +03:00
evgeny 1855c61275 video: отправка видео-сообщений (FFmpeg пробинг+транскод H.264/AAC MP4)
- src/video/: video_probe/video_get_info/video_transcode_start (FFmpeg, HAVE_FFMPEG)
- chat_core/chat_msg: media_video флаг, chat_core_submit_media_message, транскод перед index
- headless: команда attach с --video
- chatgui: диалог 'Как видео' + прогресс транскода + sendVideoAttachment
- FFmpeg в сборке (configure.ac, Makefile.am, chatgui CMake, build_nix.sh)
- debug: категория DEBUG_CATEGORY_VIDEO
2026-09-08 00:41:04 +03:00
evgeny c8bd5975ae chatgui-android: invite-ссылки с выбором узла входа в GUI + desktop dialog layout 2026-09-07 22:36:27 +03:00
evgeny ba557e2b9d chat: фикс 0:00 у длинных голосовых — b64_decode терял последний байт без padding + formatTime в мс 2026-09-07 22:36:23 +03:00
evgeny 1345f44e10 chat: invite-ссылки с выбором достижимого узла + tcp_io фикс half-close FIN
- Общий модуль invite_build: сборка utun://-ссылок, авто-выбор лучшего
  достижимого узла, список кандидатов (вынесено из android jni_bridge)
- chatgui: диалог invite с выбором узла (комбобокс), события INVITE_CANDIDATES /
  INVITE_LINK_READY через gui_bridge
- headless/chatcli: invite <ch_id> [node_id], команда invite_nodes
- tcp_io: error_cb отличает грациозный EPOLLHUP (SO_ERROR==0) от реальной ошибки —
  при приостановленном чтении FIN не роняет недослейные данные (фикс flaky half-close)
- run_test.sh: half-close echo на выделенном порту 19190 + отдельный DNAT (был недостижим)
2026-09-07 13:59:15 +03:00
evgeny a17a85a383 chat: deleted-аккаунты — колонка deleted, скрытие/зачёркивание, не подключаемся 2026-09-07 01:40:53 +03:00
evgeny 631a176944 chat: права админа + member tags (verified/deleted) + ncd revive
- chat_admin: передача канальных privkey другому узлу (encrypted+signed,
  ETCP_RT_ID_ADMIN_KEY), событие ADMIN_KEY_RECEIVED
- member_sync/bridge: tags verified/deleted, is_channel_owner, transfer_admin
  (android JNI + chatgui memberpropsdialog)
- node_conn_direct: оживление timed_out entry — обновление линков в существующем
  conn (убрать старые, добавить новые) + немедленная очистка не поднявшегося
  conn без fin_wait — фикс invite/reconnect
2026-09-07 00:53:41 +03:00
evgeny 6163f34ce1 etcp_router: эпоха reset_id на wire + master/slave — фикс потери пакетов при рестарте пира 2026-09-06 22:20:08 +03:00
evgeny 61c572cae4 etcp: reset_id эпоха ресета — фикс потери NODEINFO при флапе
Добавлен 64-битный reset_id (эпоха ресета) в INIT-фреймы и STCP-handshake:
- генерится один раз при создании conn, меняется только при фатальном ресете
  (normalizer desync / bad fragment size);
- при приёме чужого id slave принимает эпоху пира и ресетится, master держит свою;
- первый id принимается без ресета (свежие при init);
- «tcp peer clean» реинит гейтится по смене id — нет повторного сброса seq.

Это устраняет коллизию seq=1 (RX dup) из-за двойного реинита нескольких TCP-линков,
из-за которой NODEINFO пира терялся и member не возвращался online.

Также: chat member online теперь из topo_group (без БД/merkle), тест test_etcp_seq_collision.
2026-09-06 21:50:06 +03:00
evgeny f7c08f8f5e nat_detection: применять NAT-тип через topo_node_update_my_addresses с переподписью 2026-09-06 01:29:07 +03:00
evgeny 53bce51201 etcp_router: корректный handshake/restart без таймаут-петли
- START на первом пакете сеанса (seq=0), гейт отправки до первого ACK
- дедуп рестарта по payload init-пакета (ретрансмит START не триггерит рестарт)
- RST переинициализирует только отправку (без CLOSE_ALL), дедуп повторных RST
2026-09-06 00:33:45 +03:00
evgeny 232ad2de6b etcp_router: разделить служебные функции и API в хедере + чистка мёртвого кода
- etcp_router.h: 4 секции (протокол / внутренние структуры / API / служебные)
- переименование router_set_max_inflight -> etcp_router_set_max_inflight
- удалены неиспользуемые: input_q_count, waiter_register/cancel, conn_close_async
  (+ поля close_callback/close_callback_arg)
- docs: актуализировать backpressure (on_send_ready на send_q) вместо удалённого waiter API
2026-09-05 22:34:00 +03:00
evgeny 4b9b0009d8 chat: гейт авто-синхронизации db_sync до member_sync/pubkeys + member tags (verified/deleted) в chatgui 2026-09-05 20:38:22 +03:00
evgeny 01ea6913ae etcp_router: sess_id как локальный счётчик переустановок (не в протоколе) 2026-09-05 20:38:17 +03:00
evgeny 2d04bf941b etcp_router: вынести подпись/шифрование в route_crypto, одна функция отправки с флагами
- route_crypto_encode/decode: автономный пред/пост-обработчик пакета
  (encrypt-then-sign, стандартный Ed25519 вместо sc_stream_sign)
- одна функция отправки etcp_route_send/etcp_router_conn_send с bitmask mode
- формат доставки: [svc_id][src][dst][rx_flags][payload] (статус подписи/шифрования)
- ROUTER_INFLIGHT хранит финальный wire-пакет (чинит потерю подписи при ретрансмите)
- дедуп инициализации: router_conn_reset/free_queues
- учёт inflight через inflight_q count (seq присваивается на enqueue)
2026-09-05 17:40:43 +03:00
evgeny 7c95ddb09b chatgui: значок владельца канала (🔑) в шапке блока мемберов при наличии private key 2026-09-05 15:23:45 +03:00
evgeny 7817bb96aa etcp_router: START/RST handshake для переживания рестарта пира
Раньше START ставился один раз при первой отправке и терялся в ретрансмите,
а принимающая сторона молча авто-синкала rx_seq на любой первый пакет —
после рестарта пира/среднего узла seq рассинхронизировались (тихий data loss
или no-ACK close). Теперь:

- Клиент: флаг start_sent «новый» до первого ACK; пока он стоит, первый
  seq (0) шлётся со START, включая ретрансмит. Первый принятый ACK
  (tx_acked>0) сбрасывает флаг. При рестарте conn флаг снова «новый».
- Сервер: после рестарта peer_sync_done=0 («ждёт START»). Не-START пакет →
  отправка RST (через recv_conn). START → синк rx_seq + ACK → обычный режим.
- RST на принимающей стороне теперь сбрасывает локальное состояние
  (etcp_router_conn_restart), а не закрывает conn.

Тесты unit обновлены: первый data-пакет сессии инжектится со START.
2026-09-05 15:03:44 +03:00
evgeny 84015176b3 etcp_router: досылать потерянный ACK при дубле (рестарт среднего узла)
При рестарте транзитного узла ACK приёмника терялся и не пересылался:
router_schedule_ack выходит рано при rx_seq==last_sent_ack_seq, поэтому
пир ретранслил 17 циклов (~5с), роутер-conn закрывался и цикл повторялся
бесконечно — лечилось только рестартом клиента. Теперь при дубле
(ретрансмите пира) ACK досылается с троттлингом ROUTER_ACK_INTERVAL_TB.
2026-09-05 01:17:37 +03:00
evgeny 32725657dc chatgui-android: обработка скачивания голосовых (download state) + config: секция [chat] алиас для chatserver 2026-09-04 22:46:07 +03:00
evgeny 23fbdf4584 chat: запуск BGP-обмена при подключении узла к группе (invite + auto-connect) 2026-09-04 21:04:18 +03:00
evgeny 5b8c725c01 etcp_router: round-robin send + контроль заполненности send_q через waiter на send_input_q 2026-09-04 19:30:08 +03:00
evgeny 1a88ea88a8 topo_group: узлы меняются только по BGP (NODEINFO/WITHDRAW), удаление по link down через BGP-коллбэк 2026-09-04 17:57:37 +03:00
evgeny b0f149092f BGP: log node add/remove per group at info level 2026-09-04 16:45:33 +03:00
evgeny c049788c08 etcp_router: при no route выводить список узлов группы (<10) 2026-09-04 16:39:23 +03:00
evgeny ed2df9fe88 etcp_router: добавить кол-во узлов группы в лог no route 2026-09-04 16:25:21 +03:00
evgeny d092f649ac chatgui: пути db и лога относительные в конфиге, резолв к бинарнику в рантайме 2026-09-04 14:34:39 +03:00
evgeny 654202b0cb chatgui: debug-категории в секции [debug] построчно (как в utun) 2026-09-04 13:42:59 +03:00
evgeny 20eef8ad16 chatgui: лог по умолчанию в файл chatgui.log (debug_file) 2026-09-04 13:25:06 +03:00
evgeny 656da48e35 topo_group: асимметричный старт BGP (JOIN_GROUP) + вынос group_id/flags из TOPO_NODE
- UTUN: BGP инициирует только сторона-клиент ([client]) и явные etcp_connect;
  сервер добавляет пира по JOIN_GROUP (фикс обратного маршрута: сервер не
  создавал узел клиента из-за дедупа senders_list)
- topo_group_new_conn: conn_add_path до дедупа; handle_request_table без add_to_senders
- TOPO_NODE: убраны group_id/flags (shared-реестр), передаются в topo_node_serialize
- routing/control_server: явный TOPO_GROUP_UTUN
- CHAT: гейт по peers_* (get_member_channels) + пассивное add/remove в member_sync
- tests: тесты адаптированы (route_ping ждёт registry+v4_addrs)
- chatgui-android: voice_recorder_init в utun_bridge_voice_start (idempotent)
2026-09-04 13:21:24 +03:00
evgeny 7b73d99b5b chat: передавать group_id в бинарном виде в протоколе синхронизации
db_sync/merkle_sync/chat_sync шлют group_id (uint64 BE) вместо текстового
channel_id/namespace. Приёмная сторона находит группу/таблицу через
topo_groups_find(group_id) и требует существующую CHAT-группу. db_sync:
instance key заменён с SHA256(ch_id) на strtoull(ch_id)=group_id.
2026-09-04 09:02:46 +03:00
evgeny 326dfd7e2c media_delivery: маршрутизация всегда через CHAT-группу
media_download/медиа-ответы и суперузловая репликация теперь берут группу
из контекста (dl->group_id, group_id из пакета, peer->group_id) вместо
хардкода TOPO_GROUP_UTUN. node_props_changed получает channel_id.
SUPER_HELLO несёт group_id.
2026-09-03 23:24:50 +03:00
evgeny 55da8fd146 media_delivery: логирование отправки QUERY_RESP и провалов md_send 2026-09-03 21:59:57 +03:00
evgeny 01b61a457a etcp_router: fix ACK no-route — слать ACK через recv_conn
При асимметричной топологии topo_group_find_conn_for_node возвращает NULL,
ACK не уходит → ретрансмит-шторм → no-ACK close → inflight сбрасывается в 0.
Теперь ACK уходит через тот же коннект, с которого пришли данные (recv_conn),
с фолбэком на топологию. Довершает фикс из e08956e (deliver), который
аналогично не трогал router_send_ack.

Заодно удалён неиспользуемый ETCP_ID_ROUTE_ENTRY.
2026-09-03 20:05:59 +03:00
evgeny fbea9c3869 socks_proxy: fix client stall + non-blocking DNS via async_dns
- fix HTTP_STATE_RELAY == SOCKS_STATE_CONNECTING enum collision (relay data dropped)
- fix wrong waiter cancel (etcp_router_waiter_cancel -> etcp_router_cancel_send_ready)
- add backpressure retry timer, restore lost queue_resume_callback, defer FIN on pending tx_buf
- async_dns: /etc/hosts fast-path + hosts loading (avoid 4s DNS timeout for localhost)
2026-09-03 18:13:00 +03:00
evgeny f47ad7d87f tcp_proxy: fix throughput (tx_queue backpressure) + connect timeout + integration test
- Router: unified service delivery format [svc_id][src][dst][payload] (end-to-end src/dst from SVC_ROUTE header)
- tcp_proxy_client: replace single tx_buf drop-on-backpressure with bounded tx_queue + send_q waiter + retry timer (no data loss, bulk window reopen — removes 62.5ms delayed-ACK stall)
- tcp_io: configurable connect timeout (tcp_conn_set_connect_timeout)
- tcp_proxy_server: connect_timeout config option (ms, default 2000, 0=off)
- check.sh: run tcp_proxy_full integration test with sudo (skip if passwordless sudo unavailable)
- test_tcp_io: connect timeout unit test; docs (lwip_nuances.md)
2026-09-03 13:49:09 +03:00
evgeny acfaae2801 dns: заменить K&R-объявления указателей на функции на типизированные
gcc 15 (C23) трактует () как 'без аргументов', из-за чего lib/dns.c не собирался
(65 ошибок). struct dns_rr_i.sort и таблица dns_rrtypes[] переведены на явные
прототипы с typedef'ами и кастами указателей.
2026-09-02 23:59:46 +03:00
evgeny e08956e5dc etcp_router: fix NULL conn в deliver при отсутствии пути к узлу
router_handle_data_packet выбрасывал физический conn, а router_incoming_q_cb
восстанавливал его через topo_group_find_conn_for_node, который возвращает NULL,
когда у узла-источника нет пути в топологии. В итоге callback сервиса (routing,
conn_mgr) получал conn=0x0. Теперь физический conn сохраняется в rconn->recv_conn
и прокидывается в deliver.
2026-09-02 23:59:41 +03:00
evgeny 1db4fe1f01 build_linux_full.sh: автоустановка зависимостей и корректная обработка ошибок 2026-09-02 21:29:48 +03:00
evgeny 5d39083a6d auto_socket: пропускать интерфейсы без default-маршрута (опция auto_socket_skip_no_default_route, default=on) 2026-08-17 23:43:15 +03:00
evgeny f841a964aa chat: debug-логи на джойнер-стороне (channel_put, JOIN sent, WELCOME self-add) 2026-08-17 22:53:03 +03:00
jeka 10306f5613 chat: fix CHANNEL_INVITE invalid ch_sig — передавать владельца канала из БД, а не node_id отправителя 2026-08-17 21:16:44 +03:00
evgeny 7dee49ce63 android: мониторинг смены сети — NetworkChangeMonitor, перезапуск ядра при смене интерфейса 2026-08-17 21:03:05 +03:00
evgeny f3b33bcf74 android: кнопки start/stop сервиса с сохранением состояния, ядро не воскресает при остановленном сервисе
- ConfigProvider: флаг service_enabled (DataStore), flow/getter/setter
- MainActivity: автозапуск сервиса только при service_enabled; edge-to-edge со светлыми иконками статус-бара
- SettingsScreen: секция Service (start/stop), stop/exit сохраняет состояние; NativeLib.restart гейтится по isReady чтобы ядро не запускалось без сервиса
2026-08-17 14:37:54 +03:00
evgeny 3d59e5f322 chat: headless invite_to command + invite-flow integration test
- chat_headless_control.c: новая команда invite_to (inviter-side invite по адресам)
- chat_client.py / chatcli: клиентский метод и CLI-команда invite_to
- tools/chat_invite_test.py: headless-тест инвайтера-side (CHANNEL_INVITE → autojoin → member_sync)
- topo: guard node_id=0 в cm_open/process_nodeinfo + детализация send_nodeinfo warn
- cleanup: удалить устаревшие doc/*.txt, *.md заметки
2026-08-17 11:22:40 +03:00
evgeny 5316ee958a android: ограничить карточку members половиной экрана, CONFIGURE_DEPENDS для GLOB 2026-08-16 23:50:14 +03:00
evgeny 383a45ed47 chatgui: setup_firewall.bat — разрешить входящие в Windows Firewall для vibechat.exe
Отдельный self-elevating скрипт: добавляет program-правило (dir=in action=allow
profile=any) для vibechat.exe, копируется в build-каталог рядом с exe.
2026-08-16 23:23:52 +03:00
evgeny 4ca8348a75 chatgui: хранить last_channel_id в БД (ui_state) вместо QSettings
Восстановление канала только если он существует; запись из utun-потока
через trampoline, чтобы не трогать sqlite из GUI-потока.
2026-08-16 22:34:14 +03:00
evgeny eb24736e27 topo: держать актуальные self-адреса в node_addresses, обновлять/рассылать только при изменении
topo_node_update_my_addresses теперь собирает адреса во временные списки,
сравнивает со старыми и при отсутствии изменений выходит без бампа ver,
записи в БД и broadcast. Гейт member_exists убран: self-адреса пишутся
в node_addresses при любом запуске/изменении (с гарантией строки nodes для FK).
addrs_put логирует ошибки INSERT.
2026-08-16 22:34:10 +03:00
evgeny 54352c17ba chatgui: opus из исходников в CMake + build_nix.sh под Linux/BSD 2026-08-16 14:35:59 +03:00
Evgeny b4722f54f0 dns: убрать weak-атрибут у dns_random — на MinGW слабые data-символы не инициализируются → NULL-вызов → SIGSEGV 2026-08-16 12:58:40 +03:00
Evgeny 0c3398eb8c dns: исправить сборку под Windows (MinGW)
- dns.c: #define uint32_t → #include <stdint.h> (на UCRT64 stdint.h уже тянется через winsock2.h, макро даёт 'duplicate unsigned')
- async_dns.c: p->IpAddress → p->IpAddress.String (IP_ADDR_STRING.IpAddress — IP_ADDRESS_STRING, не char*)
2026-08-16 12:18:07 +03:00
Evgeny 1f307046f9 dns: async_dns модуль (libdns) + неблокирующий NTP sync
- Вендор libdns (MIT) как lib/dns.{c,h}; правка dns_quietinit для GCC 5+
- Новый lib/async_dns.{c,h}: неблокирующий A-резолвер поверх uasync
- src/ntp_time.c: убраны блокирующие getaddrinfo+select; async DNS + неблокирующая машина состояний UDP-запроса
- test_async_dns.c (юнит) + test_ntp.c сценарий hostname→DNS
- сборка: -DDNS_RANDOM=RAND_bytes, -lcrypto в COMMON_LIBS
2026-08-16 12:10:33 +03:00
evgeny d04e9d58b6 etcp: TCP-ping через STCP-хендшейк с флагом PING
- STCP-хендшейк: добавлен байт флагов (STCP_HANDSHAKE_FLAG_PING), HS_ENC 46→47
- stcp_client: stcp_ping_send() — connect+handshake(PING)+rtt+close; ошибка/таймаут → success=0
- stcp_server: детект флага, ответ с эхо-флагом, мягкое закрытие (close_after_send)
- etcp_connections: etcp_send_tcp_ping(), guard is_tcp/fd и проверка возврата send_ping_raw
- route_connectivity: probe TCP-адресов через TCP-ping (1 попытка), UDP-кандидаты без TCP-сокетов
- tests: test_stcp_ping (success+failure path), обновлена сигнатура stcp_client_connect
2026-08-16 10:10:58 +03:00
Evgeny c7b9702c88 chatgui: add CONFIGURE_DEPENDS to libutun globs — pickup new .c files without clean rebuild 2026-08-16 10:09:26 +03:00
evgeny aee44943fa topo: invite вынесен в topo_group_invite, conn_mgr упрощён до 3-фазного open; member_sync: полный пересчёт merkle-дерева при старте
- topo_group_invite.c/h: invite/join через DIRECT + INVITE_INFO_REQ/RESP (из conn_mgr)
- conn_mgr: conn_mgr_open_invite → conn_mgr_open, убран invite-режим (-480 строк)
- topo_group_connect: подключение через node_conn_direct вместо conn_mgr
- member_sync: _rebuild_all_trees сносит merkle_tree_hash канала и собирает заново из peers (чистка осиротевших листов)
- member_sync: хеш блока A/B (update_sig/adm_tags) в док-комментариях
- обновлены doc-файлы и тесты
2026-08-16 09:19:09 +03:00
evgeny 1899f3a00f chat: verify_and_purge — единственное удаление мембера (битая подпись при чтении из БД) 2026-08-16 08:35:19 +03:00
evgeny 7f160666d9 chat: BGP→member_sync→consumers callback, BGP не удаляет member, member_sync не лезет в BGP-RAM 2026-08-16 08:31:14 +03:00
evgeny 195e903fe6 chat: колонка source (плейсхолдер topo_group, вне merkle-хеша) + placeholder_put 2026-08-16 08:18:12 +03:00
evgeny b69dcb4eab chat: убрать PEER_REMOVE (conn_down ≠ удаление мембера), удалить мёртвый member_sync_del 2026-08-16 08:14:28 +03:00
evgeny f550c80592 conn_mgr: унифицировать адреса из BGP и базы, TCP sock_meta, протокол в DIRECT 2026-08-15 14:39:14 +03:00
evgeny 5c2501b4f3 chat: BGP-driven address sync, member identity verify, join_sig fix
- Адреса узлов вынесены из wire-протокола member_sync/merkle/join/welcome
  в BGP/topo_group: node_addresses теперь пишет только topo_node_sqlite_addrs_put
  (источник истины ni->v4/v6_addrs), классификация config_type/nat_type в sockmeta
- Жёсткая анти-подделка: node_id=derive(x25519), ed25519 сверяется с доверенной
  таблицей nodes, NODEINFO node_id/pubkey mismatch отклоняется как forgery
- Единые member_sync_build_join_msg (112Б) и member_sync_build_update_msg:
  устранён баг _verify_join_sig (join_ts по смещению 80 вместо 104, длина 104→112),
  который рушил приём мемберов; все 7 мест подписи/проверки join_sig через хелпер
- Единое обновление своего мембера chat_core_update_my_member (вместо sync_my_addresses)
- Android standby: пауза таймеров (broadcast/db_sync/merkle_sync/chat_sync) через standby_wait
- name[16] → name[MAX_CONN_NAME_LEN] с truncation-warning в парсере
- conn_mgr: etcp_connect_cancel_by_node (use-after-free при отмене invite),
  промоушен conn в ncd после успешного invite
- Android GUI: foreground-gating опросов, сохранение портов интерфейсов, фикс udp-log flush
- chatgui: персистентный db_path; мелкие DEBUG-логи epoll/pool
2026-08-15 12:36:05 +03:00
evgeny c07bd385ca topo_node_sqlite: fix snprintf truncation warning in member_block_put 2026-08-15 12:36:05 +03:00
evgeny a4e73acceb db_sync: work-driven peer_check timer (stops when all peers synced) 2026-08-15 12:36:05 +03:00
evgeny 4e06db77af chat: gate broadcast on CHANGED, my_name via update_sig, set_online local-only gating 2026-08-15 12:36:05 +03:00
evgeny d0784a3ed7 member_sync: apply_record compare+update per-block, put/del/set_online, send_to 2026-08-15 12:36:05 +03:00
evgeny d62c370e22 merkle_sync: per-session out_q with backpressure, WAL timer, send_to, recompute returns changed 2026-08-15 12:36:05 +03:00
evgeny ad9235f772 topo_node_sqlite: granular member block/owner put functions 2026-08-15 12:36:05 +03:00
Evgeny ce387d58f1 tcp_proxy_client: fix pcb double-free — завершать conn при обоих FIN, не держать pc->pcb до TIME_WAIT 2026-08-15 09:30:03 +03:00
evgeny b203a48063 android: standby duty-cycle в фоне — defer ping/reconnect/probe через standby_wait
standby.h/c: duty-cycle ACTIVE/SLEEP, standby_wait() будит в burst сразу,
в SLEEP — до следующего burst; notify_network_activity для раннего пробуждения.
Интеграция (по флагу UTUN_HAVE_STANDBY, десктоп без изменений):
- conn_mgr: bg_ping/candidate_ping в SLEEP defer через standby_wait
- route_connectivity: probe_node откладывается (PROBE_STATUS_DEFERRED)
- topo_group_connect: пауза между попытками — активный режим 1с, фон standby_wait
- jni_bridge/ConfigProvider/NetworkActivityMonitor: set_active + сетевая активность
- chat_setting: standby_active_sec/sleep_sec/min_sleep_sec
+ юнит-тест test_standby (31 проверок)
2026-08-14 12:28:56 +03:00
Evgeny 1ec94de852 platform_compat: фикс Windows-сборки (Ipv4IfIndex/Ipv6IfIndex отсутствуют, GetBestInterfaceEx без const) 2026-08-13 23:15:14 +03:00
Evgeny f347dc8edc vibechat: single-instance (QLockFile) + переименование Chat→vibechat (бинарник, конфиг, hostname node name) 2026-08-13 23:10:43 +03:00
Evgeny f82dca3185 auto_socket: Windows-скан + список сокетов в Network Settings (add/remove, чистый rewrite конфига) 2026-08-13 22:58:52 +03:00
Evgeny d5119ac7c6 invite: не добавлять TCP-сокет в UDP-адреса; показывать протокол в Join 2026-08-13 21:51:54 +03:00
evgeny b0753bc19d android: кнопка invite в чате + фикс тап-зоны #, вывод протокола линков участника 2026-08-13 21:26:55 +03:00
Evgeny bcea4bc335 invite: убрать BOTH-протокол — адрес либо UDP либо TCP 2026-08-13 21:24:55 +03:00
Evgeny 37ef08a015 invite: диагностика адресов в цепочке invite→etcp_connect 2026-08-13 21:11:27 +03:00
Evgeny d166c18155 tests: exclude Linux-only test_auto_socket_dynamic from Windows build 2026-08-13 20:17:22 +03:00
Evgeny ace19e3f44 transport: fix socket_monitor_fire_one IF_NAMESIZE (Windows build) 2026-08-13 19:55:25 +03:00
Evgeny 6fa91ecb03 transport: use IF_NAMESIZE for cross-platform if_indextoname (Windows build) 2026-08-13 19:52:32 +03:00
evgeny 00b251aa4a test_merkle_sync: событийная сходимость вместо 20s-ожидания sync_pending
Стресс-тест (Stage 7) ждал sync_pending==0, но счётчик был разбалансирован
(merkle_sync_start перетирает done_cb без вызова) и никогда не доходил до 0,
из-за чего тест всегда выжигал таймаут 20s.

Теперь сходимость определяется по реальному условию — совпадению корневых
хешей — финальными раундами синка, управляемыми таймерами и done-коллбэками.
Попутно: исправлен iter==50->25, удалён мёртвый код.
2026-08-13 19:48:16 +03:00
Evgeny b7348f3237 chat headless: cross-platform sockets via socket_compat (Windows build) 2026-08-13 19:45:38 +03:00
Evgeny 0a102a7f43 proxy: guard if_indextoname bind block for Windows (_WIN32) 2026-08-13 19:35:13 +03:00
evgeny 7a40a1b0a8 etcp tcp: статус-флоу как в UDP + упрощение STCP API
- etcp_link_enter_ready_tcp: убраны прямые записи got_initial_pkt/initialized/
  links_up/tx_state и fire UP/INIT; переход через etcp_conn_ready +
  loadbalancer_link_ready (как в UDP INIT-обработчиках)
- tcp_server_on_link: убран дублирующий gop/session-блок (проверка в
  etcp_link_enter_ready_tcp единообразно), sc_init_ctx перед
  sc_set_peer_public_key (фикс ctx not initialized)
- stcp_link_connect(etcp_link, addr, port) вместо config-структуры;
  stcp_conn.etcp_conn для live-чтения gop/session_id при handshake
2026-08-13 19:14:14 +03:00
Evgeny 867f48c125 freebsd tun: поддержка ручного имени интерфейса (tunX) + автономер (tun) 2026-08-13 16:21:47 +03:00
Evgeny f3679b07a1 proxy: fix double-free on exit-side tcp error (re-entrant loopback) 2026-08-12 22:51:14 +03:00
evgeny 9348706736 etcp_keepalive: протокол смены keepalive-режима (standby/normal) + событие active mode
- новый модуль etcp_keepalive.c/h: keepalive-пакеты, таймер, адаптивный период,
  согласование при handshake вынесены из etcp_connections.c как есть
- протокол KEEPALIVE_REQ/RESP (0x09/0x0A): мобильный узел при смене active mode
  шлёт желаемый режим; отвечающий считает max (standby при любом standby), не-мобильный
  применяет как есть; инициатор применяет согласованный режим после RESP
- режимы: NORMAL (3с, адаптивный рост), STANDBY (30с, ka_pinned=1 без роста)
- active mode как событие: utun_add/remove_activity_cbk; подписчики topo_group
  (nodeinfo) и etcp_keepalive (рассылка REQ всем линкам)
2026-08-12 20:32:57 +03:00
evgeny ff84d0d986 1 2026-08-12 20:05:13 +03:00
evgeny ef115e0539 android: bind v4 server to real interface IP instead of 0.0.0.0 2026-08-12 19:17:37 +03:00
evgeny 0492b19468 topo_group_connect: infinite reconnect cycle + connecting status for GUI
- dedup double ETCP_CONN_STATUS_UP (UDP+TCP links) in topo_group_new_conn
- close conn_mgr handle on TIMEOUT in Phase2/3 (fix stuck 'already connecting')
- infinite Phase1→2→3 cycle with pause (1s active / 30s standby)
- CHAT_EVT_CONNECTING_NODES: yellow circle in member list while connecting
- Android: client_activity wiring via onStart/onStop + nativeSetActive
- function-level comments in topo_group_connect.c
2026-08-12 18:49:16 +03:00
evgeny 54ad03185b topo_group_connect: infinite auto-connect cycle with pause + connecting nodes notification
- client_activity: desktop/server now ACTIVE by default, only mobile starts STANDBY
- topo_group_connect: endless Phase1→2→3→1 cycle until connected
- Pause between failed Phase2/3 attempts: 1s active, 30s standby
- on_down always triggers cycle_restart when active_conn_count=0
- re-init destroys previous state and starts fresh
- CHAT_EVT_CONNECTING_NODES event notifies GUI of current connecting nodes
- Android: ConfigProvider default mode='auto', PTT fix, UDP log timer fix
2026-08-12 18:05:44 +03:00
evgeny 5d02a6d419 android: debug categories from GUI, fix poll log format, event loop sleeps until traffic
- android_jni_bridge: utun_bridge_set_debug_level actually calls debug_set_level / debug_set_category_level_by_name (was stub)
- SettingsScreen: CATEGORIES updated to match current g_debug_categories from debug_config.c
- ConfigProvider: persist category levels in [debug] section of generated INI config
- u_async: poll() log uses computed timeout_ms instead of confusing %d.%dms format
- u_async: timeout_node char name[16] → const char* name (no truncation, smaller struct)
- memory_pool: pool alloc/free debug logs moved from DEBUG to TRACE
- instance_lite: poll timeout changed from 10ms (100tb) to infinite (-1) — sleeps until traffic
- instance_lite: removed useless heartbeat timer
2026-08-12 09:02:19 +03:00
evgeny a73cc1beea fix: conn_reset only on got_initial_pkt==0 or sess_changed
- etcp_link_enter_init: new link on live conn sends NOINIT, not full reset
- etcp_link_enter_reinit: don't conn_reinit on single link recovery, preserve inflight
- handle_nl_link: fall through to remove_iface_sockets when if_indextoname fails
- duplicate INIT: add INFO log for diagnostic

Test: sequence checking (gaps=0 dups=0), exact socket/link counts, stale link
detection, conn health checks, full disconnect+reconnect phase
2026-08-12 00:18:33 +03:00
evgeny 8583d4216f fix auto_socket: handle_nl_link silently skipped on if_indextoname failure for deleted interfaces
When ip link del removes an interface, the kernel sends RTM_DELLINK after the interface is unregistered from the name table.
if_indextoname fails and handle_nl_link silently returned without calling remove_iface_sockets,
causing old ETCP sockets to accumulate indefinitely.

Fix: fall through to remove_iface_sockets(ifindex) even when if_indextoname fails.

Test: added diagnostics (diag_dump_state, per-category debug levels to file) to verify socket/links cleanup.
2026-08-11 23:30:31 +03:00
evgeny b503fa4418 etcp: cyclic link_id, link close on addr change, inflight nullify; test rewrite 2026-08-11 23:10:24 +03:00
evgeny d4a170b5c7 auto_socket: TCP links support, chat_sync retry on socket change, link dedup by is_tcp 2026-08-11 18:57:24 +03:00
evgeny fec3ea38a7 member detail: structured display with addresses, connections, BGP summary and socket names
- Extend MEMBER_DETAIL_LINK_SIZE to 63B with 20B sock_name per link
- Replace link_count with links_up/links_down (two uint16) in sock snap
- Add BGP hop list collection via topo_node_best_hop_list, post as CHAT_EVT_BGP_INFO
- Add direct RTT summary text via same event
- Rework AccountList detail panel: header + QTextEdit with monospace layout
- Display protocol (TCP/UDP), socket name line, colored dots for connections
- Show [up/down] link counts for self listen sockets
- Remove * prefix from link count display in Android
2026-08-11 16:05:35 +03:00
evgeny 5a42741d06 android: restore live member/connection updates lost in Kotlin source restore
- ChatRepository: MemberWire, parseMemberWire, cleanSocketName, parseMemberDetail, LinkStatus.linkId, LocalSocketState.linksUp/linksDown
- ChatViewModel: handlers 24/25/22, fix 16 (selectedMember update), 17 (in-place sync + status!=1 refresh), moder param
- MemberListScreen: compact header, no-dot addresses, TextAlign.Start connections, cleanSocketName, fonts 13sp, spacing 4dp, self *[up/down]
- SettingsScreen: linksCount -> *[linksUp/linksDown]
- NativeLib: moder in setMemberFlags JNI
2026-08-11 14:06:17 +03:00
evgeny 3f5dd43109 android: restore Kotlin sources + fix build
- Restore chat Kotlin sources deleted in d773f17 (refactor STCP)
  from d773f17~1: ChatApplication, MainActivity, data/, headless/,
  viewmodel/, ui/screens/, ui/components/
- Add InviteLinkDialog.kt (new file, never committed)
- Add missing libcrypto.a for arm64-v8a and x86_64
  (cross-compiled OpenSSL 3.5.5 via NDK 29)
- Build and install successful
2026-08-11 13:50:45 +03:00
evgeny f667e7bd74 fix: prevent use-after-free in etcp_ack_recv when link closed with pending inflight
When etcp_link_close() frees a link while INFLIGHT_PACKET entries
still reference it via last_link, a later ACK causes SIGSEGV in
bbr_main(link->bbr=NULL).

Fix:
- etcp_inflight_nullify_link(): nullifies last_link in both
  input_wait_ack and input_send_q for entries pointing to dead_link
- Called from etcp_link_close() before u_free(link)
- Guard etcp_ack_recv: check acked_pkt->last_link->bbr before
  accessing BBR/inflight stats
- Also fix bbr leak in etcp_link_close !link->conn path

Verified: 30/30 runs of test_etcp_link_stress pass (was ~50% crash)
Full suite: 71/71 pass
2026-08-11 13:37:42 +03:00
evgeny 63b1f1bc16 fix: init client_type and keepalive_interval in utun_instance_init()
utun_instance_init() did not call instance_init_common(), leaving
instance->client_type and instance->keepalive_interval as 0.
This broke the INIT handshake in V4 protocol: server wrote
keepalive=0 into INIT_RESPONSE, causing negotiate_keepalive
to see 0 and client to receive empty keepalive.

Also added guard: keepalive < 100ms clamped to 100ms with ERROR log.
2026-08-11 13:31:18 +03:00
evgeny bd6cd17bdf tests: fix close/restart detection after loopback_conn change
bb0866b changed close/restart notifications from NULL to loopback_conn().
Remove !conn check — use entry->len==9 alone to distinguish (9 bytes = svc_id+node_id, data always longer).
2026-08-11 12:21:10 +03:00
evgeny 0f1bd3d120 keepalive handshake: device_type + keepalive negotiation in UDP/TCP INIT
- Добавлен device_type (1 байт, CLIENT_TYPE_*) в INIT_REQUEST и INIT_RESPONSE
- Добавлен keepalive_interval (2 байта) в INIT_RESPONSE
- STCP handshake расширен с 43 до 46 байт (device_type+keepalive)
- negotiate_keepalive(): оба desktop/server → min, иначе (есть mobile) → max
- keepalive_interval вынесен в UTUN_INSTANCE для использования в handshake
- stcp_client_connect() принимает device_type и keepalive_interval
- peer_device_type/peer_keepalive_interval передаются через stcp_link в ETCP_LINK
2026-08-11 11:16:27 +03:00
Evgeny 43aca4a41b tcp_proxy_server: fix reentrant UAF in read_queue_drain_cb via loopback 2026-08-10 22:25:16 +03:00
Evgeny e7623e38b2 config_parser: add missing tcp_proxy_server options (fib, so_mark, netif, bind_ip); tcp_proxy_server: move diag log to TRACE 2026-08-10 22:07:12 +03:00
Evgeny bb0866b4c3 tcp_proxy: split svc_id into SERVER(0x04) and CLIENT(0x08), fix loopback conn=NULL 2026-08-10 22:00:39 +03:00
Evgeny 5a404fbc97 tcp_proxy_server: add fib/so_mark/netif/bind_ip socket options 2026-08-10 20:12:07 +03:00
evgeny 544c77efff unify TCP_SOCKET into ETCP_SOCKET with is_tcp flag; add local+remote addr to member detail links with arrow display 2026-08-10 12:06:32 +03:00
evgeny 477b4dfc98 move is_server indicator from per-connection to per-link in member detail and links JSON 2026-08-10 10:58:14 +03:00
evgeny 496f0f7d2b add server/client (S/C) indicator based on ETCP_LINK.is_server in connection monitor, member detail, and Android JSON 2026-08-10 10:47:52 +03:00
evgeny cf956cf2f0 add proxy server/client indicator (S/C) in connection monitor 2026-08-10 10:30:07 +03:00
evgeny f1941e61a2 fix: MemberLinkSnap/MemberSockSnap port int→uint16_t avoids stack garbage 2026-08-10 10:03:43 +03:00
evgeny 312876a0b2 fix: add strbuf.c/h to lib/Makefile.am 2026-08-10 00:20:02 +03:00
evgeny f2228202e3 fix: FORTIFY vsnprintf crash on Android + foreground service type
Root cause: etcp_on_up/etcp_on_down/etcp_conn_process_send_queue
used fixed buffers (links_str[256], l_status[256]) with snprintf
chains in loops over links. With 15+ links (IPv6 addresses),
sizeof(buf)-offset turned negative → vsnprintf size=(size_t)(-17)
→ FORTIFY abort on Bionic.

Fixes:
- lib/strbuf.h/c: safe growable printf buffer (auto-realloc)
- etcp.c: 3 snprintf chains → strbuf
- debug_config.c: BUF_ADVANCE macro prevents offset overflow
- android_jni_bridge.c: bridge_log va_list fix, set_member_flags guard
- chat_core.c: TAGS_ADVANCE macro in tags_commit
- AndroidManifest.xml: dataSync→specialUse (time limit exhausted)
- stcp: ed25519 pubkey exchange, tcp link addr/status in JNI bridge
- etcp_router: encrypted flag + ECDH per peer-pair
- node_conn_direct: TCP link support
- topo_node: group nodes cleanup
- media_delivery: buffer overflow fix
- test_stcp: ed25519 key exchange test
- tests: invite_group_create + Makefile update
2026-08-10 00:11:30 +03:00
evgeny ae80dae8ee tests: add Stage 7 merkle stress test — 13 instances, 5s spam, hash convergence
10 spammers (1-30ms random) + 2 observers (1-10ms), star topology via hub.
5s of concurrent modifications, then verify all 13 merkle root hashes
are identical and tree is internally consistent (recompute vs stored).
2026-08-09 19:39:41 +03:00
evgeny d8921f0e3d merkle_sync: fix infinite WAL growth from bg_check
- _recompute_bucket: skip INSERT OR REPLACE when hash unchanged,
  also skip DELETE when row doesn't exist (was always writing)
- bg_check interval: 100ms → 5000ms (was writing 10x/sec × 4KB/page
  into WAL = ~1.4GB over hours)
- bg_timer: round-robin across channels instead of only first
- bg_timer: periodic WAL checkpoint (every ~60s) to prevent
  unbounded WAL growth from any source
- test: sync ms_session struct copy with real definition

- stcp_link: handle race where conn already CLOSED before close
- stcp_client: remove double-free c->free_on_close = cli
- etcp_connections: stcp_link_close on TCP link down
- u_async: round sub-ms timeouts to 1ms to avoid busy-loop
2026-08-09 19:25:36 +03:00
Evgeny a646b35b9a etcp_router: router_deliver INFO→TRACE (спам на каждый пакет) 2026-08-09 18:15:25 +03:00
Evgeny e71f68736b socket_monitor: fix RT_ROUNDUP + пропавшая скобка функции (FreeBSD) 2026-08-09 17:56:39 +03:00
Evgeny f9417f06b4 BSD route sockaddr парсинг: addr/plen/gw/dev в INFO; убрал спам v6_send/v6_recv 2026-08-09 17:51:33 +03:00
Evgeny d8bae76ac7 socket monitor: BSD ifindex + UP/DOWN + фильтр спама; fallback idx%u → ? 2026-08-09 17:39:31 +03:00
Evgeny 15e3eeae65 socket monitor: осмысленные INFO-логи с именем интерфейса (UP/DOWN/NEW/DEL/route) 2026-08-09 17:32:50 +03:00
Evgeny 2cba839de4 Убрал DEBUG_CATEGORY_DEBUG: сообщения разбросаны по модульным категориям (+CHAT=25, MEDIA=23) 2026-08-09 17:24:15 +03:00
Evgeny 869f7c60d5 debug: add PROXY category, move all proxy logs to it 2026-08-09 16:31:21 +03:00
Evgeny 0de44e3117 remove spam TRACE messages: topo_group_find_conn_for_node, raw_recv, CRYPTO_DECRYPT_OK 2026-08-09 15:55:15 +03:00
Evgeny 6b6657c6b8 fix NTP ping-pong + normalizer counters
- ntp_node_time.c: respond to TIME_SYNC only when peer is unsynced (not always)
  Prevents infinite 10-byte ping-pong that ate bandwidth when both sides synced.

- pkt_normalizer.c: add in_total_pkts/in_total_bytes increments in
  etcp_input_ready_cb (real send path), matching pn_packer_send (test path).

- conn_mgr_core.c/conn_mgr_priv.h: full INVITE_INFO_RESP with channel info
2026-08-09 15:38:56 +03:00
evgeny 229a508022 etcpmon: add Makefile to git (was ignored by *.gitignore Makefile pattern) 2026-08-09 14:20:46 +03:00
evgeny c61c0639b9 remove utun.sh 2026-08-09 14:08:48 +03:00
evgeny 9066ae6a1f tests: add test_utils.h to git (was ignored by tests/test_* rule) 2026-08-09 14:06:34 +03:00
evgeny 6337d0f608 tests: add -I$(srcdir) to AM_CPPFLAGS for test_utils.h in VPATH builds 2026-08-09 14:01:31 +03:00
evgeny 4bb9f9e9c2 crypto: EVP_PKEY_generate → EVP_PKEY_keygen + OPENSSL_API_COMPAT для BSD/LibreSSL 2026-08-09 13:59:38 +03:00
evgeny d8337b9c3b whisper: опциональный configure --with-whisper; chatgui: парсер [debug]; stdint.h в AC_CHECK_HEADERS 2026-08-09 13:54:22 +03:00
evgeny 7f901c01e1 etcp: переупорядочить close + защита state==2 во всех API-точках; topo: правильная очистка group nodes и registry; chat/member: новый модуль member_sync; chatgui: обновления транспорта и UI 2026-08-09 12:39:53 +03:00
evgeny b2e4aadbf0 e2e: AES-128-CCM encryption via BGP router (ROUTER_FLAG_ENCRYPTED), ECDH key per peer-pair from NODEINFO pubkeys, cache 8 entries
- etcp_router.h/c: new flag ROUTER_FLAG_ENCRYPTED (0x04), etcp_route_send_encrypted(), inflight/retrans/send_q support
- utun_instance.h: e2e_ctx_cache[8] — per-peer sc_context_t cache with derived session key
- topo_node.c: topo_node_sign_self() — single point for Ed25519 self-signing of NODEINFO
- topo_node.c: fix stale x25519_self_sig after address update (topo_node_sign_self call)
- nat_detection.c: fix stale x25519_self_sig after in-place socket meta modification
- test_etcp_router_unit: 5 E2E tests (basic round-trip, no NODEINFO drop, tampered, cache reuse, ENCRYPTED+SIGNED combined)
- builds clean, 70/70 tests pass
2026-08-09 08:19:20 +03:00
evgeny ca176e1f24 encrypt overhead fix (33→36, noencrypt_len), mtu default 1280, connmonitor window, total_decrypted for TCP, check.sh 2026-08-08 16:56:32 +03:00
evgeny 9846599086 stcp: fix memory leaks and cleanup architecture
- stcp_conn_do_close: always deferred free for allocated conns (was else-if)
- stcp_link_on_stcp_close: don't null link->conn (close_impl needs it for rx_queue)
- stcp_link_close_impl: free rx_queue/tx_queue before close, null conn after cleanup
- stcp_client_connect: return NULL on derive_session failure (was returning dead cli)
- stcp_frame_decrypt/stcp_recv_try: dedup CRC check into stcp_check_crc helper
- test_stcp: add test8_srv_recv_close (error-recv close chain)
2026-08-08 11:42:16 +03:00
evgeny 5c8355ee63 stcp: unified recv FSM, handshake timeout, RX backpressure, opaque pointer fix
- stcp_recv_try / stcp_recv_set: unified chunk reader with raw+stream modes
  replacing duplicated stcp_conn_process_recv in server+client (~250 LOC removed)
- stcp_conn_read / stcp_conn_do_close / stcp_tx_queue_cb: unified shared functions
- continuous stream XOR: stcp_frame_encrypt now XORs len+data+CRC
- handshake timeout via uasync_set_timeout (5s server, 10s client)
- RX backpressure: stcp_rx_push / stcp_rx_resume_if_needed with hysteresis
- stcp_server_list_remove: new API fixes opaque pointer violation in auto_socket
  and 4 double-free bugs on shutdown
- removed hs_expected_len/hs_key_processed fields (replaced by callback chain)
- removed recv_buf<4096 debug workaround
- stcp_try_send const fix (takes ownership of malloc'd data)
2026-08-08 10:48:12 +03:00
evgeny d453e73ddb etcp: TCP links skip remote keepalive gating; chatgui: show interface names in member detail, fix destructor callbacks 2026-08-08 07:31:38 +03:00
evgeny 80e77f12c2 fix: auto_socket TCP duplicate sockets + NAT v4 address resolution
auto_socket.c: create_iface_tcp_socket returns stcp_server* instead of int,
ifa->v4_tcp/ifa->v6_tcp now properly track the created server, preventing
duplicate bind/listen on every reconcile_iface call.

etcp_connections.c: tcp_socket_add now resolves interface_addr via
get_interface_ip_by_index/get_interface_ipv6_addr_nl for any type (not just
PUBLIC), preventing NAT TCP v4 sockets from being destroyed due to INADDR_ANY.
2026-08-07 20:06:30 +03:00
evgeny de61a2dacc fix: auto_socket TCP hang + missing debug config apply in utun_instance_create_from_config
auto_socket.c: remove tsrv from inst->stcp_servers before stcp_link_server_destroy
when TCP socket has null addr — freed pointer stayed in linked list corrupting it
on next stcp_server_listen->stcp_server_list_add, causing hang

utun_instance.c: apply debug_level + per-category debug from config in
utun_instance_create_from_config (was only in utun_instance_create)

chat_core.c: verbose error log on invalid args
2026-08-07 18:31:30 +03:00
evgeny ec9edb6a7c Add test_etcp_link_stress: dynamic socket/link add/remove during traffic
- 2 instances, 1 initial UDP connection, 2000ms stress test
- Sends 1 packet every 1ms during stress
- Every 100ms: adds new UDP/TCP socket+link, removes oldest with stats
- Liveness check per cycle detects hangs
- Per-link report: type, state, decrypted/encrypted/acked bytes, retrans, RTT
- Removal stats: lifetime, encrypted, decrypted, acked, retrans, RTT, inflight
2026-08-07 17:32:28 +03:00
evgeny d6c78d96c4 stcp: dedup encrypt/decrypt/send to stcp.c, add pending queue for buffering, add server pubkey verification, add 4MB bulk test 2026-08-07 13:31:24 +03:00
evgeny 087ac9b0e4 stcp: close old TCP links on reconnect, check stcp_link_is_ready before send, notify ETCP on STCP close
1a: tcp_server_on_link — close existing TCP links before creating new ETCP_LINK
1b: etcp_tcp_send — skip send if stcp_link not ready
3a+3b+3c: stcp_conn_set_on_close → stcp_link_on_stcp_close → marks link DOWN → triggers reconnect
2026-08-07 12:46:46 +03:00
evgeny fd00e5dd15 cp 2026-08-07 12:21:41 +03:00
evgeny 38c9f229bc stcp: ed25519 pubkey exchange in handshake, fix instance destroy segfault, fix chat_tcp_test
- u_async.c: process immediate_queue BEFORE epoll_wait in uasync_poll
- stcp_link.c: defensive NULL etcp_conn/etcp_link in stcp_link_close
- utun_instance.c: move stcp_server_list_destroy_all to Phase J (before ETCP),
  add deferred drain in both Phase J and L
- stcp.h/c: STCP_HS_ENC_CLIENT 6→38, STCP_HS_ENC_SERVER 7→39,
  exchange ed25519_pubkey in both handshake directions
- etcp_connections.c: copy peer_ed25519 from STCP to ETCP_CONN in
  etcp_link_enter_ready_tcp; guard link->etcp in tcp_link_close_cb
- chat_sync.c: fix lk->conn NULL dereference for TCP links
- test_stcp.c: update stcp_server_create/stcp_client_connect callsites
- tools/chat_tcp_test: change transport udp→tcp
2026-08-07 00:48:27 +03:00
evgeny a3783b5f55 fix: re-index conn_queue_entry after setting peer_node_id
tcp_server_on_link sets conn->peer_node_id after etcp_connection_create
which adds the entry with peer_node_id=0. instance_find_conn couldnt
find TCP connections until the queue entry was re-indexed.
2026-08-06 23:14:57 +03:00
evgeny adac854c94 fix: TCP data flow — tx_state, keepalive, last_recv_local_time
- etcp_link_enter_ready_tcp: set tx_state=DATA_WAIT (was 0, blocking sends)
- etcp_conn_input: update last_recv_local_time on every packet (keepalive liveness)
- link_rx_cb: handle ETCP_KEEPALIVE before etcp_conn_input (conflicts with MEAS_RESP=0x08)
- KEEPALIVE handler: set recv_keepalive, remote_keepalive, link_status, keepalive_recv_count
- test_stcp_traffic: etcp_send instead of raw stcp_link_send
- test_etcp_stcp: rewrite for utun_instance + etcp_connect
2026-08-06 23:09:01 +03:00
evgeny 928fa7731c refactor: STCP connections as ETCP links (is_tcp) 2026-08-06 22:14:44 +03:00
evgeny d773f1724a refactor: STCP connections as ETCP links (is_tcp)
- Add is_tcp, tcp_link, tcp_reconnect_* fields to ETCP_LINK
- Remove transport_link from ETCP_CONN, add tcp_link_count
- stcp_link: replace embedded etcp_conn with pointer; add etcp_link backpointer
- stcp_link: link_rx_cb routes ETCP data to etcp_conn_input
- stcp_link: call on_close_cb in close
- New TCP functions: etcp_link_enter_ready_tcp, etcp_tcp_link_start_connect,
  etcp_tcp_link_start_reconnect, tcp_link_close_cb, etcp_tcp_send
- etcp_link_new: handle NULL conn for TCP links; skip enter_init for TCP
- etcp_link_close: close stcp_link for TCP
- etcp_encrypt_send: redirect TCP to etcp_tcp_send (skips AES-CCM)
- tcp_server_on_link: create ETCP_LINK(is_tcp=1) + enter_ready_tcp
- etcp_connect: TCP addrs -> etcp_tcp_link_start_connect
- Remove ctx->tcp_link/tcp_ready
- test_stcp_traffic: find stcp_link via links->is_tcp instead of transport_link
- 66/68 tests pass; 2 STCP tests expected to fail (integration not complete)
2026-08-06 22:14:32 +03:00
evgeny 937af4c26a fix: STCP handshake ordering + invite peer_node_id match + test script
- stcp_server: send handshake response BEFORE on_ready to prevent ETCP INIT data mixing into client recv buffer (crc mismatch)
- conn_mgr: relax peer_node_id match for INVITE_INFO_RESP when peer_node_id==0 (STCP connections)
- chat_tcp_test: remove invalid 'local' outside function
2026-08-06 19:28:52 +03:00
evgeny 62504dc3b7 diag: full buffer dumps for CRC debug 2026-08-06 19:05:23 +03:00
evgeny c66b63ea39 diagnostics: stcp session_key + xor data dumps 2026-08-06 18:57:42 +03:00
evgeny dfe9d36783 wip: STCP TCP chat integration — diagnostics, invite TCP fallback, test infra 2026-08-06 18:41:41 +03:00
evgeny e2d8f68c9a fix: peer discovery queries include IPv6 (family IN (4,6))
get_public_peers, get_local_peers, get_supernode_peers
were filtering na.family=4 only — never found v6 peers.
Changed to na.family IN (4,6).
2026-08-06 14:20:33 +03:00
evgeny 25ebad9f52 fix: UDP packet size and addr_type classification
- etcp_connections.h: add WIRE_OVERHEAD, IP_UDP_OVERHEAD_V4/V6, SC_ENCRYPT_OVERHEAD
  (properly accounts for v6 IP(40)+UDP(8)+SC(3)+CCM tag(16) = 67 vs old UDP_HDR_SIZE=28)
- etcp_connections.c etcp_encrypt_send: use WIRE_OVERHEAD instead of UDP_HDR_SIZE
- etcp_connections.c send_init_response: clamp padding to max_data = mtu - WIRE_OVERHEAD
- chat_core_priv.h: add addr_type_from_ip() classifier
- chat_profile.c, chat_sync.c, member_sync.c: use classified addr_type instead of hardcoded 0
- chat_sync.c: skip zero nat_addr in JOIN, fall back to interface_addr
- topo_node_sqlite.c, member_sync.c: remove addr_type=0 filter from SELECT/DELETE
- chat_profile.c, member_sync.c: DELETE all addresses (not just addr_type=0)
2026-08-06 14:08:20 +03:00
evgeny 5066cff333 fix: IPv6 bind to permanent SLAAC address instead of ::
- etcp_connections.c: detect interface_addr before bind(),
  use specific IPv6 addr instead of in6addr_any
  (netlink path for Linux, connect trick for Android)
- chat_profile.c: filter PRIVATE/LOCAL sockets from
  node_addresses sync
- conn_mgr_core.c: diagnostic logs in cm_invite_tcp_connect,
  continue to next addr on TCP connect failure
- node_conn_direct.c: diagnostic logs in ncd_create_links
- auto_socket.c: suppress spurious ADDR_CHANGED events
- etcp_api.c: extended socket event log
2026-08-06 13:17:39 +03:00
evgeny dc8c9f7875 fix: cm_invite_pending — deferred free + cancelled guard
Корень краша: cm_handle_invite_info_resp вызывал cm_invite_cleanup(inv)
без unlink из invite_list — висячий указатель при conn_mgr_destroy → SIGSEGV.

Механизм защиты:
- cancelled:1 в cm_invite_pending — блокирует все коллбэки после cancel
- cm_invite_cancel: cancelled=1 первой строкой, deferred free через uasync_post
- Все коллбэки (NCD, TCP ready/close, overall timer): guard + DEBUG_WARN
- Success path: temp_nq=NULL (передача владения), вызов cm_invite_cancel
- INV_CANCEL логи: skip (NULL) / done — видно что реально сделано
2026-08-06 10:54:56 +03:00
evgeny f476188566 fix(chat): node online status race, WELCOME peer_needed off-by-one, JOIN missing proto byte
- node_put/node_update_verified: set online=1 on INSERT (nodes created when conn is UP)
- channel_peers_all: fix peer_needed calculation (atl byte was missing)
- cs_send_channel_join: add proto byte to address serialization matching all parsers
2026-08-06 01:18:27 +03:00
evgeny 38eb51c679 fix: Android event handler lost after restart — g_event_handler now preserved
Root cause: instance_lite_restart was creating new thread without
re-setting g_event_handler, causing all chat events (CHANNEL_UPDATED,
CONNECT_RESULT, etc.) to be silently dropped after app restart.

Fixes:
- instance_lite_restart: save/restore g_event_handler across hard restart
- instance_thread restart loop: re-call chat_event_set_handler after uasync_create
- utun_bridge_restart: call instance_lite_set_event_handler before restart
- AppEventHandler SharedFlow replay: 1 → 16 to not lose early init events
- ViewModel: added diagnostic logging for channel/connect events
- Keep diagnostic logs in instance_lite (set_handler, chat_event_forward NULL check)
- instance_event_handler: removed static for cross-block forward declaration
2026-08-06 00:06:28 +03:00
evgeny ef637d33a6 auto_socket: модуль авто-управления сокетами при изменении сетевых интерфейсов + fix pre-existing ASAN-багов
- auto_socket.c/h: создание/удаление UDP сокетов автоматически при изменении интерфейсов через netlink/route socket
  При auto_sockets=yes пропускает ручные [server] сокеты
  Тип сокета: PUBLIC если публичный IP, иначе NAT
  Для новых сокетов добавляются линки ко всем активным NCD-соединениям
  При удалении интерфейса линки убираются, сокет закрывается
  Platform-agnostic API для использования из Android/chatgui

- node_conn_direct.h/c: ncd_add_socket_links / ncd_remove_socket_links — управление линками на конкретном сокете

- socket_monitor.h/c: экспорт socket_monitor_update_if_addr для auto_socket

- config_parser.h/c: опция auto_sockets в [global]

- init_sockets(): пропуск ручных сокетов при auto_sockets=yes

- utun_instance: интеграция auto_socket_init/destroy

- media_delivery: fix stack-buffer-overflow ack[3] → ack[9]

- topo_node: fix heap-use-after-free в topo_group_update_my_nodeinfo — сохранение node_id/ver перед registry_store, использование return value
2026-08-05 23:09:59 +03:00
evgeny a0ba6dee21 fix: tgc_phase1_timeout closing all handles including connected ones 2026-08-05 18:57:35 +03:00
evgeny 513ec75c28 fix elementary format string bugs in log messages
- etcp.c:466: %d→%s for log_name (was printing garbage like -2012603448)
- etcp_connections.c:2235: Server type %d→%s with server_type_str()
- etcp_connections.c:845,1578,1587: type=%d/nat_type=%d→%s with helpers
2026-08-05 18:35:55 +03:00
evgeny 4c1ea11022 conn_mgr: split handler into router (routed) and direct paths
conn_mgr_router_recv_handler (etcp_router_bind): DIRECT_REQ, INTERM_EXCHANGE, DISCONNECT
conn_mgr_direct_recv_handler (etcp_bind): INVITE_INFO_REQ, INVITE_INFO_RESP
Each transport path now has its own handler — no more mixed concerns.
2026-08-05 18:16:18 +03:00
evgeny 85f23e0a94 conn_mgr: move etcp_router bind/unbind from per-group to instance level
svc_id=17 (ETCP_RT_ID_CONN_MGR) is a singleton resource — bind once
at utun_instance_init, unbind once at utun_instance_destroy.
Previously each conn_mgr (one per group) called bind/unbind separately,
causing 'overwriting' and 'not bound' WARNs on every group init/shutdown.
Handler dispatches by group_id from message — no need for per-group binding.
2026-08-05 18:04:43 +03:00
evgeny d296074330 etcp: show which specific link went DOWN + always use registry node_name
1. etcp_on_down: add down_link param — log the specific lost link + remaining
   Connection DOWN — link 192.168.40.247:2122 lost, remaining: [::1]:6669, [::2]:9970
   Connection DOWN — closing, links: ... (for reinit/close)

2. etcp_on_link_down: pass the triggering link through to etcp_on_down

3. etcp_update_log_name: always check registry for node_name
   (n_5a1b... → SM-A525F when name is available in registry)
2026-08-05 17:41:44 +03:00
evgeny 1b6fd1c29b etcp: use human-readable names for socket type and NAT type in logs
Add helper functions server_type_str() and nat_type_str() to convert
CFG_SERVER_TYPE_* and NAT_TYPE_* codes to readable names.
Now: Add Socket type=PUBLIC instead of type=1
     self-detect check: type=PUBLIC nat=UNKNOWN instead of type=1 nat_type=0
2026-08-05 17:32:53 +03:00
evgeny 5f24aad6a6 etcp: use node_name from registry in log_name for human-readable connection names
etcp_update_log_name: if name is empty, look up peer_node_id in topo_node_registry
and use node_name (e.g. 'SM-A525F') instead of empty or generic name.
Result: [45C2->03D3 [SM-A525F]] instead of [45C2->03D3 []]
2026-08-05 17:30:04 +03:00
evgeny 83aba82887 etcp: add link addresses to all UP/DOWN/close log messages
- etcp_on_up: list up-link addresses after connection summary
- etcp_on_down: show all link addresses that were lost
- Link UP (server/client): add addr=X
- Connection established: add addr=X
- Link closed: add addr=X before traffic stats
- Link down (keepalive timeout): add addr=X
Now: [conn] Link 1 UP (server, mtu=1600, addr=192.168.40.247:2122)
2026-08-05 17:24:41 +03:00
evgeny cf8236266b chat_sync: find_conn NOT FOUND INFO→DEBUG — fires for unrelated stub nodes during reinit storms 2026-08-05 14:26:00 +03:00
evgeny e51d8ddda2 chatgui: fix ERROR→WARN/DEBUG for normal situations
- member_sync apply_items: empty payload is normal sync completion, not error
- topo_node_sqlite node_load: no addresses is normal for stub/seed nodes
- conn_mgr_core: node skipped when not in routing (normal for nodes w/o addrs)
- topo_group send_nodeinfo: downgrade ERROR→WARN in DEBUG category
- db_sync INIT_RESP: protocol details belong in DEBUG, not INFO
2026-08-05 14:15:43 +03:00
evgeny 5feeb2c169 chatgui: clean up log — remove spam, move messages to proper categories
A-B: Remove gui_bridge_post_uasync_fn log, MainWindow::onMessageReceived, InputBar send text
B1: Move lifecycle messages to GENERAL (utun_node, main, mainwindow, animtimer, sound_manager, messagelist, db_manager)
B2: Move chat events to CHAT_SYNC (MSG_RECEIVED, CHANNEL_UPDATED, CHANNEL_PEERS_ONLINE, DB_READY)
B3: Move peer/connect events to MEMBER_SYNC (CONNECT_RESULT, NEW_PEER, AUTO_CONNECT, NODE_CHANGED)
C: Lower level of C-side spam (recalc, send OK, PUSH ACK, connect_channel) INFO->DEBUG
D: Fix duplicate PUSH log line in db_sync.c
E: Add debounce to CHANNEL_PEERS_ONLINE — log only on change
Misc: Remove sort=0 from ChannelList::loadChannels, improve DB_READY message
2026-08-05 12:50:27 +03:00
evgeny be0e76c9a6 debug: реорганизация категорий — SYS(1), ETCP(4) расширен, CONNECTION(3) очищен; fix stcp_client.c sock_err до connect()
Категории:
- DEBUG_CATEGORY_SYS=1: u_async + ll_queue + memory + timing (было 4)
- DEBUG_CATEGORY_ETCP=4: ETCP + normalizer + BBR (было 3)
- DEBUG_CATEGORY_CONNECTION=3: только события статуса линка/подключения (INFO)
- DEBUG_CATEGORY_KEEPALIVE=21: только периодический мониторинг (DEBUG)
- Удалены: NORMALIZER, BBR, CONNECTIVITY
- Устаревшие DB_SYNC → CHAT_SYNC + MEMBER_SYNC (предыдущий коммит)

CONNECTION+INFO — только: Connection UP/DOWN, Link UP/down, Link initialized/added,
  NAT address init/changed, init server/client.
CONNECTION+DEBUG — retry, collision, reinit, socket register.
KEEPALIVE — init_timer SUPPRESSED, timer started, keepalive skipped.
REINIT — ETCP+INFO.
Connection status changed — GENERAL (пользовательский callback).
Dedup link_status: if(old_link_status) перед Link down.
Убран дублирующий Conn:%s из Link down, добавлен link_id в client added link.
Молчаливые ошибки: +DEBUG_ERROR в 15 местах (merkle_sync, member_sync, db_sync).

Fix: stcp_client.c — socket_get_error() вызывался ДО connect(), проверяя
застарелое errno вместо ошибки connect(). Неблокирующий connect() возвращает
-1+EINPROGRESS (норма), но сравнивался с мусором → ложный FAIL.
Все 4 STCP-теста исправлены (68/68 pass).
2026-08-05 10:43:32 +03:00
evgeny 1895f23034 add chat_sync_invite_to_channel_with_addrs — connect with addresses, send CHANNEL_INVITE, 3s timeout, post result to GUI 2026-08-04 23:14:03 +03:00
evgeny 44c99f603d move INV button from footer into channel delegate rows (second button left of plus) 2026-08-04 22:57:36 +03:00
evgeny 24316e8cf8 split InviteDialog into ShareInviteDialog + InviteByLinkDialog, add INV button to channel list 2026-08-04 22:42:22 +03:00
evgeny 5632bc9c8b invitedialog: fix IPv4 byte order, show v6 prefix, add channel_id to diagnostic 2026-08-04 22:30:56 +03:00
evgeny 262dc9bb51 invitedialog: show decoded link info on text change (paste), not just on Send click 2026-08-04 22:23:54 +03:00
evgeny f8d98cab47 add error logging to all silent returns in chat_sync invite/join chain + GUI self-invite check 2026-08-04 22:20:01 +03:00
evgeny 21a0ee6caf conn_mgr: fix use-after-free in invite NCD callback cleanup
cm_invite_fail and cm_handle_invite_info_resp: always close invite's NCD
handle before freeing the invite struct. Previously the NCD close was
skipped when nq->handle != NULL (success path never closed it at all),
leaving cm_invite_ncd_callback registered with a dangling pointer.
On subsequent NCD DOWN event the stale callback caused SIGSEGV.
2026-08-04 20:53:38 +03:00
Evgeny fa5878ef4b media_download: fix md_mkdir_parent for Windows (drive letter, backslash), add errno to fopen errors, sanitize base_filename with WARNING 2026-08-04 19:09:46 +03:00
Evgeny cf1843e586 fix: etcp_connect_timeout_tb=0 in utun_instance_create (missing init)
utun_instance_create() allocated with u_calloc (zeroed) but never
set etcp_connect_timeout_tb. utun_instance_create_from_config() had it
at line 371, but utun_instance_create (used by chatgui) was missing.
Timer with tb=0 fires instantly → CLOSE sent before handshake completes.
Diagnostics confirmed: value_tb=0, tb=0, delta_tb=0 on Windows chatgui.
2026-08-04 18:25:24 +03:00
Evgeny 2b10f53466 diagnostics: ncd connect timer SET/FIRE logs + WSAEWOULDBLOCK fix + cm_invite_fail details
- u_async.c: log ncd_connect timer expiration calculation (tb, now_tb, exp_tb, exp_ms, delta_tb)
- node_conn_direct.c: SET logs at 4 timer creation points with value_tb/now_tb/path; TIMEOUT log with now_tb
- conn_mgr_core.c: cm_invite_fail entry log + CLOSE/SKIP branch detail
- stcp_client.c: fix WSAEWOULDBLOCK (10035) treated as error on Windows (add ERR_WOULDBLOCK check)
2026-08-04 18:16:53 +03:00
Evgeny 9d797c91eb chatgui: postChatSetting в хедер, chat_core_get_inst include, chat→chatserver 2026-08-04 17:28:05 +03:00
Evgeny 3b20c84c4d mkdtemp: #ifdef _WIN32 в локальных обёртках, без test_utils.h 2026-08-04 17:16:55 +03:00
Evgeny ebd975a03b mkdtemp → test_mkdtemp из test_utils.h для кроссплатформенной сборки (MSYS2/MinGW) 2026-08-04 17:13:45 +03:00
Evgeny 98f3049e6b mkdir → utun_mkdir в тестах для кроссплатформенной сборки (MSYS2/MinGW) 2026-08-04 17:02:31 +03:00
Evgeny 1b1bf91e79 Убран неиспользуемый #include <netdb.h> ломающий сборку на MSYS2/MinGW 2026-08-04 16:42:03 +03:00
Evgeny d816493af6 Инвайт-ссылка chatserver печатается в лог всегда, не только в foreground 2026-08-04 16:32:17 +03:00
Evgeny ae1add5437 parse_chatserver: молчаливый провал, ошибка только после chat_setting_set 2026-08-04 16:28:18 +03:00
Evgeny 876155688f Устранено дублирование: storage_total_size/unit_size вместо storage_maxsize_gb/autoload_maxsize_mb 2026-08-04 16:17:06 +03:00
Evgeny 7a8d249915 Убраны аудио-настройки из chatserver — перекодировка на стороне клиента 2026-08-04 16:09:22 +03:00
Evgeny 74bef5110c Обновлён сэмпл конфига: chatserver — db_path + chat-настройки 2026-08-04 16:08:01 +03:00
Evgeny c22991f7d9 Добавлена секция [chatserver] в конфиг — объединяет активацию чата и настройки в одной секции 2026-08-04 16:05:06 +03:00
evgeny 712d1b53df 1 2026-08-04 15:29:25 +03:00
Evgeny 26f4da1b7c socket_monitor: cross-platform support (Linux netlink, FreeBSD AF_ROUTE, Windows IP Helper)
- platform_compat: get_interface_ipv6_addr_nl stub uses get_interface_ipv6_by_index fallback
  - socket_monitor: extract common update_if_addr/fire_one, platform dispatch
  - FreeBSD: PF_ROUTE socket with RTM_IFINFO/NEWADDR/ADD event monitoring
  - Windows: NotifyUnicastIpAddressChange + NotifyRouteChange2 via uasync_post
2026-08-04 15:20:20 +03:00
Evgeny fe79dcf318 libopus: remove -DHAVE_ALLOCA_H — alloca.h not available on FreeBSD 2026-08-04 15:07:36 +03:00
Evgeny 5cbd12f8dd platform_compat: guard linux/netlink.h and nl function with __linux__ (FreeBSD) 2026-08-04 15:05:28 +03:00
Evgeny 6a35fc4b45 u_async: add pipe() fallback for non-Linux POSIX wakeup (FreeBSD) 2026-08-04 15:01:36 +03:00
evgeny de9e467b39 media_delivery: BLOCK_PROCESSING + relay streaming + per-file download limit
- block_availability: status column (0=processing, 1=completed)
- md_ba_complete_block(): DELETE old + check exists + INSERT new id
- BLOCK_PROCESSING subcmd (0x10) — INSERT with new id, no REPLACE/IGNORE
- BLOCK_RELAY_FULL subcmd (0x11) — redirect when source/relay at capacity
- relay_block_ctx: downstream forwarding from chunk file
- md_file_load: per-file source download limit (max_downloads_per_file)
- RELAY_FULL from source node when at capacity
- relay_full handler: retry existing peers, not just new ones
- sync.md: full documentation
2026-08-04 11:42:59 +03:00
evgeny 7263bdcf56 media_download tests: backpressure, block relay, OOB block_id, assembly
5 new tests in test_media_delivery_download:
- conn_cb starts only first block (backpressure)
- conn_cb with multiple peers
- BLOCK_DONE triggers next block relay from same peer
- assembly after all blocks validated
- peer with more blocks than dl (OOB fix verification)

md_dl_conn_cb made non-static for test visibility
md_dl_send added NULL guard for test instances without routing
media_download.h: added conn_cb extern declaration
2026-08-04 00:22:18 +03:00
evgeny 41e9c9b0e6 media_download: fix block_id OOB, add backpressure, close conn_mgr
- md_dl_send_block_req now takes explicit block_id (not peer-local index
  into dl->block_ids) — fixes OOB 'aaaaaaaa...' garbage block_id
- conn_cb sends only first unstarted block per peer (backpressure)
- BLOCK_DONE success triggers next block from same peer
- conn_mgr handles stored in peer->cm_handle, closed on assembly/cancel
2026-08-04 00:02:20 +03:00
evgeny 3665a9bc23 chat_msg: fix strstr on non-null-terminated blob/text data
chat_core_attachment_download: use sqlite3_column_text for null-term
on_msg_inserted: copy data to local null-term buf before strstr
Both caused b64_decode to misread body from garbage memory
2026-08-03 23:31:56 +03:00
evgeny 4c366f3884 desktop messagelist: decode *<b64> prefix for media message display
setVoiceMessageRoles now checks for '*' marker, translates URL-safe
base64 (-_ → +/) and decodes to restore original display data.
Fixes attachment filenames and voice waveform for new messages.
2026-08-03 23:17:28 +03:00
evgeny a0049a4187 b64 prefix: * marker to distinguish base64-encoded display data from old plain text
* prefix prepended in on_media_registered before b64 data
on_msg_inserted, chat_core_attachment_download, get_messages_json
only attempt b64_decode when prefix starts with '*' marker
2026-08-03 23:10:15 +03:00
evgeny ea6eeae210 msg b64: base64-encode display prefix in d-field to avoid pipe/quote breaking parse
b64_encode/decode (URL-safe A-Za-z0-9-_) in chat_core_priv.h
on_media_registered encodes msg_data before full_data
on_msg_inserted/chat_core_attachment_download decode base_filename
android_jni_bridge decodes prefix for UI display
2026-08-03 23:02:49 +03:00
evgeny 6a02a16543 attachments: preview bar + combined send (Android + desktop)
Android: file picker → pending preview bar (name+size+X) →
         combined send with text; cache copy, save dialog for downloads

Desktop: attach stores pending, preview bar before send,
         combined with text in onSendClicked

C: attachment_sender.c — include filename in msg data for display
2026-08-03 22:36:33 +03:00
evgeny e3df5008da etcp: fire link_status_cbk in etcp_link_new for new links 2026-08-03 21:46:22 +03:00
evgeny 13c6ef2210 member_sync: fix p-6 for ip pointer, add stcp link addr+status to JNI bridge 2026-08-03 21:13:08 +03:00
evgeny b6f9ebc8a8 chatgui: TCP link status via stcp_link_is_ready 2026-08-03 21:04:46 +03:00
evgeny 5d23f461a7 member_sync/topo_sqlite: add proto byte to address serialization
- _compute_bucket_hash: SELECT protocol, write proto byte
- _member_get_items: SELECT protocol, write proto byte
- _member_apply_items: consume proto byte in skip calculation
- topo_node_sqlite_channel_peers_all: SELECT protocol, write proto byte
All 5 writers and 5 readers now consistently include protocol byte.
2026-08-03 20:59:26 +03:00
evgeny 8023f6ee70 stcp: separate v4/v6 sockets, protocol byte in chat addr format, error on bind fail
- stcp_server_create: support AF_INET and AF_INET6 V6ONLY=1 (no dual-stack)
- Add listen_family to stcp_link_config
- init_sockets: return error (not partial) on any bind failure
- add chat_core_sync_my_addresses() call at chatgui startup
- collect_my_addrs: add proto byte (1=UDP, 2=TCP) to binary format
- chat_sync.c JOIN/WELCOME/PEER_UPSERT: read proto byte, write protocol column
- member_sync.c: read proto byte, dynamic protocol bind
- topo_node.c: debug logs for nodeinfo changed detection
- node_config.cpp: type=public for all servers
2026-08-03 20:35:46 +03:00
evgeny 3deaa0eabe stcp: multiple servers support via linked list in UTUN_INSTANCE
- Replace instance->stcp_server (single pointer) with stcp_servers linked list
- Add stcp_server_list_add/destroy_all/count API in stcp_link.h/c
- Fix port extraction for both AF_INET and AF_INET6 in init_sockets
- Filter out 0.0.0.0 and [::] addresses in chat_profile.c TCP address sync
- Add type=public to servers in node_config.cpp saveFull()
2026-08-03 19:39:50 +03:00
evgeny e1e54df4f4 fix: invite не затирает имя узла, имя читается из member_sync peers.userinfo
- conn_mgr_core.c: убрана запись node_name из cm_handle_invite_info_resp
- topo_node_sqlite.c: name в nodes не перезаписывается пустой строкой
- android_jni_bridge.c: bridge_get_node_name читает peers_<ch>.userinfo если nodes.name пуст
2026-08-03 18:50:11 +03:00
evgeny 253129cb27 topo_node_sqlite: убрал запись node_addresses из BGP-пути (только nodes). Адреса — через member_sync. 2026-08-03 15:17:51 +03:00
evgeny ddf696bcf6 refactor: TCP socket addresses — sync to chat DB, topo_node, config file single source
- chat: collect_my_addrs extends to TCP + IPv6, chat_core_sync_my_addresses writes TCP addrs
- topo_node: replace CFG_SERVER with TCP_SOCKET for TCP addresses, simplify NAT logic
- etcp_connections: add IPv6 interface addr resolution for TCP sockets
- Android: INI config file as single source of truth for servers, remove JSON/Gson
- Android: filter temporary IPv6 addresses, fix restart with socket cleanup
- instance_lite: sync addresses after socket creation, proper restart cleanup
2026-08-03 13:57:42 +03:00
evgeny 740ed2a93a fix: NCD UAF, conn_mgr invite protocol, node_id derivation, link reuse
node_conn_direct:
- ncd_init_cb: skip UP when links_up==0, ncd_up_cb delivers when link comes up
- close/force_close: handle_count safety checks, callback clearing
- open/open_node case 2: create links on reused dead conn
- ncd_create_links: skip duplicate link on same conn (stale->etcp==conn)
- set_callback: optional callback change on live handle

conn_mgr:
- cm_ncd_callback: DOWN only delivers event (handle stays alive)
- cm_ncd_callback: TIMEOUT force_close BEFORE cm_deliver_event
- cm_deliver_event: read handle fields to locals before callback (reentrany-safe)
- INVITE_INFO_REQ/RESP: add duplicate cmd byte for direct-send protocol
- cm_handle_invite_info_resp: search invite by group_id in channel conn_mgr

Android:
- node_id computed by sc_derive_node_id_from_pubkey in C, not Kotlin
- utun_bridge_join_channel: recompute node_id, log mismatch
- invite_link_c: compute nodeId from pubkey via sc_derive_node_id_from_pubkey
- NativeLib.deriveNodeId, InviteLink.kt uses C function
2026-08-03 09:51:06 +03:00
evgeny 4b07d59043 chat: skip own addresses from remote in member_sync_recv
Remote peers should not overwrite local node_addresses.
When member_sync_recv processes data about our own node_id,
we trust locally-synced addresses (from sync_my_addresses)
and ignore address data from the remote peer.
2026-08-02 20:42:40 +03:00
evgeny 63272abe53 chat: fix duplicate v6 address from v4 socket's interface_addr
sync_my_addresses used interface_addr even when its family
differed from the socket's local_addr family. A v4 socket
bound to 0.0.0.0 could pick up a v6 interface_addr on
dual-stack Android, inserting a duplicate v6 entry.
Now interface_addr is only used when families match.
2026-08-02 20:38:42 +03:00
evgeny c2191aad3b chat: user-friendly INFO logs for invite/connect flow
- chat_sync.c: invite entry, JOINED/TIMEOUT messages
- conn_mgr_core.c: invite start (addrs+udp+tcp), NCD UP/DOWN/TIMEOUT,
  TCP connect/ready/close, membership check send/receive/confirm/deny,
  overall timeout
- node_conn_direct.c: NCD event names instead of numbers
2026-08-02 20:32:01 +03:00
evgeny c35135cb52 topo_group: migrate CONN_MGR entries to ll_queue, simplify TOPO_GROUP_NODE
- CONN_MGR entries: replace dynamic array with ll_queue (hash=256, key_size=8)
  cm_find_entry now O(1) via queue_find_data_by_index
  cm_ensure_entry uses queue_entry_new + queue_data_put_with_index
  cm_entry_cleanup removes from queue and frees entry

- TOPO_GROUP_NODE: replace handles** array with single CONN_MGR_HANDLE*
  remove topo_node_handle_register/unregister/count — dead indirection
  nq->handle = h / nq->handle = NULL in all call sites

- TOPO_GROUP_NODE: remove dead fields dirty, best_socket, tranzit_data/count
  dirty replaced with local variable in nat_detection.c
  tranzit_data removed from wire format (TOPOMSG_NODE), serialize, deserialize
  update struct layout: one field per line with comments
2026-08-02 16:45:52 +03:00
evgeny f5efaa495b socket_monitor: extract socket monitoring from etcp_connections, add per-instance context
- New files: transport_layer/socket_monitor.c/h — socket state monitoring with per-instance ctx
- etcp_api: expose get_latency/1s_pkt_count measuring, add etcp_hashrnd for tests
- utun_instance: store socket_monitor per instance, add is_socket_ok check
- topo_node: detect IPv4 addresses with port from listening sockets
- topo_group: use node_name for display in routes
- Makefile.am: add socket_monitor.c to build
2026-08-02 12:16:42 +03:00
evgeny 14d1673f06 AGENTS.md: update structure to reflect actual codebase layout
- Debug categories: 25→27, add CONNECTIVITY and DB_SYNC
- Directory structure: reflect real subdirs (transport_layer, routing_layer, chat, proxy, lwip_tcp, media_delivery, media_async)
- File overview: reorganize by actual subdirectories, fix paths
- BBR: src/BBR/ → src/transport_layer/BBR/
- topo_node_lmdb → topo_node_sqlite
- Remove sha256.c/h (deleted), add missing lib/ modules (sqlite3, opus_codec, audio_compressor, json_flat)
- Add missing modules: ntp_time, ntp_node_time, media_delivery, media_async, nat_detection, topo_recovery, topo_group_connect, node_conn_direct, socket_monitor
- Add chatgui-android to tools/
2026-08-02 12:16:25 +03:00
evgeny fdbb5b7bc0 ipv6: use netlink RTM_GETADDR to properly detect permanent vs temporary IPv6 addresses
getifaddrs() ifa_flags contains interface flags (IFF_UP etc.), NOT address
flags (IFA_F_TEMPORARY). Since IFA_F_TEMPORARY == IFA_F_SECONDARY == 0x01,
and 0x01 in interface flags is IFF_UP, ALL addresses appeared as 'temporary',
making the permanent>temporary priority non-functional.

New netlink-based get_interface_ipv6_addr_nl() reads real IFA_FLAGS from
RTM_GETADDR response, correctly identifying:
- Temporary:     IFA_F_TEMPORARY=1, IFA_F_MANAGETEMPADDR=0
- Stable/mngtmp: IFA_F_TEMPORARY=1, IFA_F_MANAGETEMPADDR=1
- Legacy static: IFA_F_TEMPORARY=0

Verified: wlo1 permanent (3a7a:eff:fef5:9637) now correctly selected
for prefer_stable=1, temporary (1fb5:b7c:7538:dcf4) for prefer_stable=0.
2026-08-02 11:13:47 +03:00
evgeny f67bc23cd3 ipv6: fix sin6_scope_id and socket classification for link-local addresses
- CM_V6_ANY for :: socket (matches all v6 addr classes)
- sin6_scope_id from s->netif_index (not from local_addr where :: has scope=0)
- getnameinfo in stcp_link_connect to preserve scope_id in addr string
- scope_id in cm_invite_tcp_connect, ncd_create_links, connect_create_links_v6, cm_bg_ping_to_node
2026-08-02 10:45:18 +03:00
evgeny 50156e12af android: member list screen + link status dots, ipv6 autoconnect fixes
Android chatgui:
- New MemberListScreen with member list, detail card, edit dialog
- Link status dots (green/red/grey) per address in detail card (1s poll)
- Self-node name coloured green
- ChatScreen: people icon in TopAppBar

IPv6 autoconnect fixes (3 bugs in conn_mgr/topo):
- topo_node_sqlite_node_load: load v6 addresses from DB
- topo_node_registry_store: merge v6_addrs into existing node
- cm_sock_v6_classify: wildcard [::] treated as CM_V6_DIR
- cm_add_v6_link: htons(port) was missing

Memory fixes:
- strdup→u_strdup in bridge (7 instances)
- u_realloc cap update before success check (3 instances)
- char sql[256]→[512] for truncated SELECT

Chat member hash now includes addresses for change detection

Linux chatgui:
- Link status dots in AccountList detail panel (1s poll)
- Self-node name in green in member list
2026-08-01 23:40:58 +03:00
evgeny 6719a1e23c IPv6: STCP dual-stack + Android config + invite fixes
- stcp_server: dual-stack (AF_INET6 + IPV6_V6ONLY=0) instead of IPv4-only
- stcp_client: getaddrinfo(AF_UNSPEC) instead of inet_pton(AF_INET)
- stcp_link: detect address family from remote_addr->ss_family
- platform_compat.h: add <netdb.h> for getaddrinfo on POSIX
- ConfigProvider.kt: add [server:main6] addr=[::]:PORT for Android
- invite_link_c.c: invite_serialize_addrs handles IPv6 (was IPv4-only)
2026-08-01 20:19:00 +03:00
evgeny f5e18891bc chat_sync: invite — process IPv6 addresses from invite link
cm_invite_trampoline() previously skipped all IPv6 addresses from
invite links (if (family != 4) continue). Now:
- DB save: save IPv6 addresses to node_addresses with family=6
- TOPO_NODE: create TOPO_ADDR6 entries, populate ni->v6_addrs
- ncd_create_links already handles IPv6 sockets correctly
- Condition check includes v6_addrs alongside v4_addrs
- Cleanup frees v6_addrs chain
2026-08-01 20:03:21 +03:00
evgeny af3e2ee869 chatgui: IPv4+IPv6 dual-stack listen by default
- Default config creates [server:lan] (0.0.0.0) + [server:lan6] ([::])
- Network settings save also writes [server:lan6] section
- Sets IPV6_V6ONLY=1 on IPv6 sockets to avoid conflict with IPv4
- interface_addr for [::] bind prefers permanent IPv6 over temporary
2026-08-01 19:52:28 +03:00
evgeny 7799d54c0c conn_mgr: обновить API на единую точку входа conn_mgr_open_invite, поправить порядок destroy, обновить всех потребителей
- conn_mgr_cb_t: 5 параметров (h, node_id, group_id, event, arg)
- conn_mgr_open_invite: единая функция (ni=NULL — обычное, ni!=NULL — invite)
- CONN_EVENT_JOIN для успешного invite
- struct CONN_MGR_HANDLE: добавлен group_id
- bind/unbind в conn_mgr_init/destroy (один на инстанс, перезапись безвредна)
- utun_instance_destroy: conn_mgr_destroy до закрытия сокетов
- cm_ncd_callback: занулять ncd_handle при DOWN, защита от вызова после destroy
- topo_group_connect: свой трекинг handle'ов, conn_mgr_close для отмены
- обновлены: topo_recovery, media_download, media_delivery, chat_sync, тесты
2026-08-01 17:52:56 +03:00
evgeny 625382de34 node_conn_direct: add specific_sock param, move registry to per-instance; expose sc_hex_to_binary; add CONFIG_CONN_HANDLE 2026-08-01 12:34:53 +03:00
evgeny 8237d92be9 fix: NCD close UAF + conn name + init_connections per-link ni_tmp
- ncd_down_cb: defer callback removal+close via uasync_call_soon to avoid
  UAF in etcp_cbk_fire iteration (removing ncd_up_cb corrupts saved next ptr)
- ncd_recv_control_handler: use deferred close for CLOSE handlers instead of
  direct etcp_connection_close to avoid nested etcp_cbk_fire UAF
- node_conn_direct_open/open_node: use ni->node_name if available for
  conn name, fallback to 'n_<node_id>' instead of NULL
- init_connections: build ni_tmp in link loop with only current link's addr,
  preventing NCD from creating links for ALL addrs on first socket
- utun_instance_reload: same per-link ni_tmp fix
- init_connections/reload: set ni_tmp.node_name = client->name for
  test find_client_link by name
2026-08-01 12:30:31 +03:00
evgeny 4ba4ab038c cleanup: remove dead nd_handle from CONN_MGR_HANDLE, fix double semicolons in node_conn_direct, protect close from NULL conn 2026-08-01 09:48:16 +03:00
evgeny eb2a8f6975 node_conn_direct: add open_node variant, refactor internals 2026-08-01 09:45:32 +03:00
evgeny 153483fd96 media_delivery: обновление и новый тест conn_mgr_handles 2026-08-01 08:40:18 +03:00
evgeny d310bccf08 fix: memory pool alloc mismatch in tests, db_sync SEND_DATA buffer overflow
- test_etcp_router_reconnect, test_etcp_connect, test_stcp_traffic:
  replace u_calloc with memory_pool_alloc for TOPO_SOCKMETA4/TOPO_ADDR4
- db_sync.c: fix req[14] -> req[15] — SEND_DATA request header is 15 bytes
  (type:1 + from:4 + count:2 + vp:4 + want_from:4)
2026-07-31 21:44:44 +03:00
evgeny fe9be74e29 topo refactoring (WIP): topo_node.h + AGENTS.md safety rules 2026-07-31 16:01:47 +03:00
evgeny 51d6c782ac voice playback: miniaudio cursor ground truth + EMA smooth decay
- SoundManager: add currentPcmCursor()/isCurrentPcmPlaying() via ma_sound_get_cursor_in_pcm_frames / ma_sound_is_playing
- messagedelegate: replace wall-clock position tracking with real miniaudio cursor, remove 200ms hardcoded delay
- messagedelegate: end-of-playback detection via isCurrentPcmPlaying() instead of wall-clock comparison (was killing audio ~200ms early)
- messagedelegate: EMA smoothing of displayFrac/displayElapsed (0.15 factor, 16ms timer)
- messagedelegate: timer continues after audio stops to animate EMA decay to 0 (not mouse-move dependent)
- media_delivery: download progress callback, Ed25519 sig verify, author fallback, content hash check
- conn_mgr: already-connected returns OK with callback instead of error
- chatgui: voice download-pending auto-play, file download progress bar, media bar colors
2026-07-31 00:17:03 +03:00
evgeny 641c9f574f db_sync: cascade insert + SYNC_DONE fixes + done_cbk + test stabilization
- db_record_insert_cascade: unified insert with on_insert + PUSH cascade
- SYNC_DONE moved from INIT_RESP to DATA handler (master sends after slave response)
- db_sync_add/remove_done_cbk: global sync-completion subscription (etcp pattern)
- REQUEST_SYNC handler: sync_state != 1 instead of == 0
- SYNC_DONE mc==pc hash mismatch: accept as converged
- Removed unconditional sync_state=2 from DATA handler
- test: done_cbk for sync phases, count-only for PUSH phases, removed dead B<>C reinitiate
2026-07-30 20:23:47 +03:00
evgeny c093630d4d db_sync: async batch chain_hash recalc (batch 50, timeout 0→100tb), flush before protocol chain_hash reads, remove db_cascade_from/range + do_cascade param 2026-07-30 15:53:23 +03:00
evgeny df13b16fad docs: AGENTS.md chat module descriptions, improved ETCP debug logging, android AGENTS update 2026-07-30 12:20:04 +03:00
evgeny 6f502243db fix: etcp_conn_reset resume output_queue+ack_q+normalizer; collision client-timer cancel; ETCP full state dump to debug log on sync TIMEOUT 2026-07-30 09:17:28 +03:00
evgeny a3b00984ed fix: stable ETCP reconnect (no REINIT loop), own voice playback, peersOnline via conn state, UDP log diagnostics 2026-07-30 01:21:40 +03:00
evgeny 70797f921e feature: media auto-download + manual download + UI download state indication (Android + desktop)
C-layer:
- chat_event.h: CHAT_EVT_ATTACHMENT_DOWNLOADED (14)
- chat_msg.c: fixed md_auto_download (non-NULL done_cb, correct media_base from db_path, use chat_setting for autoload checks)
- chat_msg.c: added chat_core_attachment_download + trampoline for manual download
- chat_core.h: exported attachment_dl_req + trampoline

Android (Kotlin + JNI):
- Message: downloadState field (parse from localAttrs)
- ChatRepository: parse localAttrs for download state
- NativeLib: attachmentDownload() JNI
- MessageBubble: FileBubble with download status icon + tap-to-download
- ChatViewModel: downloadAttachment() + CHAT_EVT_ATTACHMENT_DOWNLOADED handler
- ChatScreen: wire onDownloadFile handler
- android_jni_bridge: utun_bridge_attachment_download + JNI + localAttrs in JSON

Desktop (Qt/C++):
- messagedelegate.h: MsgFileDownloadStateRole, MsgChannelIdRole
- messagelist.cpp: parse localAttrs for file download state
- messagedelegate.cpp: FileBubble with download icon/text + click handler via chat_core_attachment_download_trampoline
- gui_bridge.h: GUI_EVT_ATTACHMENT_DOWNLOADED (14) + callback typedef
- gui_bridge_impl.cpp: event dispatch + setter
- mainwindow.cpp: callback registration -> refresh message list
2026-07-30 00:01:38 +03:00
evgeny b8920cf483 chat_settings: [chat] config section + live sync + fix test segfault
- New chat_setting.c/h: unified chat settings API (int/str getters)
- Config parser supports [chat] section, calls chat_setting_set per key
- chat_core exposes chat_core_set_setting / trampoline for GUI bridge
- chatgui: AudioDevice + Storage pages persist to [chat] config + live apply
- chatgui-android: ConfigProvider, NativeLib, headless handle chat settings
- MessageList: 200 msg limit, fixed saveChannelState via trampoline
- test_media_delivery_full: fix segfault at shutdown (mon() timer vs freed conns)
2026-07-29 21:18:49 +03:00
evgeny efbb495016 media_delivery: fix SUPER_REPL spam + streaming verification
Bugs fixed:
- md_super_peer_add: timeout_tb = 0 → MEDIA_REPL_TIMEOUT_TB (timer fired every poll)
- md_repl_retrans_cb: md from peer->connect_timer (garbage) → peer->md
- md_repl_retrans_cb: removed inflight_count=0 reset (don't resend acked entries)
- stream_ctx: removed unused block_data field (was NULL, caused crash in BLOCK_DONE)

Test improvements:
- stream_completed flag for reliable streaming verification
- File created in <db_dir>/media/ (matches media_index path logic)
- 9/9 full test, 33/33 total
2026-07-29 20:42:47 +03:00
evgeny b4824de3b8 media_delivery: fix streaming crash, correct file path via media dir
- Fix NULL block_data deref in stream_send_chunk_cb
- Test B1: create file in <db_dir>/media/, commit to media_files
- Store media_base in ui_state for streaming handler path resolution
- 8/9 tests pass (streaming works but timing issue with SUPER_REPL flood)
2026-07-29 20:16:47 +03:00
evgeny a6321e15b3 media_delivery: streaming, fallback, admission control, 8/9 tests
- md_handle_block_req: real chunk streaming via etcp_router_on_send_ready
- stream_ctx state machine: backpressure waiter → send chunk → repeat
- content_hash + block_sig on stream completion
- Fallback to author when supernode has no block info
- media_download: author_node_id for peer fallback
- test_media_delivery_full: 8/9 pass (SUPER_HELLO, HAVE_BLOCK, replication, admission, file create)
- Known issue: streaming test hung (BGP route for n1→n2 not established in time)
2026-07-29 19:52:13 +03:00
evgeny 57e6ffed16 media_delivery: admission control + OVERLOADED + full integration test
- MEDIA_BLOCK_OVERLOADED (0x0F) subcommand + packed struct
- Admission control: track active_streams, reject > 3 with OVERLOADED
- media_delivery_stream_done() to decrement counter after stream finish
- Auto-download trigger in chat_msg.c (parse media metadata, check settings)
- test_media_delivery_full.c: 9/9 tests on 4 instances
  (SUPER_HELLO, HAVE_BLOCK, replication, admission control, OVERLOADED)
- All 33 existing tests still pass (15+5+4+9)
2026-07-29 19:29:56 +03:00
evgeny 5e739176fa media_delivery: fix etcp_route_send svc_id + integration test
Root cause: etcp_route_send reads dgram[0] as svc_id, not subcmd.
router_deliver prepends svc_id byte on receive.

Fixes:
- md_send/md_dl_send/msend: prepend ETCP_RT_ID_MEDIA_DELIVERY to dgram[0]
- md_etcp_recv: read subcmd from data[1] (was data[0])
- etcp_api.h: document dgram[0]=svc_id convention
- media_delivery_set_supernode(): public API for dynamic switching

Tests:
- test_media_delivery_integration: 4/4 (SERVE_REG, HAVE_BLOCK, supernode stop/start)
- All 24 existing tests still pass
2026-07-29 18:31:18 +03:00
evgeny a863da7737 media_delivery: add tests + fix queue_new NULL ua and index_offset bugs
- test_media_delivery_sql: 15 tests (SQL tables, CRUD, packed structs)
- test_media_delivery_download: 5 tests (chunk write/append, sig verify, assembly, cancel)
- Fix: queue_new(NULL) returns NULL — pass inst->ua everywhere
- Fix: queue index offset must be offsetof(struct, key_field) not 0
- Move struct media_download to header for test visibility
2026-07-29 17:53:46 +03:00
evgeny fb3e34080d media_delivery: implement full media delivery subsystem (stages 1-8)
- topo_group: add BGP node event callbacks (NEW/UPDATE/REMOVE)
- member_sync: add multi-subscriber node_props_changed callback (adm_tags)
- media_delivery: SQLite tables (block_availability + super_sync), full protocol handler
- media_delivery_proto.h: packed protocol structures (15 subcommands)
- media_download: download management, chunk streaming, block assembly, HAVE_BLOCK with retry
- Makefile.am: add media_download.c to builds
2026-07-29 17:14:32 +03:00
evgeny 7d9cbad8c5 media_delivery: add architecture description and implementation plan 2026-07-29 16:53:30 +03:00
evgeny 65dbfa6813 feat: media_async + media_index modules, media_files table, tests
- media_async: thread-per-task async engine with primitives (sha256, sign, copy, uuid, file_size)
- media_index: media_files DB table with register_async for indexing chunks
- chat_msg: async media submission via register_async, simplified chat_msg_submit fields
- voice_recorder/attachment_sender: write single file (no chunk splitting on caller side)
- inputbar/messagelist: updated to new simplified submit interface
- test_media_async: 4-thread stress test (200K ops / 2 sec, 0 errors)
- test_media_index: 9 cases covering commit edge cases + async flow
- build: Makefile.am, CMakeLists.txt, utun_sources.cmake updated
2026-07-29 14:15:10 +03:00
evgeny ce5a84da24 fix: uasync epoll event ordering + test_uasync_socket_race fix 2026-07-29 11:10:55 +03:00
evgeny b7988f0ef8 refactor: single source of truth — PlaybackState in ViewModel
VoiceBubble is now a pure renderer — no local state, no LaunchedEffect, no timer.
All state from ViewModel.playbackState StateFlow:
  - playing/paused/elapsedMs/progress/filePath — one atomic snapshot
  - isActive = playbackState.filePath == message.filePath — only active clip shows cursor
  - onProgress callback from AudioRecorderManager feeds real elapsed into ViewModel
  - ViewModel timer (80ms) updates progress between onProgress callbacks

Removed: VoiceBubble's playing/paused/seekVersion/PlayState/onDone/onPaused.
Removed: AudioRecorderManager's isPaused/elapsedMs/pausedElapsedMs.
2026-07-29 02:29:53 +03:00
evgeny 9742772880 fix: monotonic playGeneration instead of re-settable stopped flag
stopped flag was set before join() but cleared after, causing race:
if thread was still blocked in decodeRead when stopped was reset,
onComplete fired anyway. Monotonic playGeneration ++ never decreases,
thread captures myGen at start, compares after completion —
correctly skipped if generation changed (another stop/seek started).
2026-07-29 02:20:25 +03:00
evgeny e4af56856c fix: atomic PlayState replaces non-atomic seekVersion+elapsed+progress
PlayState data class bundles playId, elapsedMs, progress into one atomic snapshot.
LaunchedEffect keyed on playId — when it changes (seek/resume), the timer reads
elapsedMs and progress from the SAME snapshot, eliminating the race between
seekVersion++ and elapsedMs=seekMs.
2026-07-29 02:13:47 +03:00
evgeny 1481d1244b fix: seekVersion forces LaunchedEffect restart on seek/resume
When playing=true and seek happens, LaunchedEffect(playing) key didn't change,
so the elapsed timer kept using the old start time. Added seekVersion counter
as second key — increments on waveform tap and play/resume click, forcing
the timer coroutine to restart with the new elapsedMs position.
2026-07-29 02:10:51 +03:00
evgeny 0afc74e567 fix: stop/pause no longer fires onComplete from killed thread
When stopPlayback/pausePlayback kills the playback thread, the thread's
onComplete callback was still firing, resetting UI state (playing=false,
elapsed=0). Added @Volatile 'stopped' flag — onComplete only fires on
natural completion, not on external stop/pause.
2026-07-29 02:08:51 +03:00
evgeny 2edd9bcc06 cleanup: remove dead currentPlayPath, fix seekVoice _currentPlayingPath reset
- AudioRecorderManager: removed unused currentPlayPath (duplicate of ViewModel state)
- ChatViewModel: seekVoice now resets _currentPlayingPath on complete (prevents
  pause-on-already-ended after seek finishes)
2026-07-29 02:03:09 +03:00
evgeny 2e8aa89397 voice: fix waveform — pointerInput on Box, cursor + white bars stay on pause
- Canvas pointerInput moved to wrapping Box (state updates trigger recomposition)
- showCursor = playing || paused (cursor and white bars visible on pause)
- elapsed/dur stays visible on pause too
2026-07-29 02:00:43 +03:00
evgeny 6980f34df9 voice: real pause (saves position), waveform tap seek, resume from position
- AudioRecorderManager: playFromPosition(startMs), pausePlayback, elapsedMs tracking
  playVoiceFile now delegates to playFromPosition. fade skipped PCM frames on seek.
- ViewModel: playVoiceMessage with pause toggle, seekVoice method
- VoiceBubble: click tap on waveform → seek, pause state (⏸ keeps cursor visible)
  Webform canvas with detectTapGestures for seek input
2026-07-29 01:57:31 +03:00
evgeny 887bbdbf84 voice: play/pause toggle, stop previous on new track
- ViewModel: _currentPlayingPath tracks active track. Tap same → stop.
  Tap different → stop old, play new. onComplete clears currentPlayingPath.
- AudioRecorderManager: stopPlayback() already called at start (existing)
- VoiceBubble: simplified click, always sends to ViewModel for state management
2026-07-29 01:51:25 +03:00
evgeny b8e15ad1f2 voice: played bars white, yellow cursor line on waveform 2026-07-29 01:48:16 +03:00
evgeny ee8167793c voice: waveform idle bars lighter (#CCD4 80% — near white) 2026-07-29 01:45:03 +03:00
evgeny 109326717e voice: play button white circle, black icon 2026-07-29 01:44:26 +03:00
evgeny f745fb25e8 android: lighten waveform bars (idle: #A0A0B0 70%, playing unplayed: 35% alpha) 2026-07-29 01:43:55 +03:00
evgeny 4ffa59d968 unify voice waveform format: base64 wf={100chars};dur={ms}; for both desktop and Android
- inputbar.cpp: CSV waveform → base64 log-scale (same as voice_recorder.c)
- messagelist.cpp: sendVoiceMessage removes displayName|dur_sec| prefix, uses wfStr directly
  setVoiceMessageRoles: decodes new format wf={base64};dur={ms};|... from data field
- messagedelegate.cpp: waveform values now pre-normalized 0..1, no log recalculation needed
- VoiceEncoder: no changes needed (was already compatible)

Both platforms now store identical format in chat_msg_submit.data:
  wf=ABCDef...100chars;dur=NNNN;|file_size|block_size|num_blocks|sigs_hex
2026-07-29 01:43:11 +03:00
evgeny b65a091b00 voice: base64 compact waveform (100 bytes), progress bar, play/pause auto-reset
- voice_recorder: 100-char base64 (A-Za-z0-9+/) log-scale waveform, ~120 bytes total vs ~600 CSV
- ChatRepository: parse base64 'wf=<100 chars>;dur=NNNN;' → normalized 0..1 List<Float>
  CSV fallback for old messages (length < 50 or contains comma)
- VoiceBubble: progress bar (blue played / gray unplayed), ▶/⏸ toggle, elapsed timer
- onComplete callback wired through ViewModel → AudioRecorderManager → VoiceBubble auto-reset
2026-07-29 01:36:43 +03:00
evgeny 3c75c8f687 icons: replace emoji text with PNG resources + add media_dir debug logs 2026-07-29 01:27:23 +03:00
evgeny 491fa84db8 voice: fix waveform display + text buffer, 75% width, play/pause state
- bridge: txt buffer 256→2048, limit 250→2000 (voice d-field can be ~900 bytes)
- VoiceBubble: Column layout — play+waveform on top, duration|timestamp on bottom
- Play/pause toggle with ⏸/▶ icons
- Voice messages use 75% screen width max (300dp for others)
2026-07-29 01:21:35 +03:00
evgeny 94a537ebdd voice: waveform recording + display, playback wiring
- voice_recorder: waveform_levels[100] accumulated during feed, normalized to 100 values in stop()
  stored as 'wf=...;dur=...;' in chat_msg_submit data
- bridge: filePath in get_messages_json from local_attrs.fp, g_db_path moved global
- Message model: waveform: List<Float>, voiceDurationMs: Int, filePath: String
- VoiceBubble: Canvas waveform (100 bars, log scale), play button, duration
- ChatViewModel.playVoiceMessage → AudioRecorderManager.playVoiceFile
2026-07-29 01:15:57 +03:00
evgeny 39d6f19d97 fix: chat_msg media_dir path — strip filename from g_cc.db_path
g_cc.db_path is '/data/.../files/chats.db' (file path), but media blocks
are written to '/data/.../files/media/<channel>/'. chat_msg was concatenating
'chats.db/media/...' — now strips the database filename, using only the
directory portion for the media path.
2026-07-29 01:04:27 +03:00
evgeny b11e705f7a android: input bar — single right icon (PTT when empty, Send when text) 2026-07-29 00:58:26 +03:00
evgeny b90728be3c android: round send button with \u27A4 icon (matching desktop chatgui) 2026-07-29 00:55:39 +03:00
evgeny 9fca8d5e1f android: fix voice file creation (mkdir errno logging), PTT while(true)
- voice_recorder: proper mkdir with errno/EEXIST check, errno logged on fopen failure
- MessageBubble: pointerInput wrapped in while(true) so PTT works after first gesture
2026-07-29 00:53:23 +03:00
evgeny 1dac0b8c81 unify compressor: remove C++ duplicate audiocompressor, use lib/audio_compressor.c in chatgui 2026-07-29 00:52:05 +03:00
evgeny 00afc5c80a android: fix PTT — one Box across all states, release now works; 1s min duration
- PTT Box: same composable identity in idle/recording states (just changes bg color)
  Gesture coroutine survives recomposition → onPttReleased() fires on finger lift.
  When locked, Box is replaced by cancel/send IconButtons.
- Min voice duration: 500→1000ms (C and ViewModel)
2026-07-29 00:32:10 +03:00
evgeny 90de87b13d move audio_compressor from libutun_lite/ to lib/ — shared library
- audio_compressor.h/c moved to lib/ (alongside opus_codec, mem, debug_config)
- lib/Makefile.am: added to libuasync_a_SOURCES
- utun_sources.cmake: removed from _cfg_src (GLOB from lib/ picks it up)
- tests/Makefile.am: test_audio_compressor added to check_PROGRAMS
- test includes simplified (debug_config.h, mem.h — lib/ in include path)
2026-07-29 00:26:45 +03:00
evgeny d22eac55f7 fix audio_compressor: flat array instead of ring buffer, +unit test (411 asserts)
Ring buffer (pending_head/pending_tail/cap) replaced with simple flat array (pending_count/cap).
u_realloc copies linearly but ring buffer needs memmove for wrap-around — mismatch caused
SIGSEGV write to 0x10 on ARM64. Flat array: append to end, process from [0], memmove down.
Performance unchanged (max ~15 blocks).

Tests: create/destroy, passthrough disabled, basic push+flush, silence, reset, stress (200 cycles)
2026-07-29 00:11:29 +03:00
evgeny fc2adcf81d android: fix voice recording — init g_rec, move includes, check startRecording result
- voice_recorder_init(db_path) in utun_bridge_init (was never called, g_rec stayed NULL)
- voice_recorder_deinit() in utun_bridge_destroy
- ViewModel: set _isRecording only after startRecording() succeeds
2026-07-28 23:52:33 +03:00
evgeny 50214d50c1 android: fix PTT button — Box+pointerInput(hasMicPerm) instead of IconButton+onClick conflict 2026-07-28 23:35:53 +03:00
evgeny 1f8f41bda7 android: tabbed settings (Settings/Audio) with mic test, codec preset, compressor sliders
- voice_recorder: compressor config API (max_gain, lookback, lookahead, rise_rate, target_level)
- bridge/JNI: compressor_config export
- AudioRecorderManager: startMicTest/stopMicTest for level preview
- SettingsScreen: TabRow with Settings + Audio tabs
- Audio tab: mic test with live level bar, Opus preset selector, 5 compressor sliders matching desktop
2026-07-28 23:33:19 +03:00
evgeny 9b6dd6476c android: PTT swipe-to-lock, recording overlay with level bar, cancel/send buttons
- voice_recorder: peak_level tracking in feed(), voice_recorder_get_peak_level()
- bridge/JNI: voice_get_peak_level API
- ViewModel: isRecordingLocked, recordingDurationMs with 80ms timer, lock/send/cancel
- InputBar: 3 states (idle/holding/locked), custom pointerInput gesture with swipe-up lock
- ChatScreen: poll peak level 12x/sec during recording
2026-07-28 23:28:19 +03:00
evgeny 0f1e6e0006 android: fix NDK build — add Opus sources, fix Kotlin composable context, fix bridge db_path param 2026-07-28 23:20:19 +03:00
evgeny 5b6979da27 android: voice messages recording + playback, file attachments
- C99 port of AudioCompressor (libutun_lite/audio_compressor.h/c)
- Voice recorder: PCM buffer + compressor + Opus encode + block splitting + uasync send
- Attachment sender: file splitting into blocks + uasync send
- Bridge API + JNI: voice start/feed/stop/cancel, attachment send, Opus decode for playback
- Kotlin: AudioRecorderManager (AudioRecord + JNI feed), AudioPlayer (AudioTrack + JNI decode)
- UI: PTT button with press-to-talk, attach button with file picker, recording overlay
- Message display: voice messages (play button), file attachments (icon + name)
- Settings: Opus preset selector (Low/Standard/High), compressor toggle
- AndroidManifest: RECORD_AUDIO, storage permissions
2026-07-28 23:15:59 +03:00
evgeny e751e6ee69 android: voice messages + attachments architecture plan, build fixes 2026-07-28 23:07:23 +03:00
evgeny f8d7c21a79 Android: unify DB access via JNI/JSON, fix instance lifecycle, add restart + health check
- ChatRepository: remove SQLiteDatabase (READONLY 2nd handle), use JNI→shared g_cc.db (FULLMUTEX, like desktop)
- instance_lite: simplify start/stop lifecycle — start no longer waits for init, stop waits for thread exit with timeout, add internal restart (no thread kill)
- Restart trampoline: reinit UDP log target on config change
- Health check: ping/uasync_post round-trip every 3s, red dot on stall
- Settings: Restart instance button
- Fix: g_inst check instead of g_running for is_running()
- Fix: SharedFlow replay=1 + proactive DB_READY check in ViewModel
2026-07-28 21:13:11 +03:00
evgeny 03b34e9580 android: fix QR join flow + cleanup log levels + remove log artifacts
- MainActivity: call NativeLib.joinChannel directly (bypass ViewModel null repo)
- LogManager: add Log.d for Kotlin logs visibility in logcat
- ChatViewModel/NativeLib: add diagnostic logs for join path
- chat_sync: JNI-safe logs via __android_log_print, add error on !g_cs
- etcp_debug: DUMP INFO→DEBUG (remove 99.8% packet spam)
- secure_channel/etcp_connections/topo_node: keepalive PING/PONG spam INFO→DEBUG
- .gitignore: logreceiver output + build artifacts
- Remove tracked log files and build dir from repo
2026-07-28 18:11:11 +03:00
evgeny 0216b0bf50 android: utun shutdown diagnostics + non-blocking stop + logreceiver doc
- instance_lite: cleanup moved to uasync thread (self-detach), events on start/stop
- udp_log: cancel_timer before uasync_destroy, flush on stop, stopping flag
- jni_bridge: fixed UDP stop order (cancel_timer → destroy → flush)
- instance_lite: u_report_unfreed_blocks at all shutdown paths
- chat_event: CHAT_EVT_SERVICE_STARTED/STOPPED for GUI notifications
- AGENTS: logreceiver section
2026-07-28 17:06:09 +03:00
evgeny 064bed06f2 android: fix key regeneration, Clear DB, uasync eventfd, udp log from config
- jni_on_event: fix JNI_EDETACHED — AttachCurrentThread/DetachCurrentThread
- u_async.c: pipe→eventfd for reliable cross-thread wakeup
  add process_posted_tasks in poll() path, fix EINTR (saved_errno)
- bridge_debug_hook: keep udp_log_write, remove mutex from udp_log
- memory_pool.c: alloc/free logs INFO→DEBUG (reduce spam)
- instance_lite.c: chat_event_set_handler before ensure_keys,
  KEYS_GENERATED events, fchmod 0600 on keys.conf,
  UDP log from config [log_udp] section, regenerate_trampoline
- config_parser: parse [log_udp] ip/port
- NativeLib.kt: remove callbacksSet, simplify init/stop/destroy
- ChatViewModel.kt: unified event callback in init, pubKey StateFlow,
  requestRegenerateKeys, clearState fix
- SettingsScreen.kt: pubKey.collectAsState(), Clear DB via restart()
- HeadlessService.kt: remove setUdpLogTarget (now from config)
- android_udp_log.c: simplified — no mutex, buffer+send+timer in one thread
- utun_instance.c: NULL-check tcp_connections before destroy
- chat_event.h: +CHAT_EVT_KEYS_GENERATED 11, -NODE_CHANGED
- logreceiver: UDP log receiver tool
2026-07-28 16:29:33 +03:00
evgeny d0cca27edd invite: add proto field (UDP/TCP/BOTH) to invite link binary format
Format per v4 addr: family(1) + socketId(1) + proto(1) + addr(4) + port(2) = 9 bytes (was 8)
Proto values: 0x01=UDP, 0x02=TCP, 0x03=BOTH
Affects: invite_link_c.h/c (Android), invite_link.h/cpp (desktop), chat_sync.c, conn_mgr.c, joindialog.cpp, utun_node.cpp
2026-07-28 00:34:07 +03:00
evgeny 5276c1288c android: real utun instance + SQLite channels + reactive logs + QR join
- instance_lite.h/c: full uTun stack from INI config text
  (keygen, uasync, UTUN_INSTANCE, SQLite, chat_core/sync, BGP, etcp_router)
- jni_bridge: nativeStart/Stop, nativeJoinChannel → chat_sync_connect_from_invite,
  nativeSubscribeLogs, C logs → logcat + Kotlin LogManager
- Kotlin: direct read-only SQLite queries (channels, messages),
  ViewModel with live event callbacks, reactive LogScreen (tail -f),
  CameraX QR scanner with proper lifecycle, JoinChannelDialog
- HeadlessService: generates INI config from Kotlin settings → C instance
2026-07-28 00:01:57 +03:00
Evgeny d1f6f6fca9 fix WELCOME deserialization: skip atl+adm_tags_sig fields before address data 2026-07-27 23:48:23 +03:00
evgeny 9e58160480 android: initial chatgui-android project with QR join
- libutun_lite: C99 core, invite_link_c.h/c (compatible with desktop)
- jni_bridge: android_jni_bridge.c/h with logcat logging
- app: Jetpack Compose UI, CameraX + ML Kit QR scanner
- JoinChannelDialog, QrScanScreen with proper camera lifecycle
- headless: control socket with join command
- OpenSSL prebuilt libs for arm64-v8a + x86_64
2026-07-27 22:50:29 +03:00
Evgeny a47ef86810 uasync: fix stall detection — record timestamp after poll wait, not before 2026-07-27 19:42:35 +03:00
Evgeny 48bfe644c4 uasync: add event loop stall detection (warn if >10ms between poll entries) 2026-07-27 19:35:13 +03:00
Evgeny 2d53b3120f ffmpeg: add build script and CMake integration 2026-07-27 17:59:07 +03:00
Evgeny d497a8dc92 media_blocks: fix missing QJsonArray include 2026-07-27 17:16:02 +03:00
Evgeny d4a396f888 media: Ed25519-подписи блоков, local_attrs, MediaBlockReader, файловые вложения 2026-07-27 17:09:31 +03:00
Evgeny 6b765de5cc feat: voice playback delay compensation + waveform click-to-seek 2026-07-27 14:53:53 +03:00
Evgeny da953c6c7a fix: mic test level bar dynamic color too 2026-07-27 14:25:47 +03:00
Evgeny f08b63466c исправлены include-пути в chatgui GUI-файлах и utun_node.cpp после переноса chat в src/chat/ 2026-07-27 13:35:40 +03:00
Evgeny ed6132fcda перенос chat-логики из chatgui/transport в src/chat/
- Создан src/chat/ с chat_event.h/c — система нотификаций
- Перенесены chat_core, chat_msg, chat_channel, chat_profile, chat_status
- Перенесены chat_sync, member_sync, merkle_sync, db_sync
- gui_bridge_post() заменён на chat_event_post() с возможностью регистрации обработчика
- chat_event_set_handler() — chatgui регистрирует мост в gui_bridge
- Без обработчика: события логируются через DEBUG_DEBUG + log_dump
- chat интегрирован в utun_instance.c (при db_path в конфиге)
- Обновлены include-пути, Makefile.am, CMakeLists.txt
- Из chatgui удалены старые transport/ файлы, обновлён utun_node.cpp
2026-07-27 12:58:33 +03:00
Evgeny 4595052b5b fix: level bar dynamic color — red only above 85% 2026-07-27 12:20:06 +03:00
Evgeny d5b97ff104 fix: immediate compressor apply + dead code removal + qDebug→DEBUG_* conversion 2026-07-27 12:11:50 +03:00
Evgeny 5ec1cb78be fix: use DEBUG_* macros instead of qDebug in AudioRecorder/AudioCompressor 2026-07-27 11:56:49 +03:00
Evgeny f1b599f0c0 feat: voice compressor with RMS+peak leveling, sliding window envelope, rate-limited gain 2026-07-27 11:24:47 +03:00
Evgeny 7004b110ae feat: mic level bar in recording overlay
- InputBar: QProgressBar with gradient (green→yellow→red) in recording overlay
- updateRecordingUI: computes dB from peakLevel, updates bar every 100ms
- showRecordOverlay: resets bar to 0 on new recording
2026-07-27 09:35:16 +03:00
Evgeny 5593bfdc0c fix: logarithmic waveform scale + visible idle color + duration display
- Waveform bars: linear→logarithmic (dB mapping -60..0dB)
- Colors: idle gray visible (130 alpha), played blue, unplayed dim blue
- Min bar height: 1→2px
- Duration: fallback to MsgVoiceDurationRole when not playing (was 0:00/0:00)
- Debug log: wfN, width, first value, dB, barH
2026-07-27 09:19:34 +03:00
Evgeny e050f3f1f9 fix: waveform repaint via viewport()->update() instead of activeWindow()
- Timer now calls s_voiceView->viewport()->update() for correct delegate repaint
- s_voiceView captured from option.widget in paint() voice section
- Previously activeWindow()->update() didn't reach QListView viewport
2026-07-27 09:13:33 +03:00
Evgeny 05c7b4d47e feat: waveform display + 3px progress bar in bubble + sliding window RMS
- AudioRecorder: sliding window RMS (avg 5 frames), waveformLevels() getter
- InputBar: downsample RMS to 100 samples, encode as wf0,wf1,... in textData
- MessageList: parse 3rd pipe-separated field as waveform, store in MsgVoiceWaveformRole
- MessageDelegate: new voice layout (16px play btn + 25px waveform + 3px progress bar)
- MessageDelegate: waveform bars colored by progress (blue before, gray after)
- Progress bar integrated at bottom of waveform area as colored fill
2026-07-27 09:07:14 +03:00
Evgeny 70a0a70b88 fix: crash — no delete of ma_sound/ma_audio_buffer during playback
- stopCurrentPcm(): only ma_sound_stop, no delete/uninit in runtime
- playRawPcm(): just stop old sound, create new objects (old ones leak until shutdown)
- shutdown(): cleanup accumulated ma_sound + ma_audio_buffer after engine uninit
- root cause: deleting audio objects from GUI thread while miniaudio worker reads them
2026-07-27 08:16:37 +03:00
Evgeny 8bd4a7218c fix: true pause/resume with position preservation
- SoundManager: pauseCurrentPcm()/resumeCurrentPcm() — stop/start without destroy
- VoicePlayState: added paused/pausedPosSec fields
- editorEvent: 3-state logic — playing→pause, paused→resume, stopped→new
- Different file click: stops old with full cleanup, plays new from start
- Timer: skips paused items, full cleanup on auto-complete
2026-07-27 08:07:19 +03:00
Evgeny 830f729f52 fix: pause support + old message skip + stopCurrentPcm
- SoundManager: added stopCurrentPcm() for pause/stop of PCM playback
- messagedelegate: call stopCurrentPcm on pause, new play, and timer complete
- messagelist: skip old double-nested JSON messages (raw starts with '{')
- sound_manager.h: void* for ma_audio_buffer to avoid header dependency
2026-07-27 00:38:15 +03:00
Evgeny 2687c3ed82 fix: play button not responding + debug logging to log file
- chatview: QListView::mousePressEvent(event) instead of event->accept()
  (Qt needs base class press to set pressedIndex for editorEvent to fire)
- messagedelegate: GUI_DEBUG replaces qDebug (writes to log file)
- voiceplayback: DEBUG_DEBUG/DEBUG_WARN + debug_config.h include
- voicemessageencoder: same debug migration
- inputbar: GUI_INFO replaces qDebug
- messagelist: DEBUG_DEBUG/DEBUG_WARN replaces qDebug/qWarning
2026-07-27 00:30:09 +03:00
Evgeny 01012675d0 fix: voice playback coordinate bug + debug logging + device name encoding
- messagedelegate: editorEvent now adjusts me->pos() by option.rect.topLeft()
- voiceplayback: enhanced debug logging for file open/decode/play
- messagelist: debug voice file path + exists check
- inputbar: debug mediaDir path on record
- sound_manager: debug in playRawPcm, decodeDeviceName() for mixed encodings
- voiceencoder: debug file write path
2026-07-27 00:22:00 +03:00
Evgeny 324d285145 fix: voice message display and playback — complete flow
- db_manager: getMessagesSince() now parses m.contentType from JSON (was missing)
- messagelist: setVoiceMessageRoles uses data directly as payload, not re-parsed as JSON
  (data is already the 'd' field value extracted by db_manager)
2026-07-27 00:13:58 +03:00
Evgeny fd3b125ecb fix: voice message JSON double-nesting, '{' display
- req->data now stores 'filename|duration' instead of full nested JSON
- chat_core_submit_message builds the JSON with d='filename|duration'
- setVoiceMessageRoles splits d field by '|' to extract filename and duration
2026-07-26 22:16:09 +03:00
Evgeny ca9d1384f2 fix: PCM copy, device name encoding, double shutdown
- playRawPcm: ma_audio_buffer_init_copy instead of ma_audio_buffer_ref (copies PCM)
- enumerateDevices: fromLocal8Bit instead of fromUtf8 for device names
- AudioRecorder: m_deviceStarted flag prevents double ma_device_uninit
- stopMicTest: delete instead of deleteLater to avoid deferred double shutdown
2026-07-26 22:01:46 +03:00
Evgeny a4ae1c6b41 fix: RMS mic level, 25ms timer, speaker test, overlay recording indicator
- AudioRecorder: peak replaced with RMS (sqrt mean of squares)
- AudioDeviceSettings: level meter timer 100ms -> 25ms
- SoundManager: playRawPcm uses ma_audio_buffer_ref instead of ma_decoder (fixes speaker test)
- InputBar: recording overlay positioned over text input, no layout shift
- messagelist: removed old recording indicator widget
2026-07-26 21:27:57 +03:00
Evgeny 850a260aac fix: voice message display, playback, recording UI, audio settings
- messagelist: parse content_type from JSON when DB column is empty
- messagelist: recording indicator (red dot + timer + cancel button)
- messagedelegate: editorEvent for play/pause toggle + progress bar
- messagedelegate: VoicePlayState tracking with QTimer
- InputBar: proper recordingContext (mediaDir + channelId)
- DbManager: dbPath() for media directory resolution
- AudioRecorder: peakLevel() for mic level meter
- VoiceEncoder: codec quality presets (Low/Standard/High)
- audiodevicesettingspage: Default device option first
- audiodevicesettingspage: mic level meter + test microphone
- audiodevicesettingspage: test speaker button
- audiodevicesettingspage: codec preset combo (Low/Standard/High)
- mainwindow: load opus_codec_preset from ui_state on startup
2026-07-26 21:19:29 +03:00
Evgeny 4fde0b0325 feat: integrate Opus codec into lib/libopus + voice messages in chatgui
- Add lib/libopus/ (opus v1.6.1 embedded source: CELT 18 .c, SILK 77 .c, SILK float 28 .c, OPUS core 14 .c)
- Add lib/opus_codec.h/.c (wrapper: encoder, decoder, raw PCM int16)
- Add tests/test_opus_codec.c (7 tests: silence, sine, compression, PLC, null, defaults, multi-frame)
- chatgui: AudioRecorder (miniaudio capture, 48kHz mono)
- chatgui: VoiceEncoder (PCM→Opus file with custom container)
- chatgui: VoicePlayer (Opus→PCM playback via SoundManager)
- chatgui: PTT button in InputBar, voice message send/receive in MessageList
- chatgui: AudioDeviceSettingsPage (device selection, persisted in ui_state)
- chatgui: MessageDelegate voice message rendering (play button + duration)
- Update autotools: configure.ac, Makefile.am, src/Makefile.am, tests/Makefile.am
- Update CMake: chatgui CMakeLists.txt, libutun CMakeLists.txt (link libopus_internal.a)
2026-07-26 20:34:38 +03:00
Evgeny 4a23593020 member_sync: add merkle_sync_broadcast relay to other SYNCED sessions 2026-07-26 18:44:45 +03:00
Evgeny ef1e064bdc merkle_sync: broadcast relay for star topology + all 6 test stages pass
- Replace active/synced with sess_state (SESS_SYNCING/SESS_SYNCED)
- Add merkle_sync_broadcast() API for relaying changes to SYNCED peers
- apply_items callback now receives from_peer parameter
- _handle_hashes: reset pending_count on session reuse (overwrite fix)
- _handle_batch: send broadcast to synced sessions on new items
- Test Stage 5: sequential B->A + C->A with relay (no 3rd sync needed)
- Test Stage 6: cancel + start_overwrite pass
- Fix _wait_for: remove i_phase global timeout dependency
2026-07-26 18:20:10 +03:00
Evgeny b9bde23d51 Stage 5 WIP: sequential syncs for 3-instance test, increased timeouts 2026-07-26 17:53:43 +03:00
Evgeny a1ecab997e merkle_sync: fix 4 protocol bugs + add test suite (Stages 1-4)
- g_merkle singleton → per-instance msync (utun_instance.h)
- level_prefix: off-by-one shift (63→64)
- _handle_hashes: differs |= for shared buckets with different hashes
- _handle_batch: sub_pl double-level-offset (next_lvl→lvl)
- pending[] array replaces pending_requests counter
- member_sync: mask_shift 63→64 alignment
- test_merkle_sync.c: prefix arithmetic, tree building, 2-instance integration, randomized 50-iter sync
2026-07-26 17:42:27 +03:00
Evgeny 0e11d858e8 chatgui: fix all animations showing the same TGS (rlottie model cache key collision)
lottie_animation_from_data uses 'key' as a cache key for parsed models.
All LottieIcon instances were created with key="", causing rlottie's
model cache to return the first loaded animation for all subsequent
requests. Only visible on Windows where rlottie cache is enabled by
default.

Fix: pass unique animPath as key to each LottieIcon constructor.
Also replaced fprintf(stderr) diag with GUI_INFO logging via
AnimTimer::logRegisteredIcons() in debug_ui.h.
2026-07-26 15:18:59 +03:00
Evgeny 49da899964 chatgui: channel sorting by name or last message time with preview
- Add last_msg_at column to channels table
- Update last_msg_at on every message insert (send/receive)
- Parameterize getChannels() with ChannelSortMode enum (Alphabetical/Recent)
- Add getChannelLastMessage() for preview text in channel list
- Add sorting combo to Profile settings page (stored in ui_state)
- Reload channels on message receive when sort is recent
2026-07-26 11:08:59 +03:00
Evgeny 332df381d1 chatgui: propagate local nick changes to message authors via MsgAuthorNodeIdRole + reloadLocalNicks 2026-07-26 10:11:31 +03:00
Evgeny 22db84816e chatgui: add local_nick TEXT to peers table DDL, remove ALTER TABLE hacks 2026-07-26 09:45:00 +03:00
Evgeny 86843693c9 chatgui: ensure local_nick column exists before SELECT in getChannelMembers 2026-07-26 09:43:39 +03:00
Evgeny df3fc04523 chatgui: RenameDialog — quick rename via context menu, separate mini dialog 2026-07-26 09:33:31 +03:00
Evgeny 85bdc00aaf chatgui: local nicknames per channel member — edit via MemberPropsDialog, override display everywhere 2026-07-26 09:19:31 +03:00
Evgeny 373a2e4687 chatgui: member detail panel redesign — compact format with relative time, colored online dot, 5s auto-refresh 2026-07-26 09:09:21 +03:00
Evgeny 11ab897fe5 chatgui: fix RTT display 10x overstatement (DB stores 0.1ms, displayed as ms); remove unused rtt_avg_100 2026-07-26 08:57:02 +03:00
Evgeny 12904a97e0 chatgui: AccountDelegate — palette-consistent colors (light/alternateBase instead of hardcoded) 2026-07-26 07:43:01 +03:00
Evgeny 77d89b841a chatgui: double-click member edit, fix delegate backgrounds; transport: invite_sync/member_sync robustness 2026-07-26 07:18:27 +03:00
Evgeny cd3c3ad07d merkle_sync: fix level 0 SQL range, rebuild trees at startup
- _get_level_hashes: use separate SQL without prefix64<=? for level 0.
  UINT64_MAX cast to sqlite3_int64 becomes -1, making prefix64<=-1 always
  false for positive prefixes → bitmap always 0 → sync never exchanges data.
- member_sync_init: add _rebuild_all_trees to scan all peers_* tables and
  call merkle_sync_recompute_path for each member. Ensures merkle_tree_hash
  has entries for members added before merkle sync existed.
2026-07-25 23:31:36 +03:00
Evgeny 6bf8482558 merkle_sync: fix BATCH parse, synthetic HASHES, initiator done_cb
- Add 2-byte dlen prefix to terminal BATCH/HASHES data so receiver knows
  exact byte count — fixes rem=0 bug that truncated multi-item BATCHes.
- Fix synthetic HASHES prefix_bytes: use merkle_sync_prefix_bytes(next_lvl)
  for both declaration and _prefix_write, not current level's pb_i.
- Fix initiator never getting _session_done: responder sends confirmation
  HASHES(is_data=1,level=0) before _session_done so initiator's done_cb fires.
- Add level==0 guard in _member_get_items/_member_update_bucket_hash
  to prevent SQL queries with bogus mask on virtual root level.
2026-07-25 23:11:53 +03:00
Evgeny be10714972 merkle_sync: fix prefix reconstruction, remove timeouts/retries
- _get_level_hashes: remove (0x1F << parent_shift) from range_end so
  bitmap only includes children of the actual parent prefix (not all
  parents). Add level 0 virtual root that queries all level-1 entries.
- merkle_sync_start: send HASHES at level 0 (virtual root) instead of
  level 1/prefix 0. Root-level REQUEST prefix reconstruction now uses
  the correct level-1 prefix from DB.
- _handle_batch: use merkle_sync_prefix_bytes(next_lvl) for synthetic
  child HASHES instead of current level's pb_i (could be wrong size).
- Remove timeout/retry mechanism: delete _session_timeout_cb,
  _session_start_timer, timer/retries fields. ETCP provides reliable
  transport — no need for application-layer retries.
2026-07-25 23:00:42 +03:00
Evgeny 332e3a61e5 fix merkle_sync: prefix serialization writes LSB bytes instead of MSB
_prefix_write was writing the least significant pb bytes of the 64-bit
prefix, but the protocol expects the most significant pb bytes (bits
63..64-pb*8). For pb<=4 these ranges are disjoint → wire always got
zeros. _prefix_read also lacked the final left-shift to align the
result at bit 63.

This caused all level>=2 merkle sync requests/batches to receive
prefix=0 instead of the real prefix, making get_items/update_bucket_hash
find zero members → handle_batch len=7 with no data → sync completed
without actually exchanging member data.
2026-07-25 21:53:06 +03:00
Evgeny ccc5396167 merkle_sync: fix _handle_batch — call _session_done after terminal buckets 2026-07-25 21:32:00 +03:00
Evgeny d512bacc81 merkle_sync: fix message format — type after ns (match chat_sync) 2026-07-25 20:56:05 +03:00
Evgeny ecb03ca290 conn_mgr: detect duplicate conn in handshake; etcp: loud collision errors in link/peer_id 2026-07-25 19:43:26 +03:00
Evgeny a4582202a8 merkle_sync: fix range_end formula — cover all parent+child bucket bits 2026-07-25 17:58:51 +03:00
Evgeny be5a760de9 merkle_sync: add debug logs to _get_level_hashes (SQL error + bitmap result) 2026-07-25 17:43:22 +03:00
Evgeny 5983acd988 merkle_sync: fix _get_level_hashes — range query instead of exact mismatch (prefix encoding bug) 2026-07-25 17:37:12 +03:00
Evgeny 7a9e3bea5f db_sync: mc>pc guard with last_peer_count — detect stalled sync when pc doesn't change 2026-07-25 16:56:30 +03:00
Evgeny 3490789249 db_sync: skip SYNC_DONE in count>0 handler when received==0 — breaks bad_sig recv→send loop on client side 2026-07-25 16:49:46 +03:00
Evgeny c9436219a0 db_sync: fix synced_pos overwrite in SYNC_DONE — use max to preserve total_processed advance 2026-07-25 16:28:09 +03:00
Evgeny 34404403ef db_sync: verify sig before send + fix total_processed double-count 2026-07-25 16:10:20 +03:00
Evgeny e9fce43be5 db_sync: fix bad_sig infinite loop — delete+rechain, advance synced_pos past dup/bad_sig, accept on covered tail 2026-07-25 15:58:18 +03:00
Evgeny 27906e9f91 db_sync: односторонний sync, sparse-таблица, vp в SEND_DATA, PUSH-релей 2026-07-25 15:00:21 +03:00
Evgeny fe18c454e2 chatgui: add diagnostic logging for Windows emoji animation bug investigation 2026-07-24 22:11:03 +03:00
Evgeny e071a9e2f8 db_sync: fix test sync protocol isolation (add auto_sync flag, destroy order, test refactor)
- utun_instance: fix destroy order (topo_groups before conn_mgr to avoid UAF in topo_recovery)
- topo_recovery: add NULL guard for instance/ua in try_next
- db_sync_instance_add: add auto_sync flag (1=send INIT_SYNC, 0=quiet)
  replaces separate _quiet variant, cleaner API
- db_sync_peer_set_state, db_sync_reinitiate: test-only peer control utilities
- test_db_sync: full rewrite — stable, isolated sync protocol testing
  all 3 branches (peer_empty, hash_MATCH, divergence) tested without PUSH contamination
  Phase 6 extended: triple merge + tail PUSH + mid PUSH cascade on star topology
  runs in 1.4s, consistently passes 10/10
- db_sync_doc: updated protocol architecture description
- chatgui: update db_sync_instance_add call with auto_sync=1
2026-07-24 21:27:23 +03:00
Evgeny ad56767646 db_sync: notify all peers when cascade_from changes chain (lock-free sync)
После cascade_from в db_handle_send_data оповещаем ВСЕХ пиров
с synced_pos >= fix_from о том что цепочка изменилась — откатываем
их synced_pos. Это устраняет неэффективность конкурентных sync:
раньше только пир-отправитель обновлялся, остальные видели устаревшие
хеши и обнаруживали ложное расхождение на следующем цикле.
2026-07-24 17:26:09 +03:00
Evgeny c46cb9a979 db_sync: fix si_send_data_batch column bug (author→node_id), add cascade_from to PUSH, add divergence+cascade test phases
- si_send_data_batch: неправильный SQL SELECT с несуществующей колонкой author
  вместо node_id → ломал всю передачу данных в REFINE/SEND_DATA
- Добавлена проверка si_prep и возврат int с кодом ошибки
- Все вызовы si_send_data_batch проверяют возврат, при ошибке сброс sync_state
- db_handle_push: добавлен db_cascade_from после вставки (раньше chain_hash
  не пересчитывался после PUSH)
- db_handle_refine: DEBUG_WARN при scnt==0 (fm за пределами диапазона)
- Добавлена публичная db_sync_chain_verify() для верификации цепочек
- test_db_sync: фаза 4 (divergence через разныe hashes + REFINE/SEND_DATA),
  фаза 5 (PUSH без хвоста + cascade verification)
- Убран мёртвый код db_sync_on_new_conn
- chatgui: адаптивная раскраска пузырей исходящих сообщений
2026-07-24 16:55:08 +03:00
Evgeny df8d910efa chatgui: fix Ctrl-C on Linux — remove thread_local from g_currentNode
SIGINT on Linux can be delivered to any thread (not just worker).
signal_handler uses g_currentNode->requestStop(); with thread_local,
g_currentNode was nullptr in non-worker threads — Ctrl-C did nothing.

Removing thread_local fixes signal delivery: g_currentNode is set once
before signal handler is registered, pointer read is atomic on x86/x64.
2026-07-24 16:02:49 +03:00
Evgeny e8920fe00e chatgui: MsgIsOutgoingRole support for outgoing message alignment 2026-07-24 15:55:34 +03:00
Evgeny fbc1bff1e1 chatgui: proper animation shutdown + TerminateProcess for rlottie threads
- AnimTimer::shutdown(): stop QTimer, clear m_icons before icon destruction
- ~MainWindow: AnimTimer::shutdown() -> cleanupAnimations() (correct order)
- main: replace ExitProcess with TerminateProcess(GetCurrentProcess,0)
  TerminateProcess skips DLL_PROCESS_DETACH, avoiding libwinpthread
  pthread_join hang on rlottie/miniaudio threads.
  rlottie C API has no global shutdown - internal RleTaskScheduler threads
  cannot be stopped through public API. Same for miniaudio WASAPI threads.
  All owned resources already freed before TerminateProcess.
2026-07-24 15:49:58 +03:00
Evgeny 73be79d204 chatgui: fix rlottie thread hang on Windows shutdown
Root cause: librlottie RleTaskScheduler creates 32 worker threads on first
lottie_animation_from_data call. These pthreads wait in WaitForMultipleObjects(INFINITE)
and never exit. On MinGW, exit()/pthread_join blocks forever.

Changes:
- messagelist: s_icons moved to file-static; cleanupAnimations() clears them,
  calling lottie_animation_destroy BEFORE static destruction
- mainwindow: ~MainWindow calls MessageList::cleanupAnimations() after node stop
- main: ShutdownSentinel counts threads at exit; if >4 remaining, calls
  ExitProcess(0) to bypass CRT pthread_join hang
- main: if rlottie properly shuts down (<=4 threads), normal return (no ExitProcess)
2026-07-24 14:54:12 +03:00
Evgeny 74c6b0d102 chatgui: add shutdown diagnostics — ~DbManager, ~SoundManager, g_shutdown_sentinel
Three diagnostic points covering the gap between ~MainWindow done and CRT exit:
- DbManager: destroyed — confirms local dtors started after ~MainWindow
- SoundManager: static destructor entered — confirms static destruction phase
- main: static destructors done (g_shutdown_sentinel dtor) — confirms ALL statics done

If process hangs, we know exactly which phase failed.
2026-07-24 14:34:00 +03:00
Evgeny 0d558245ce chatgui: move SoundManager shutdown to ~MainWindow before node cleanup
- SoundManager::shutdown() called explicitly in ~MainWindow (not static destruction)
- Before ma_engine_uninit: DEBUG_INFO diagnostic in timestamped log
- After ma_engine_uninit: DEBUG_INFO diagnostic
- Double-call safe: static destructor hits m_initialized=false, returns immediately
2026-07-24 14:22:28 +03:00
Evgeny fbaf58530f chatgui: rework shutdown — main thread owns cleanup, remove glШбаls
Changes:
- UtunNode::requestStop() — async stop from any thread (m_stop+wakeup)
- UtunNode::finalize() — cleanup in main thread (etcp_router, chat_sync, chat_core, utun_instance, uasync)
- UtunNode::stop() — requestStop + join + finalize (for tray Quit path)
- Worker runLoop: only exits poll loop on m_stop, then emits stopped — no cleanup
- stopped() handler: calls finalize() in main thread, then qApp->quit()
- Tray Quit: calls requestStop(), stopped handler does finalize+quit
- Ctrl-C (Windows): SetConsoleCtrlHandler calls requestStop() via g_ctrl_node
- Ctrl-C (Linux): signal_handler calls requestStop() via g_currentNode
- Removed: g_shutdown, g_signal_ua, utun_node_signal_shutdown globals
2026-07-24 14:14:27 +03:00
Evgeny 4c623e25ce chatgui: add shutdown diagnostics (poll exit reason, cleanup steps, app.exec return) 2026-07-24 13:56:29 +03:00
Evgeny faddbef32a chatgui: add SetConsoleCtrlHandler for reliable Ctrl-C on Windows 2026-07-24 13:51:23 +03:00
Evgeny a7ec45c4af db_sync: replace all sync logs with unified narrative format sync [TBL:NODE]
- Unified format: 'sync [SHORT_TBL:PEER_HI] <-/->  action: key_data'
- Each log line explains what the algorithm is doing and why
- Added batch progress tracking in dump-all and tail SEND_DATA loops
- Track dropped records (duplicates, bad_sigs, parse_fails) in SEND_DATA recv
- Show synced_pos transitions (old->new) in SEND_DATA and SYNC_DONE
- Show all launched syncs in PEER CHECK log
- Convert all silent returns to ERROR logs with [TBL:NODE] context
- Convert old WARN/ERROR formats to new sync [TBL:NODE] format
- Add sync_state reset details on ERROR and timeout paths
2026-07-24 13:23:17 +03:00
Evgeny ce9e58a925 chatgui: add scroll-to-bottom floating button, incremental message refresh via getMessagesSince 2026-07-24 04:07:51 +03:00
Evgeny b6d44bd947 chatgui: don't save scroll position when at bottom — restore to bottom on reopen
When scrollbar is at max, save lastPosMsgId=0 so restoreScrollPosition
falls through to scrollToBottom() instead of restoring a stale mid-scroll pos.
2026-07-24 03:52:05 +03:00
Evgeny 6cafff200b chatgui: add SIGINT/SIGTERM handler (as in utun.c), save scroll pos on every debounce
Signal handler via g_shutdown + uasync_wakeup ensures clean uasync poll exit.
updateReadPosition() now saves both lastReadMsgId and lastPosMsgId on every
scroll stop (300ms debounce), not just when read position changes.
connect(stopped, quit) triggers graceful Qt shutdown after uasync cleanup.
2026-07-24 03:47:15 +03:00
Evgeny 83ca0c65a4 chatgui: fix corrupted channel_id from non-null-terminated binary protocol
QString::fromUtf8(ch_id) read past length-prefixed string, capturing garbage.
Use fromUtf8(ch_id, ch_id_len) — length already available from bridge but was unused.
Messages now appear on refresh and unread badge correctly matches channel.
2026-07-24 03:33:14 +03:00
Evgeny fdc30d1a4e chatgui: synchronous DB write in saveChannelState/updateReadPosition, add shutdown debug logs
Replace gui_bridge_post_uasync_fn (async queue lost on shutdown) with direct
chat_core_save_ui_state (sync SQLite write). SQLite WAL handles concurrent access.
Add DEBUG_DEBUG traces in closeEvent, ~MainWindow, aboutToQuit.
2026-07-24 03:29:37 +03:00
Evgeny 393b7d36c8 chatgui: save scroll/read position on app quit via aboutToQuit
Make saveChannelState() public, connect to QApplication::aboutToQuit so
state is persisted on SIGINT, tray quit, or any other shutdown.
2026-07-24 03:24:54 +03:00
Evgeny 6ef21613c8 chatgui: pass author node_id in MSG_RECEIVED event, skip sound for own messages
Add author_node_id:8 to GUI_EVT_MSG_RECEIVED data format.
Callback now includes uint64_t author parameter.
SoundManager::play('msg_incoming') only when author != myNodeId.
2026-07-24 03:22:10 +03:00
Evgeny 37c5fa5607 chatgui: fix scroll jump on refresh, force debounce, add DEBUG logs
- Same-channel refresh: save scroll pos in memory, restore from memory (no DB race)
- Force m_readDebounce->start() after loadChannel (scrollToBottom may not trigger valueChanged)
- DEBUG_DEBUG logs in updateReadPosition, saveChannelState, onMessageReceived, setChannelUnreadCount
2026-07-24 02:59:15 +03:00
Evgeny 0c4862c66a chatgui: track read position via debounced scroll, only fully visible msgs are read
- Debounce timer (300ms) advances lastReadMsgId only when message is fully in viewport
- Unread badge no longer resets on channel select — only when actual scroll reads msgs
- saveChannelState() uses tracked m_lastReadMsgId instead of viewport bottom
- getChannels() reads lastReadMsgId from ui_state (was reading unused channels column)
2026-07-24 00:30:44 +03:00
Evgeny 216e516192 build: use explicit file list for liblmdb in CMake instead of GLOB
file(GLOB) + regex FILTER didn't work reliably on Windows. Replace with
explicit listing of mdb.c + midl.c (same as autotools).
2026-07-24 00:13:25 +03:00
Evgeny 726a4b8303 Revert unnecessary #ifndef _WIN32 guards in mplay.c — CMakeLists.txt filter handles exclusion 2026-07-24 00:03:03 +03:00
Evgeny e5dd41e5b2 build: exclude LMDB utilities from Windows CMake build, guard POSIX includes in mplay.c
file(GLOB) in libutun/CMakeLists.txt picked up mplay.c, mdb_dump.c and other
standalone LMDB tools that are not library code and fail on Windows. Filter them
out, keep only mdb.c + midl.c.
2026-07-23 23:58:39 +03:00
Evgeny 86d227bffe chatgui: remember last read message, restore scroll position, show unread count badge
Store read/scroll positions per channel via ui_state table (key-value).
Draw red +N badge on channel list items for unread messages.
Fix getMessages() to fetch last N messages (was fetching first N).
2026-07-23 23:47:35 +03:00
Evgeny de14f82048 chatgui: replace manual source lists with file(GLOB) in libutun/CMakeLists.txt
libutun/CMakeLists.txt: 151→84 lines (-67)
UASYNC_SOURCES: file(GLOB /*.c /liblmdb/*.c)
UTUN_COMMON_SOURCES: file(GLOB_RECURSE /*.c) excluding
  lwip_orig/, uip/, transport_layer/BBR/, utun.c
New .c files in src/ or lib/ are now automatically included.
2026-07-23 22:46:48 +03:00
Evgeny cf6108b1f1 chatgui: unify logging — debug_ui.h delegates to debug_config.h
debug_ui.h (50→10 lines): GUI_ERROR/WARN/INFO/DEBUG now delegate to
DEBUG_ERROR/WARN/INFO/DEBUG(DEBUG_CATEGORY_DEBUG, ...) from debug_config.h
Eliminates dual logging output, dual log file configuration.
Removed guiDebugFile() — debug_enable_file_output() already handles it.
Removed debug_ui.h include from utun_node.cpp (unused).
2026-07-23 22:45:26 +03:00
Evgeny 4aede4c6bd chatgui: extract MainWindow constructor into sub-methods
MainWindow (~15 lines): setupUi+setupTray+setupSoundFromConfig+
setupBridgeCallbacks+setupMessaging+setupNode+setupConnects
setupSoundFromConfig(): sound config parsing + loading
setupBridgeCallbacks(): 7 gui_bridge callback registrations
setupConnects(): 5 Qt signal connections
2026-07-23 22:44:32 +03:00
Evgeny 6c59ed43b2 chatgui: move crypto from MainWindow to transport layer
chat_channel.c: new chat_core_create_channel_auto(name) — generates
X25519/Ed25519 keys, random channel_id, Ed25519 signature, creates channel.
chat_core_create_channel_auto_trampoline for gui_bridge_post_uasync_fn.

mainwindow.cpp: onCreateGroupRequested() reduced from 100 to 15 lines.
Removed openssl/evp.h and openssl/rand.h includes from UI code.
2026-07-23 22:43:13 +03:00
Evgeny 3f662d85ba chat_core: split 912-line module into 5 files + delete dead chat_conn_mgr
chat_core.c (236 lines): init/destroy, getters, g_cc, helpers, si-registry
chat_msg.c (223 lines): submit_message, count, chain_hash, list_*, load_nodeinfo
chat_channel.c (197 lines): ensure_ready, create_channel, connect_channel, sign_join helper
chat_profile.c (189 lines): update_my_name, sync_my_addresses, save_ui_state
chat_status.c (118 lines): collect_status (NTP + ETCP connections)
chat_core_priv.h: shared g_cc extern + helpers

chat_conn_mgr.c/h: dead code (never compiled, never called)
CMakeLists.txt: add 4 new .c files to build
2026-07-23 22:41:28 +03:00
Evgeny 3448b0a482 channellist: fix dual blue selection — block signals + explicit select
- selectionCommand: remove Current flag to avoid redundant
  currentChanged emission during mouse press
- loadChannels: wrap clear+repopulate in blockSignals(true/false)
  to prevent selectionChanged and currentChanged from firing
  on intermediate model states
- selectChannel: use explicit selectionModel()->select() with
  ClearAndSelect + setCurrentIndex with NoUpdate
- simplified selection restore: only track selectedCid by ID,
  removed fallback-to-row logic
2026-07-23 21:31:00 +03:00
Evgeny 4ec930c8f0 chatgui: fix dual selection highlight and missing online count
- channellist.cpp: selectionChanged guard now checks currentIndex().row()
  is within model->rowCount() to avoid re-selecting stale index after
  model clear, which caused both channels to stay blue when switching.
- chat_sync.c: cs_refresh_channels now posts CHANNEL_PEERS_ONLINE
  for all channels after loading, fixing online count showing only
  on the last connected group.
2026-07-23 21:25:39 +03:00
Evgeny cbd9536a28 chatgui: fix invite link and Russian channel name encoding
- chat_sync.c: after invite trampoline, check if connection to invite
  node is already UP and trigger cs_on_conn_up to send CHANNEL_INFO_REQ.
  Fixes invite not completing when connection is reused (Fix 1 side effect).
- mainwindow.cpp: use name.toUtf8().constData() instead of qPrintable()
  to preserve UTF-8 encoding on Windows (Russian channel names).
2026-07-23 21:07:41 +03:00
Evgeny 23db012064 conn_mgr: fix duplicate connection and double-release bugs
- conn_mgr_connect_node: check instance->connections globally to reuse
  existing ETCP_CONN across groups instead of creating a new one (DUP addr)
- cm_start_phase_direct: close dead connection when etcp_link_new fails
  to prevent shadowing live connection in instance->connections hash
- cm_cleanup_db_node: remove redundant topo_node_registry_release
  (already done inside topo_node_free_lists)
2026-07-23 20:36:55 +03:00
Evgeny 58ad643516 Add debug logs to Ed25519 self-sig flow: computation, verification, skip due to no key 2026-07-23 20:00:11 +03:00
Evgeny 85269d1da3 Ed25519 self-signature in NODEINFO + fix sync sequence and zero-key bug
Add x25519_self_sig[64] to TOPOMSG_NODE/TOPO_NODE:
- topo_node.h: add field to both structs (wire format + memory)
- topo_node.c: compute Ed25519_sign(ed25519_privkey, x25519_pubkey) for own node
- topo_node.c: serialize/deserialize x25519_self_sig
- topo_group.c: verify Ed25519 self-sig on NODEINFO receive, reject forgery

Fix Ed25519 pubkey zero-key bug:
- topo_node_sqlite.c: get_ed25519_pubkey rejects all-zero key (MUST be non-zero)

Delay db_sync until Ed25519 key is available:
- db_sync.c: db_sync_on_conn_up checks db_get_ed25519_pubkey before starting
- db_sync.c: db_sync_peer_check_cb also checks Ed25519 key availability
- This prevents sync starting before NODEINFO with verified Ed25519 arrives
2026-07-23 19:55:15 +03:00
Evgeny 3890d2038d db_sync: add missing ERROR/WARN logs to all error branches
- db_record_insert: error logs for si_prep failures (SELECT, INSERT, cascade SELECT/UPDATE)
- si_parse_record: WARN logs for truncated header, data/sig overrun
- db_chain_hash_at: ERROR logs for missing row, corrupted blob
- db_prev_chain_hash: ERROR log for si_prep failure
- db_cascade_from: ERROR logs for si_prep failures
2026-07-23 18:16:53 +03:00
Evgeny acd4a189ae db_sync: add author_sig VERIFY FAIL diagnostics + SYNC_DONE retry limit
- db_verify_author_sig: show ed_pubkey and sig first bytes on failure
- SI_PEER: add sync_retry_count field
- db_handle_sync_done: limit re-initiate to 3 retries before giving up
- db_sync_initiate_sync / db_sync_on_conn_up: reset retry_count
2026-07-23 18:13:14 +03:00
Evgeny b48d3b4a77 db_sync: add diagnostic logs for message sync debugging during join
- db_handle_init_sync: TRACE→INFO with peer_cnt/my_cnt/tp_offset
- db_handle_send_data: detailed batch info (range, remaining count)
- db_sync_initiate_sync: add table name to log
- db_sync_on_conn_up: add first table name to summary log
2026-07-23 16:47:22 +03:00
Evgeny 3e95cf5ca6 fix: dialog hangs on join success + remove per-packet spam logs
- chat_sync.c: move pending_invite clear from INFO_RESP to WELCOME,
  post GUI_EVT_CONNECT_RESULT on invite success in cs_handle_welcome
- etcp_connections.c: NORM_DECRYPT_TRY/CRYPTO_DECRYPT_OK DEBUG_DEBUG->TRACE
- secure_channel.c: ENCRYPT/DECRYPT DEBUG_DEBUG->TRACE
2026-07-23 16:36:40 +03:00
Evgeny 4f831703f0 fix: etcp_conn_set_peer_node_id переиндексация при state=0 + conn_mgr ставит peer_id сразу 2026-07-23 16:19:03 +03:00
Evgeny 511a2dcd06 fix: авто-подключение Phase 2/3 — запрос всех нод из node_addresses, без привязки к peers_<ch> 2026-07-23 14:41:22 +03:00
Evgeny 196c2896d3 debug: sochet count after instance_init (категория debug, видно при текущем конфиге) 2026-07-23 14:17:58 +03:00
Evgeny 01a0a568a6 debug: crypto_ctx session_key логгирование — CRYPTO_CONN_READY, CRYPTO_ON_UP, CRYPTO_DECRYPT_OK, DECRYPT_FAIL+seskey 2026-07-23 14:04:44 +03:00
Evgeny 3e0a4e0eb1 fix: замена malloc на u_strdup в channelSelected — u_free падал на чужой памяти 2026-07-23 13:12:04 +03:00
Evgeny 0e324cf36d chat_core: connect_channel API — запуск topo_group_connect_init при выборе канала из GUI, убрано дублирование создания групп в utun_node 2026-07-23 12:51:45 +03:00
Evgeny be258eea4d fix: remove stale peers_* CREATE TABLE from chat_core, drop ORDER BY node_RTT
chat_core_ensure_channel_ready() had a duplicate CREATE TABLE peers_*
with OLD schema (no node_RTT, node_type, connected). Since it was called
from 4 places WITHOUT topo_node_sqlite_channel_put() first, the old
schema was created first, and IF NOT EXISTS in the new DDL was a no-op.

Removed the entire old block. Only topo_node_sqlite_channel_put() now
creates peers_* tables with the full schema.

Also removed ORDER BY p.node_RTT from get_public_peers/get_local_peers —
ordering not needed for sequential connect attempts.
2026-07-23 00:24:16 +03:00
Evgeny 8facce46ac feat: auto-connect Phase 3 (local/netif addresses) + restart fix
Phase 1: connected peers (connected=1)
Phase 2: direct/EIM (addr_type=1,2), exclude self
Phase 3: local/strict NAT (addr_type=0,3), exclude self
Timeout: 5000→2000ms per attempt
Restart: DOWN triggers reconnect when phase=DONE & active=0

Debug logs: init, Phase1 per-peer, UP phase info, restart branches
2026-07-22 23:55:44 +03:00
Evgeny 7dee05c628 fix: node_registry crash — TOPO_NODE_REG_ENTRY node/lldata overlap
struct TOPO_NODE_REG_ENTRY имел node и ll.data[0] (гибкий массив) на одном смещении.
memcpy(re->ll.data, &node_id, 8) затирало re->node значением node_id,
из-за чего node_registry_find возвращал не указатель а битый node_id → SIGSEGV.

Убрал struct TOPO_NODE_REG_ENTRY, перешёл на явные смещения:
data[0..7]=ключ, data[8..15]=указатель, queue_entry_new(16).
2026-07-22 23:05:04 +03:00
Evgeny 6edd40d141 topo_group_connect: авто-подключение CHAT-групп + conn_mgr_connect_from_invite + замена chat_conn_mgr в chatgui
Добавлено:
- conn_mgr_connect_from_invite() — подключение к новому узлу по invite-данным
  (INIT → INVITE_INFO_REQ/RESP → проверка членства в группе → сохранение в БД)
- topo_group_connect.c — авто-подключение при старте CHAT-группы:
  Phase 1: одновременный connect всех connected=1 пиров, таймаут, отмена незавершённых
  Phase 2: последовательный перебор публичных адресов до первого успеха
- Счётчик active_conn_count с проверкой nodeinfo-путей (UP/DOWN)
- restart() при phase=DONE и count==0

БД:
- connected INTEGER NOT NULL DEFAULT 0 в peers_<ch> + ALTER TABLE для существующих
- topo_node_sqlite_set_connected / get_connected_peers / get_public_peers

Chatgui:
- Включён topo_group (убран utun_instance_set_topo_group_enabled(0))
- Создание TOPO_GROUP_TYPE_CHAT при channel_ready и при старте
- Invite connect через conn_mgr_connect_from_invite
- Удалён chat_conn_mgr (из сборки) + весь старый auto-connect код
2026-07-22 22:48:12 +03:00
Evgeny 4215349349 topo_recovery: каскадное восстановление узлов с группировкой по next_hop
При разрыве ETCP-соединения модуль собирает каскадно отвалившиеся узлы:
- add_node из hop_list вычисляет next_hop относительно failed_peer,
  группирует узлы в отдельные recovery-контексты (direct/indirect списки)
- start запускает асинхронный перебор для каждой группы параллельно
- is_next_hop-узел пробуется первым с приоритетом по мин. RTT
- таймаут 2с на попытку, conn_mgr_cancel_callback при отмене
- отмена: topo_group_new_conn (on_up) находит узел в любом ctx -> ctx целиком
- завершение: оба списка пусты -> освобождение ctx
2026-07-22 19:01:55 +03:00
Evgeny f6b37a857b conn_mgr: bind to topo_group, DB node loading, configurable DIRECT timeout, tests
- conn_mgr_init(TOPO_GROUP* group) — работает внутри заданной группы
- conn_mgr_connect_node: загрузка узла из SQLite если нет в BGP-таблице
- topo_node_sqlite_node_load(): загрузка TOPO_NODE (ref=0) из БД
- cm_start_phase_direct: db_loaded-ветка — перебор всех адресов без NAT-фильтра
- cm_cleanup_db_node: удаление TOPO_NODEQ из группы при неудаче, только если нет путей
- cm_start_phase_indirect: исправлено зависание при best_candidates==0 (cm_deliver_result)
- conn_mgr_set_direct_timeout_ms(): конфигурируемый таймаут фазы DIRECT
- conn_mgr_destroy до etcp_router_destroy (исправлен порядок)
- test_conn_mgr: +2 сценария (DB-успех, DB-недоступен), проверка topo_group
2026-07-22 17:33:58 +03:00
Evgeny 852ce401e7 Добавлен group_id в etcp_router и topo_group, удалён msg_transport
- etcp_router: SVC_ROUTE_HDR (+8B), ETCP_ROUTER_CONN/TRANSIT_QUEUE хеши расширены под group_id
- Все caller'ы (proxy, routing, conn_mgr, nat, chatgui, tests) обновлены
- topo_group: глобальный реестр TOPO_NODE по node_id, ref_count=0 при создании
- BGP-обмен активируется для всех групп (а не только дефолтной)
- Протокол: group_id добавлен в WITHDRAW/TABLE_REQ/TABLE_COMPLETE/ERR_GROUP_MISMATCH
- Per-connection коллбэки заменены на instance-level conn_status
- msg_transport полностью удалён из проекта
57/57 тестов
2026-07-22 15:51:03 +03:00
Evgeny 5265783164 fix: self node online=0 after chat_core_init refactor — add topo_node_sqlite_node_set_online(self, 1) 2026-07-22 13:54:36 +03:00
Evgeny e2d9879636 refactor: remove direct nodes table writes from chat_core/conn_mgr, use topo_node_sqlite API
- chat_core_init: replace INSERT INTO nodes with topo_node_sqlite_node_update_verified, remove redundant topo_node_sqlite_init call
- chat_core_update_my_name: replace UPDATE nodes with topo_node_sqlite_node_update_verified
- chat_conn_mgr_connect_from_invite: simplify invite node insert (x25519 only, no ed25519)
- topo_node_sqlite: add detailed column comments to peers table DDL (both server and chatgui)
- chat_core: add detailed column comments to peers table CREATE TABLE
2026-07-22 13:38:17 +03:00
Evgeny 7aa4a34f35 fix chatgui: members showing offline after JOIN — always refresh channel cache before posting online count 2026-07-22 12:19:59 +03:00
Evgeny a7346c25e3 fix chatgui: members not showing (schema mismatch) and username showing JSON instead of name
- topo_node_sqlite_member_put: remove joined_at column from INSERT (not in server schema, duplicates join_ts)
- chat_core: rename peers table column name→userinfo to match server schema + add ALTER TABLE migration
- chat_sync: remove 3 redundant INSERT INTO nodes that wrote raw JSON userinfo into nodes.name, overwriting correct plain name from topo_node_sqlite_node_update_verified
2026-07-22 12:03:57 +03:00
Evgeny df809f251c ntp: use NTP-corrected time everywhere instead of unixepoch()/gettimeofday
- db_sync_next_timestamp(): ntp_time_get_us() via si->db_sync->inst
- topo_node_sqlite_node_put: +time_t now_sec param, replace unixepoch()
- topo_node_sqlite_node_update_verified: +time_t created_at param
- topo_node_sqlite_member_put: joined_at = join_ts (already NTP-corrected)
- All INSERTs (nodes, local_identity, accounts, peers):
  replace DEFAULT(unixepoch()) with explicit ntp_time_get_seconds()
- ON CONFLICT DO UPDATE: created_at = COALESCE(nodes.created_at, excluded.created_at)
- Updated callers: topo_group.c, chat_sync.c, chat_conn_mgr.c, chat_core.c
2026-07-22 11:39:10 +03:00
Evgeny fab1c77cfb chatgui: remove unused addServer/addClient/removeServer/removeClient 2026-07-22 10:22:37 +03:00
Evgeny 6574c58257 chatgui: unify config update logic, fix node_id derivation, fix test
- config_updater.{h,cpp}: single implementation — config_update_in_lines
  for batch updates + config_update_option for file-level. REPLACE/APPEND
  modes, section auto-creation
- node_config.cpp: generateIdentity() now derives node_id via
  SHA256(pubkey) (like sc_derive_node_id_from_pubkey) instead of random.
  save() uses config_update_in_lines for line-level update when file
  exists, saveFull() only for new file and addServer/addClient.
  Removed duplicate findSectionBounds/updateOptionInSection/configUpdateOption
- test_config_updater.cpp: Q_ASSERT replaced with VERIFY macro — Q_ASSERT
  in Release mode does not evaluate the expression, causing false positives.
  Test now works in both Debug and Release
2026-07-22 10:13:40 +03:00
Evgeny ab252a0984 chatgui: include platform_compat.h for htole64 on Windows 2026-07-22 09:23:35 +03:00
Evgeny b1cbd5acf9 topo_node_sqlite: dynamic idx_sql allocation instead of fixed 128-byte buffer 2026-07-22 09:18:54 +03:00
Evgeny 2785845acf platform_compat: use #include <process.h> instead of getpid macro 2026-07-22 09:13:36 +03:00
Evgeny b1bf27d023 fix Windows warnings: getpid, htole64, gettimeofday, socket_t mismatches, radix min 2026-07-22 09:08:19 +03:00
Evgeny b9a31b9fba proxy: fix Windows warnings - getsockopt cast, uasync_add_socket_t 2026-07-22 08:55:48 +03:00
Evgeny 17202beb0a test_ntp: use compat headers instead of arpa/inet.h, add rule to AGENTS.md 2026-07-22 08:47:56 +03:00
Evgeny 090acc1f00 etc/rtt: UPSERT node_update_verified; gui: account delegate (rtt+nat subline), context menu member props (admin/supernode), cleanOrphanedPeers, refreshRtt timer 2s 2026-07-22 01:15:33 +03:00
Evgeny c367144776 etcp: rtt_last→ping_update_rtt раз в минуту; route_ping: ping_update_rtt на PING_RESP 2026-07-21 22:29:19 +03:00
Evgeny ad42f66df5 feat: PING/PONG RTT exchange + burst sender + TCP server; fix INIT peer_id offset
- etcp_connections: add flags byte to PING/PONG wire format (WANT_RTT/SEND_RTT)
- etcp_connections: add etcp_link_burst_start/check for BDP measurement
- etcp_connections: add tcp_server_on_link for STCP→ETCP bridge
- topo_node: add ping_request_cbk/ping_update_rtt for live RTT tracking
- topo_node: add last_ping_time field in connectivity
- topo_node_sqlite: extend DB schema for RTT data
- fix: INIT path read peer_id from offset 1 (not 2), PING offset stays 2
2026-07-21 20:50:14 +03:00
Evgeny 542694a0e9 fix: restore original source content after refactoring (accidental code changes removed) 2026-07-21 20:35:02 +03:00
Evgeny fcf3fb7f51 refactor: move transport (etcp/stcp/normalizer) to transport_layer/, routing (bgp/pings/topo*) to routing_layer/ 2026-07-21 20:25:41 +03:00
Evgeny 3dda701c67 schema: rm is_dm, rename name→userinfo (json_flat), fix join_msg format
- Remove is_dm column from channels table (dm detection by member count)
- Rename member name column to userinfo — extensible json_flat format
- Add json_flat.c/h library for compact JSON serialization
- Fix join_sig message format: use ch_x25519+ch_ed25519 instead of channel_id string
- Remove name from join_msg body (now in userinfo via json_flat)
- Update chat_core, chat_sync, member_sync, db_manager to new format
- Add topo_strategy.txt — архитектурные заметки по топологии
2026-07-21 00:11:50 +03:00
Evgeny 88d51349ea statuspage: компактный блок часов, offset в секундах
- Шрифт часов 20 (был 22), bold оставлен
- Убраны зазоры: padding 10px→1px, margins 12,8→4,0, spacing 8→2
- Offset в NTP-статусе: микросекунды → секунды (114.8 s)
- Убран зазор (пустая строка) между NTP и Connections
2026-07-20 01:31:49 +03:00
Evgeny 333acb2fe5 ntp_node_time: учёт пол-RTT в корректировке времени
offset = t4 - t_sender_us - avg_rtt/2
RTT усредняется по всем линкам соединения (rtt_last в 0.1ms).
2026-07-20 01:19:33 +03:00
Evgeny f83efda54e ntp_node_time: исправлен знак offset — корректировка шла в обратную сторону
ntp_time_get_us() вычитает offset (local_us - offset_us), поэтому
offset_us должен быть local_raw - peer_time, а не наоборот.
Было peer_time - local_raw → corrected = 2*local - peer (ошибка).
Стало local_raw - peer_time → corrected = peer_time (верно).
2026-07-20 01:07:09 +03:00
Evgeny fdaaa41010 ntp_node_time: фикс ложных докладов о дрейфе часов
check_drift() вызывался с сырой разницей часов между узлами, а не
с изменением offset со временем. При разнице часов между машинами
в 229 секунд это давало спам WARN на каждый TIME_SYNC обмен.

Теперь: первый обмен сохраняет offset без проверки; последующие
сравнивают дельту и warn только при реальном дрейфе (изменении offset).
2026-07-20 00:53:53 +03:00
Evgeny 15ca0675c0 Безопасность: pubkey check при reusing линка + node_changed при конфликте адреса чужого конна 2026-07-20 00:47:38 +03:00
Evgeny ccd35074d8 Заменил find_link_index на ll_queue с хеш-индексом: универсальный ключ port(2)+addr(16)+family(1)=19 байт, IPv4 дополнен нулями 2026-07-20 00:21:12 +03:00
Evgeny 7a5df3e196 etcp: add node_changed callback chain + chatgui subscription
- etcp.h: add node_changed_cbks field to ETCP_CONN
- etcp_api.h/c: add etcp_conn_node_changed() + add/remove_node_changed_cbk
- etcp_connections.c: fire node_changed callbacks on pubkey mismatch
  (with DEBUG_WARN(GENERAL) logging source addr, peer_id, old/new pubkey)
- etcp.c: cleanup node_changed_cbks on connection close
- chat_conn_mgr.c: subscribe to node_changed — clear DB addresses,
  reset in-memory cache, close connections (deferred for current conn),
  post GUI_EVT_NODE_CHANGED
- gui_bridge.h: GUI_EVT_NODE_CHANGED=11
- gui_bridge_impl.cpp: handle GUI_EVT_NODE_CHANGED
2026-07-19 22:09:23 +03:00
Evgeny 1b21ee9d73 chatgui: live NTP-corrected clock (HH:mm:ss) on Status page with per-second sync 2026-07-19 20:38:22 +03:00
Evgeny a7d842440c chatgui: split chat_core into chat_conn_mgr — connection manager with rotation, max_parallel slots, background ping, per-connection DOWN callback
- chat_conn_mgr.c/h: new connection manager (1 per instance, global in g_cc)
  - cm_rotation() — single rotation method: random addr pick, INIT, retry
  - 10 parallel slots per node (CM_MAX_PARALLEL), evict worst RTT on excess
  - DEAD slot reuse before close — close only when replacement ready
  - per-connection DOWN callback via etcp_conn_add_down_cbk
  - background ping cycle: random node→random addr RTT measurement
- chat_core.c: added g_cc.conn_mgr field, init/destroy/getter
- chat_sync.c: migrate to new API, add_node() at WELCOME/JOIN/PEER_UPSERT
- Removed old ca_state/ca_ctx/ca_* single-flight code
2026-07-19 19:00:29 +03:00
Evgeny 98bfb9dcd2 chatgui: per-event sound settings with DB persistence
- Sound files loaded from filesystem (sound_dir in INI config, absolute or relative)
- Per-event enable/disable + volume stored in DB (ui_state table)
- SoundManager: setEvent(), effective volume = master * per-event
- SoundSettingsPage UI: 4 event rows (checkbox + slider)
- chat_core_save_ui_state() for async DB writes via uasync thread
- DbManager::getUiStateInt() for reading settings from GUI thread
- Default sounds copied to sounds/ dir next to DB, config entries written
2026-07-19 15:17:52 +03:00
Evgeny 1593079da8 chatgui: fix sound_manager use-after-free crash on Windows
- Keep mp3Data + ma_decoder alive for sound lifetime (was freed before audio thread)
- Remove MA_SOUND_FLAG_DECODE — stream from decoder instead of pre-decoding
- Shutdown in reverse order: sound → decoder → mp3Data
2026-07-19 14:58:34 +03:00
Evgeny c2ad2e78c3 chatgui: add sound playback via miniaudio (MP3 notification sounds)
- SoundManager singleton wraps ma_engine + ma_sound (non-blocking, background thread)
- MP3 files embedded in QRC, decoded to PCM at startup with MA_SOUND_FLAG_DECODE
- miniaudio.h (MIT) in lib/, IMPLEMENTATION in separate miniaudio_impl.c
- Sound settings page in SettingsDialog: enable/disable, volume slider (0-100%)
- Config keys [gui] sound_enabled, sound_volume
- Incoming messages trigger msg_incoming.mp3 sound
2026-07-19 14:46:21 +03:00
Evgeny f6f7e12192 refactor: demote high-frequency debug logs (raw_recv/dispatch→TRACE, normalizer→DEBUG), NTP drift→WARN 2026-07-19 13:53:27 +03:00
Evgeny 42041dfe84 feat: log connection status changes at general=info level (NEW/UP/DOWN/DELETE) 2026-07-19 13:41:25 +03:00
Evgeny beed7a4e07 fix: INIT reset logic — reset only on session change or got_initial_pkt, always sync session_id 2026-07-19 13:18:42 +03:00
Evgeny 58df4d8571 ntp_node_time: переход с etcp_router на raw ETCP (P2P)
- Добавлен ETCP_ID_NTP_TIME=0x04 (raw ETCP packet ID)
- Удалён ETCP_RT_ID_NTP_TIME=0x12 (router-сервис)
- ntp_node_time.c: etcp_router_bind/send → etcp_bind/etcp_send
- send_time_sync() принимает ETCP_CONN* вместо dst_node_id
- recv_cb разбирает raw dgram вместо SVC_ROUTE_HDR
- Обновлены doc-файлы
2026-07-19 12:16:37 +03:00
Evgeny 2e7c32c893 fix(db): use INSERT ... ON CONFLICT instead of INSERT OR REPLACE for nodes table
INSERT OR REPLACE deletes the old row and inserts a new one,
resetting unspecified columns (online, update_ts) to DEFAULT 0.
This was wiping online=1 set by cs_on_peer_status_changed/member_sync_set_online.

Replaced 5 INSERT OR REPLACE INTO nodes with INSERT INTO ... ON CONFLICT
DO UPDATE SET — only updates the specified columns, preserving online status.

Files:
- chat_core.c:638 (invite flow)
- chat_sync.c:1045 (CHANNEL_INFO_RESP), 1193 (JOIN), 1449 (PEER_UPSERT)
- topo_node_sqlite.c:121 (BGP node_put)
2026-07-19 11:16:26 +03:00
Evgeny cf46a59d3a fix(chatgui): replace qDebug with DEBUG_INFO for online trace logs 2026-07-19 11:04:35 +03:00
Evgeny ac0d1c18ed debug(chatgui): add online status trace across full chain
- cs_is_peer_online: log found/init/links/online
- cs_on_peer_status_changed: log channel_count + found_in_channel
- cs_handle_welcome/cs_handle_channel_join: log before cs_post_channel_online
- MainWindow::onChannelPeersOnline: log cid+online
- ChannelList::setChannelPeersOnline: log search/rows, each row cid+match, SET or NOT FOUND
- ChannelList::loadChannels: log each added row cid+name
2026-07-19 10:58:25 +03:00
Evgeny 9de16be012 fix(chatgui): remove duplicate loadChannels after JoinDialog close, remove 800ms delay
- showJoinDialog: don't call loadChannels after dialog close — channel already in model via CHANNEL_UPDATED event, online badge via CHANNEL_PEERS_ONLINE. Duplicate loadChannels was resetting the peer online count.
- onConnectResult: remove 800ms QTimer — dialog closes immediately. Channel data arrives independently through the event pipeline.
2026-07-19 10:50:25 +03:00
Evgeny 3a7b359f87 fix(chat_sync): online status for joiner on inviter side not sent to GUI/peers
- cs_handle_channel_join: call cs_post_channel_online immediately after adding joiner
- member_sync_start with _on_invite_sync_done callback: on MT_OK push online=1 via merkle_sync_push_update
- DEBUG per-peer online status in cs_post_channel_online
2026-07-19 09:30:51 +03:00
Evgeny e3286d409f chatgui: кнопка + для invite в каналах, автокопирование ссылки, автоподключение из буфера
- Замена контекстного меню 'Пригласить' на кнопку + (18x18) справа снизу каждого канала с hover-эффектом
- InviteDialog: авто-копирование ссылки в буфер (Clipboard + Selection) при открытии
- JoinDialog: параметр autoJoinLink — если в буфере валидная utun:// ссылка, диалог сразу стартует подключение
2026-07-19 09:05:55 +03:00
Evgeny 501b636d19 topo_group: disable in chatgui; fix NAT detection for matching addresses
- Add utun_instance_set_topo_group_enabled() flag (default=1, chatgui sets 0)
- Move nat_detection create/destroy from topo_group.c to utun_instance.c (unconditional)
- NAT detection: simplify third_node search (any connected node)
- NAT detection: allow NULL topo_groups in receive_cbk and handlers
- NAT detection: handle NAT_INFO without topo_group (update socket directly)
- route_ping: make TOPO_GROUP optional in send_req_addr/handle_req
- topo_node: when NAT verified and address matches interface, set entry type to NAT instead of INTERFACE + socket_id|1
- topo_node: detect nat_type change in changed computation for rebuild
- test_nat_detection: fix socket_id check (|1 for NAT entries)
- chatgui: call utun_instance_set_topo_group_enabled(0)
2026-07-19 08:33:33 +03:00
Evgeny 208a0c8ae8 fix(chat_sync): peer online status not updated in invite flow
- cs_on_conn_up: call cs_on_peer_status_changed(peer,1) before return
  in invite path (was skipped, leaving inviter at online=0 in DB)
- cs_handle_welcome: remove unconditional set_online(0) — online
  status is managed by conn_up/conn_down, not by WELCOME
- cs_handle_welcome: add cs_post_channel_online() after WELCOME
  to send PEERS_ONLINE to GUI when channel is fully loaded
- cs_post_channel_online: add DEBUG log for PEERS_ONLINE event
- db_sync peer_check: suppress spam log when no peers connected
2026-07-19 07:54:06 +03:00
Evgeny ca6a27aadb BGP: 2 addr per socket (remove TOPO_ADDR_REAL), config IP priority, private IPs in SQLite, NAT addr != INTERFACE 2026-07-19 07:00:04 +03:00
Evgeny 4df7195a58 debug: add ADDR_SYNC logs for BGP node_put DELETE and member_sync recv ac 2026-07-19 00:09:46 +03:00
Evgeny 586d46f1e6 fix: WELCOME wire format — flags+update_sig parsing, peer_needed gap, invalid join_sig skip 2026-07-18 23:53:04 +03:00
Evgeny 39a376e255 fix: BGP и чат больше не затирают адреса друг друга в node_addresses
- BGP DELETE: +AND socket_id != 0 — не трогает чат-адреса (invite/JOIN/WELCOME)
- Чат invite DELETE: +AND socket_id=0 — не трогает BGP-адреса
2026-07-18 23:23:14 +03:00
Evgeny 70930b784f chatgui: fix — локальные адреса не записывались (g_cc.initialized до sync_my_addresses) + таймаут коллбэк 12→20 байт 2026-07-18 22:18:01 +03:00
Evgeny 57f861f360 chatgui: диагностические логи ADDR_SYNC во всех точках записи/удаления адресов в БД
- chat_core_sync_my_addresses: DELETE + INSERT с подсчётом строк, CHECK verify
- chat_core_connect_from_invite: DELETE + INSERT с IP:port
- member_sync_put: DELETE + INSERT с IP:port, warn если addrs_data=NULL
- cs_handle_channel_info_resp: адрес инвайтера из ETCP link
- cs_handle_channel_join: адреса джойнера
- cs_handle_peer_upsert: адреса пира
- cs_handle_welcome: адреса из WELCOME

Все логи с префиксом [ADDR_SYNC] в DEBUG_CATEGORY_DEBUG
2026-07-18 21:46:46 +03:00
Evgeny 0b385dca50 chatgui: обновление статуса/адресов узлов через чат-протокол
- merkle_sync: synced-флаг в сессии, MSG_ITEM_UPDATE(0x04), push_update() — рассылка лёгких дельт (online-статус) всем synced-соседям
- member_sync: apply_update в ops, node_updated_cb для GUI-нотификации внешних обновлений
- chat_sync: cs_on_peer_status_changed() — единая функция: БД + GUI + push_update + throttled member_sync_start (1/сек)
- cs_on_conn_down: исправлено — теперь сбрасывает nodes.online=0
- flush при destroy
2026-07-18 21:28:59 +03:00
Evgeny 20f5e6c953 sqlite: add cache_size/mmap_size/temp_store PRAGMAs to db_sync and topo_group — 159x insert speedup, fix test_chat_sync_stress timeout 2026-07-18 20:16:24 +03:00
Evgeny 0e0ad9316a fix race condition: DB_READY lost when utun thread starts before gui_bridge_init
- mainwindow.cpp: swap setupMessaging() before setupNode() so g_receiver is created first
- gui_bridge_impl.cpp: add DEBUG_WARN when gui_bridge_post drops event due to NULL g_receiver
2026-07-18 18:08:09 +03:00
Evgeny 29e2527b6b consolidate 3 SQLite connections into 1 shared handle
- utun_node.cpp: open topo_sqlite_db with FULLMUTEX + all PRAGMAs (WAL, FK, NORMAL sync, checkpoint)
- db_sync.c: reuse inst->topo_sqlite_db if available (shared_db flag)
- chat_core.c: reuse inst->topo_sqlite_db if available (shared_db flag)
- db_sync_destroy/chat_core_destroy: skip close on shared handle
- Only topo_groups_destroy closes the shared handle
- Fallback: each module opens own connection if topo_sqlite_db is NULL
2026-07-18 17:33:29 +03:00
Evgeny 853297290f chatgui: fix NodesPage truncated Node ID + add DB diagnostics
- NodesPage: show full 64-bit Node ID (hex16) instead of truncated 4 hex chars
- NodesPage: show diagnostic when DB handle is NULL (was silent return)
- DatabaseSettingsPage: show 'Database not connected' when DB handle is NULL
- DatabaseSettingsPage: add refreshStatus() called on tab select
- SettingsDialog: refresh Database tab on category change
2026-07-18 17:04:50 +03:00
Evgeny 93665c50f0 etcp: remove dead set_*_cbk functions, use add/remove only 2026-07-18 16:52:43 +03:00
Evgeny ef1922327a rename ready_cbks → init_cbks (connection init, not fully ready)
ready_cbks/etcp_conn_set_ready_cbk/add_ready/remove_ready → init_cbks/..._init_cbk
ca_ready_cb → ca_init_cb
cm_direct_ready_cb → cm_direct_init_cb
cm_reverse_ready_cb → cm_reverse_init_cb
ntp_node_on_conn_ready → ntp_node_on_conn_init
connect_ready_cb → connect_init_cb

init accurately describes when conn->initialized=1, before keepalives establish UP.
Also fire etcp_fire_conn_status INIT at this point instead of create time.
2026-07-18 16:38:24 +03:00
Evgeny ef3d7c3f37 chat_core: call ensure_channel_ready on invite — create msg/peers tables + register in db_sync
Without this, invite creates the channel in SQLite but db_sync never
knows about it, so peer sync fails with 'instance NOT FOUND'.
2026-07-18 16:14:53 +03:00
Evgeny 4188afb3c5 db_sync: lazy-register channel instance on INIT_SYNC when not pre-registered
When INIT_SYNC arrives for an unknown hash, scan all msg_* tables in
sqlite_master, compute SHA256 of channel_id, match the hash, and
auto-register via db_sync_instance_add. Fixes invite flow where
channel was created but db_sync wasn't notified.
2026-07-18 16:08:47 +03:00
Evgeny a510cb0018 auto_connect: add DB dump diagnostics (node_addresses state, node existence, db path+status)
- chat_core.c: dump node_addresses total, per-node counts, nodes lookup before addr query
- topo_group.c: log db path, rc, and db pointer after sqlite3_open
2026-07-18 15:47:05 +03:00
Evgeny 3aadaa3396 fix: sqlite3_open wrong path in topo_group.c (db_path vs db_path/chats.db), keep separate connections for perf
- topo_group.c: open {db_path}/chats.db instead of {db_path} — was opening a directory
- chat_core.c: remove inst->topo_sqlite_db = g_cc.db — keep topo_sqlite_db as separate
  uTun-only connection without FULLMUTEX, avoiding contention with GUI reads
2026-07-18 15:41:02 +03:00
Evgeny 9245a30fa5 auto_connect: diagnostic logging for early FAIL paths in chat_core_connect_auto
- chat_core.c: add DEBUG_INFO before each synchronous cb(FAIL) — pinpoints exact failure reason
- chat_sync.c: add log in ac_fill when ca_state=NULL after chat_core_connect_auto call
2026-07-18 15:06:23 +03:00
Evgeny aa1ed006d5 etcp: instance-level conn status callbacks (NEW/UP/DOWN/DELETE), fix member online in chatgui
- Add etcp_add_conn_status_cbk/remove/set — unified callback for all connections
- 4 statuses: NEW, UP, DOWN, DELETE — one subscription covers all connections
- Fire points in etcp.c: create→NEW, on_up→UP, on_down→DOWN, close→DELETE
- chat_sync.c: replace cs_on_new_conn+per-conn subs with cs_on_conn_status
- db_sync.c: same migration to instance-level callback
- cs_is_peer_online: use conn->links_up instead of link->link_status (was 0 during INIT handshake)
2026-07-18 14:51:01 +03:00
Evgeny ef27633a88 chatgui: clear clipboard before setText for reliable copy on Linux 2026-07-18 13:53:01 +03:00
Evgeny e71cae16c1 db_sync: unified msg table — one DB, one table per channel
- db_sync: rename column author→node_id, open chats.db instead of sync file,
  db_sync_instance_add takes table_name directly
- chat_core: remove CREATE TABLE msg_ch_*, remove on_db_sync_insert (74 lines),
  remove fallback INSERT in submit_message, add minimal on_msg_inserted
  for GUI notification only
- db_manager: update getMessages for new schema (parse JSON from data column),
  fix ORDER BY to use author_signature instead of datahash,
  update syncCursorNext to parse JSON
- si_register simplified: db_count from same table, no more msg vs sync gap
2026-07-18 13:50:35 +03:00
Evgeny 7c2798ec0e debug: add diagnostic logging for ETCP recv/INIT/dispatch paths
- etcp_connections.c: raw_recv log, INIT conn lookup details, insert_link DUP details
- etcp_api.c: dispatch id/cb logs with DROPPED warning when no handler
- chat_sync.c: etcp_send rc log, fix ac_result_cb to always clear ca_state
2026-07-18 13:04:10 +03:00
Evgeny a65478edcc chatgui: fix duplicate connection on invite when node already connected
- chat_core_has_conn: search all connections (pending+ready) by peer_node_id
- chat_core_connect_from_invite: skip chat_core_connect_auto if already connected
- ca_timeout_cb: close timed-out connection to free socket link slot
  + cancel remaining timers + ca_cleanup on all-failed
2026-07-18 11:24:51 +03:00
Evgeny 93871c592b db_sync: timeout for stuck sync_state=1, handle ERROR properly
- Add sync_start_tb to SI_PEER, set in db_sync_initiate_sync
- peer_check_cb: after 15s timeout reset sync_state=0 with WARN
- db_handle_error: accept si, reset sync_state=0, DEBUG_ERROR with details
- DB_SYNC_SYNC_TIMEOUT=15 added to db_sync.h
2026-07-18 10:52:54 +03:00
Evgeny de4099a26b logging: cleanup connection establishment logs
- Fix wrong categories: BGP/DEBUG -> ETCP for connection lifecycle events
- Fix level: WARN -> DEBUG for normal out-of-order pkts before init
- Fix level: INFO -> DEBUG for per-packet TX DATA spam
- Remove duplicate socket init logs (GENERAL duplicates of ETCP)
- Add missing INFO: INIT_RESPONSE received (client), INIT_RESPONSE sent (server)
- Unify all connection establishment messages under ETCP category
- Remove redundant DEBUG address prints in INIT send (info already in INFO msg)
- PING/PONG operations: BGP -> CONNECTION
- DIRECT IP detection: BGP -> NAT
2026-07-18 10:35:31 +03:00
Evgeny 3cd494bead chatgui: auto-clean build dir on generator change 2026-07-18 03:31:55 +03:00
Evgeny 28c788658a chatgui: Ninja + parallel build for Windows speedup 2026-07-18 03:30:40 +03:00
Evgeny 8f695cc828 AGENTS.md: add chatgui config and log file paths 2026-07-18 03:22:29 +03:00
Evgeny 3bf5a7b01d Revert hardcoded default for db_sync in chatgui — enabled via config file instead 2026-07-18 03:21:37 +03:00
Evgeny 2a826b62bc chatgui: enable db_sync=info debug category by default 2026-07-18 03:20:03 +03:00
Evgeny 4b7c7738de debug: add DEBUG_CATEGORY_DB_SYNC (26) + TRACE logs for db_sync, merkle_sync, chat_sync, member_sync 2026-07-18 03:19:46 +03:00
Evgeny 9e34e9f838 fix chatgui: self in member list, online status on reconnect, lb spam
- cs_handle_welcome(): add self to local peers_<ch_id> so
  joiner sees own account in channel member list
- cs_on_conn_up(): post CHANNEL_PEERS_ONLINE after reconnect
  so GUI shows connected status (was only posted on conn_down)
- etcp_loadbalancer: DEBUG_WARN -> DEBUG_DEBUG for no suitable
  link: tmr:wait — normal shaper behavior, not a warning
2026-07-18 02:55:10 +03:00
Evgeny 3e0b9483b7 fix chatgui: channel persistence & peer list after restart
- chat_core_sync_my_addresses(): remove private IP filter (192.168, 10.x, etc.)
  so LAN addresses are stored for direct connections
- topo_node_sqlite_channel_peers_all(): include peers without addresses
  in WELCOME message (was skipped entirely, causing empty peer list)
- chat_core_create_channel(): pass real local addresses to member_sync_put()
  instead of NULL,0
- chat_core_init(): log number of channels loaded from DB on startup
2026-07-18 02:46:53 +03:00
Evgeny 6354318255 fix: replace arpa/inet.h with platform_compat.h in chat_core.c too 2026-07-18 00:51:18 +03:00
Evgeny 93f265c0a9 fix: replace <arpa/inet.h> with platform_compat.h in chatgui (cross-platform ntohs) 2026-07-18 00:48:11 +03:00
Evgeny c594375023 fix: derive node_id from pubkey instead of privkey
sc_derive_node_id() hashed the private key, while invite links
(joindialog.cpp) hashed the public key — giving two different
node_ids for the same keypair.

Removed sc_derive_node_id(), kept only sc_derive_node_id_from_pubkey().
All callers updated to use pubkey-based derivation.
2026-07-18 00:41:38 +03:00
Evgeny 6f4025e420 refactor: move sqlite3 db from topo_group to UTUN_INSTANCE, remove conn_mgr/topo_group deps from chatgui
- topo_sqlite_db moved from TOPO_GROUPS to UTUN_INSTANCE (direct access)
- topo_groups_set_sqlite_db() removed
- chatgui: chat_sync_connect_node → chat_core_connect_auto (direct ETCP, no conn_mgr)
- chatgui: chat_core_connect_from_invite saves to SQLite → auto connect (no BGP)
- chatgui: removed _on_node_updated BGP callback from member_sync
- chatgui: removed conn_mgr_get_status from status dump
- chatgui: removed all conn_mgr.h and topo_group.h includes
- chatgui: CONN_MGR_ERR_* constants → CC_* in chat_core.h
- removed unused cc_parallel_* code, conn_state_str/conn_type_str helpers
2026-07-18 00:10:06 +03:00
Evgeny 52587c4f2b move ETCP_ID_NAT_DETECTION to etcp_api.h alongside other ETCP_IDs 2026-07-17 21:32:48 +03:00
Evgeny b238089678 refactor(nat_detection): remove BGP dependency, auto trigger on conn_up, clean API
- find_third_node: use nd->inst->connections instead of group->senders_list
- on_conn_up callback auto-calls trigger_checks (no BGP handshake needed)
- Remove group param from link_ready, request_check_all — nat_detection self-sufficient
- Remove trigger_checks + route_ping_cancel_for_conn from public API (internal only)
- topo_group.c: removed last nat_detection_trigger_checks call — completely decoupled
- Header: 4-line summary, no duplicate function list, 7 public functions
2026-07-17 21:23:40 +03:00
Evgeny 90363650e3 refactor(nat_detection): self-sufficient module with own ETCP_ID (0x02) and docs
- Assign ETCP_ID_NAT_DETECTION = 0x02, etcp_bind in create, etcp_unbind in destroy
- nat_detection_receive_cbk: own packet dispatcher (PING_REQ/RESP, NAT_INFO/CHECK_REQ)
- 4 handle_* become static internal, send_nat_check_req stays static (dead code)
- Rename TOPO_* -> NATDET_*: all subcommands, structs, message types
- Remove all NAT/PING constants and structs from topo_group.h
- Remove dispatch lines and subcmd_name cases from topo_group_receive_cbk
- API: clear sections (lifecycle/internal service/control/test helpers)
- trigger_checks: marked with IMPORTANT note about call order (must be after add_to_senders)
2026-07-17 20:45:49 +03:00
Evgeny 910cac8dce refactor: separate NAT detection from topology into standalone nat_detection module
- Extract NAT detection logic (~250 lines) from topo_group.c into new nat_detection.c
- NAT_DETECTION struct owns ping_pending, next_ping_req_id, allow_nat_check_local (removed from TOPO_GROUP)
- route_ping.* signatures changed: TOPO_GROUP* -> NAT_DETECTION* (+ TOPO_GROUP* where needed)
- topo_group.c now only dispatches BGP subcommands to nat_detection_* handlers
- UTUN_INSTANCE gets struct NAT_DETECTION* nat_det field (init/destroy in topo_groups_init/destroy)
- External callers (etcp_connections, etcp, control_server) call nat_detection_* directly
- nat_detection_cancel_for_conn / nat_detection_send_nat_info / nat_detection_link_ready / nat_detection_request_check_all
- route_ping_destroy_pending properly implemented (was dead code)
- TOPO_SUBCMD_PING_REQ/RESP moved from route_ping.h to topo_group.h (all BGP subcommands in one place)
- Tests updated: test_nat_detection, test_route_ping - both PASSED
- Build: auto(CORE_SOURCES+libutun) and cmake(chatgui/libutun)
2026-07-17 19:49:54 +03:00
Evgeny 8180676120 fix: peers table schema mismatch causing silent member write failures
chat_core_ensure_channel_ready() created peers_<ch_id> with 7 columns
while topo_node_sqlite_member_put() INSERT expected 9 (name, creator_sig).
Fixed schema alignment + reordered calls in create_channel() so
topo_node_sqlite_channel_put() (full 11-column schema) runs first.
Added error logging to all 6 member_put/member_sync_put call sites.
2026-07-17 18:16:21 +03:00
Evgeny 0381f4df89 Убраны TCP_TIMESTAMPS и TCP_KEEPALIVE из lwip_tcp_opts.h (не используются, конфликтовали с ws2ipdef.h на Windows) 2026-07-17 17:49:15 +03:00
Evgeny 706ef5b472 etcp_connections.h: data[0] -> data[] (C99 flexible array, fix -Wstringop-overflow) 2026-07-17 17:40:25 +03:00
Evgeny 3884af7eb8 ntp_time.h: replace netinet/in.h with platform_compat.h for Windows build 2026-07-17 17:36:23 +03:00
Evgeny 960cde3eda chatgui: add Status page in settings (NTP + connections status) 2026-07-17 16:25:55 +03:00
Evgeny f0bb03c2cf db_sync: datahash removed, Ed25519 signature mandatory, PK=(timestamp,author_signature), chain_hash includes signature, signed message=ts||json 2026-07-17 15:44:19 +03:00
Evgeny 05b24d36bb db_sync: fix dh_match premature sync complete — send tail when mc > tp+1
Root cause: db_handle_init_resp marked sync_state=2 when my_dh==peer_dh
without checking that all my records fit in the confirmed prefix.
If mc > tp+1 (I have records beyond the common prefix), the tail was
never sent — lost records.

Fix:
- Remove special-case 'sc==0 && my_dh==peer_dh' (redundant, same bug)
- In general dh_match: if mc > tp+1, send remaining records [tp+1,mc-1]
  using the same inline loop pattern as 'peer empty' branch

Test: 3 phases covering all dh_match paths:
- Phase 1: B empty+late instance → dh_match at tp=0 (sc=0) → tail-send 49
- Phase 2: A=80 B=50 → dh_match at tp=49 (sc>0, sparse) → tail-send 30
- Phase 3: fresh instances → peer empty → send all 30
2026-07-17 11:48:56 +03:00
Evgeny 2792586de7 db_sync: add INFO diagnostics for message sync flow tracing
- db_sync_on_conn_up: per-instance synced/skipped counters
- db_sync_peer_check_cb: summary INFO of check results
- db_sync_instance_add: peers_found + peers_synced counts
- db_handle_init_sync/initiate_sync: enhanced with tbl name
- db_handle_init_resp: tp + my_mc in sync complete/dh match
- db_handle_send_data: range + SYNC_DONE details; stop reason when mc>pk
- db_handle_sync_done: both counts + datahashes in mismatch; dh in confirm
- db_sync_recv_cb: INFO when instance NOT FOUND (timing race)
- cs_handle_channel_info_resp: sync readiness after channel ready
- cs_handle_welcome: note about db_sync peer_check timer
- on_db_sync_insert: counter (#1-3 then every 10) with ch/n/ts/dh/ct
2026-07-17 00:13:27 +03:00
Evgeny db67767405 fix: demote 6 crypto diagnostic logs from INFO to DEBUG
ECDH, ENCRYPT, DECRYPT, INIT_SEND, NORM_DECRYPT_TRY, INIT_DECRYPT_TRY
all moved to DEBUG_DEBUG — visible only with debug=crypto=debug
2026-07-16 23:35:08 +03:00
Evgeny 4fa5b0361f feat: show decoded invite details (node id, address:port) in JoinDialog
Auto-decodes invite link with 400ms debounce, shows 'Node: 0x... | IP:port'
2026-07-16 23:22:32 +03:00
Evgeny 55e3a7af94 fix: getInviteAddresses — use active direct-IP connections instead of DB peer list
Priority: iterate instance->connections for links with NAT_TYPE_DIRECT
(connected and reachable). Fallback: own socket addresses.
Removed broken getOnlinePeerAddresses() which returned other nodes' addresses.
2026-07-16 23:20:26 +03:00
Evgeny 2aa559acaf debug: add full ECDH/session-key diagnostics (6 points)
K1: sc_set_peer_public_key — priv, peer_pub, shared_secret, session_key
K2: sc_encrypt — session_key, tx_counter, nonce, plaintext
K3: sc_decrypt — session_key, nonce from packet, ciphertext_len
K4: INIT_SEND — salt, obfuscated_pubkey, peer_pubkey, my_pubkey, session_key
K5: NORM_DECRYPT_TRY — session_key, recv_len, rx_counter
K6: INIT_DECRYPT_TRY — session_key, recv_len, recv_len-40, salt, encrypted_pubkey

All via DEBUG_INFO(DEBUG_CATEGORY_CRYPTO) — enable with debug=crypto=info
2026-07-16 22:13:35 +03:00
Evgeny cdcc32cf84 fix: move DECRYPT FAIL/collision/reinit logs from DEBUG_CATEGORY_DEBUG to proper categories
- DECRYPT FAIL (1554): DEBUG_DEBUG → ETCP (main failure signal)
- not an init packet (1623): DEBUG_DEBUG → CONNECTION (was invisible ERROR)
- pkt_pool exhaust (1537): add ERROR log (was silent)
- REINIT/collision events (1473,1751-1838): DEBUG_DEBUG → CONNECTION
2026-07-16 22:03:33 +03:00
Evgeny 95061d3723 fix: demote INIT retry spam (link/send logs) to DEBUG_DEBUG
Keep at INFO only 'Sending INIT request' line per retry.
2026-07-16 21:41:50 +03:00
Evgeny 5b88d58dae fix: demote spam auto_connect logs to DEBUG_DEBUG
Remain at INFO: attempt + result FAIL. Demoted to DEBUG: TIMEOUT,
CANCELLED, GC closing, flight launched, AUTO_CONNECT in_progress.
2026-07-16 21:11:53 +03:00
Evgeny f039031883 fix: check conn_queue_entry->peer_node_id instead of conn->peer_node_id
etcp_conn_set_peer_node_id at state=0 updates conn->peer_node_id but
does not reindex the queue entry (key stays 0). Check the queue entry
directly to detect unindexed outbound connections.
2026-07-16 21:05:08 +03:00
Evgeny c3354938e1 fix: peers_* table creation + GUI_ERROR to log + cross-connect conn reuse
- chat_core.c: CREATE TABLE IF NOT EXISTS peers_* in ensure_channel_ready
- utun_node.cpp: GUI_ERROR redirected to debug log file via guiDebugFile()
- etcp_connections.c: reuse unindexed outbound conn (peer_node_id==0)
  on cross-connect INIT to avoid 'Can not insert link to socket'
2026-07-16 20:51:24 +03:00
Evgeny abee0ef3a9 chatgui: add --asan flag to build_linux.sh 2026-07-16 19:33:52 +03:00
Evgeny f3ac1cbb06 chat_sync: fix double-free in auto_connect — clear ca_state in ac_result_cb on success 2026-07-16 19:24:13 +03:00
Evgeny bfa3e666d4 fix test_uasync_socket_race: increase listen() backlog from 32 to 1024
Root cause: accept queue overflow (backlog=32) caused kernel SYN drops.
Children waited ~1s for SYN retransmit timeout (5 gaps × 1s = 5s out of 7.2s).
With backlog=1024 the timeout drops from ~7.2s to ~0.065s (110× faster).
2026-07-16 19:22:06 +03:00
Evgeny a8ce7d47b7 test_bgp_triangle: reduce keepalive_timeout from 2000ms to 200ms (10x speedup, 13.2s -> 1.27s) 2026-07-16 19:13:05 +03:00
Evgeny 3d04c37f56 stcp: add on_close callback to stcp_server_create/stcp_client_connect; member_sync: add join_ts/update_sig/update_ts fields, verify update_sig in apply; topo_node_sqlite: add join_ts/update_sig/update_ts to member_put, add member_get_join, add node_update_verified; chat_sync: include join_ts in join_msg, add update_sig/update_ts to CHANNEL_INFO_RESP; WELCOME verify join_sig; tests: use generated node_ids, reduce max ticks 2026-07-16 19:04:00 +03:00
Evgeny 0137a1bad4 chatgui: preserve input focus on chat refresh (save/restore focusWidget in loadChannel) 2026-07-16 13:03:23 +03:00
Evgeny 51d47ea6a8 chatgui: restore input focus after sending message 2026-07-16 12:54:34 +03:00
Evgeny c4d1f984bd chatgui settings: add Database tab with integrity check (quick/full) and optimize 2026-07-16 12:46:40 +03:00
Evgeny 03310db2cd test_chat_sync_stress: simplify to 20 batches, add profiling (avg per-record time, sync time, total) 2026-07-16 12:45:28 +03:00
Evgeny bb425c6722 db_sync v2: positional sync per-peer (synced_pos), datahash instead of chain_hash, no cascade on PUSH, single cascade after SEND_DATA batch, startup chain verification. PRAGMA synchronous=NORMAL + wal_autocheckpoint=10000 for performance. 2026-07-16 12:31:52 +03:00
Evgeny 0479c309bb Диагностика синхронизации: merkle_sync DEBUG_DEBUG логи обмена; убран спам auto_connect no IPv4; сохранение адреса инвайтера из ETCP; детали DECRYPT FAIL (link_state/enc_errs) 2026-07-16 00:41:51 +03:00
Evgeny 0d5a72c40e chat_sync: node_name теперь в пакетах — CHANNEL_INFO_RESP/JOIN/PEER_UPSERT передают имя, верификация использует имя из пакета (не из БД) 2026-07-16 00:32:09 +03:00
Evgeny 3f0fdad754 config_updater: поддержка создания конфига с нуля (без файла); новый файл — собирается с нуля, существующий — insert_or_replace 2026-07-15 23:47:00 +03:00
Evgeny e325b84a8e config: одно открытие файла — parse_config_from_buf вместо повторного fopen; убран read_file_to_mem 2026-07-15 23:39:01 +03:00
Evgeny ef24af7d24 Ключи: node_id из SHA-256(privkey) через весь конфиг; приоритет privkey — pubkey/node_id всегда выводятся из него; CHANNEL_INFO_RESP: верификация inviter_join_sig через ETCP-ключи 2026-07-15 23:13:29 +03:00
Evgeny da86841516 Диагностика: WARN при мутации конфига (ключи/node_id) + детали при invalid sig в JOIN/CHANNEL_INFO_RESP + предупреждение при расхождении node_id инвайта и ETCP 2026-07-15 22:44:33 +03:00
Evgeny 895eb0abbf Убрал спам-логи ETCP SEND/RECV/DECRYPT OK + добавил диагностику синхронизации мемберов (DEBUG_CATEGORY_DEBUG) в ключевые точки topo_group, merkle_sync, member_sync, chat_sync 2026-07-15 22:25:19 +03:00
Evgeny e708940bcd debug: upgrade router_deliver to DEBUG_INFO, show cb ptr
Router deliver now visible without trace — shows svc_id, len, remote_node_id, conn ptr, callback ptr
This will confirm whether CHANNEL_INFO_REQ reaches chat_sync on the server side
2026-07-15 22:06:54 +03:00
Evgeny d161782b46 fix: merkle_sync.c linked-list -> queue lookup, upgrade debug logs to INFO
- merkle_sync.c ms_find_conn_for_node: replace c=c->next with queue_find_data_by_index
- chat_sync.c cs_find_conn_for_node, cs_on_conn_up: DEBUG_DEBUG -> DEBUG_INFO
- etcp.c queue_set_ready: DEBUG_DEBUG -> DEBUG_INFO
2026-07-15 21:59:29 +03:00
Evgeny d71753fa79 debug: add diagnostic logs for chat invite connect flow
- cs_find_conn_for_node: show queue state, entry found/missing, conn flags
- cs_on_conn_up: log when skipping due to !initialized
- etcp_conn_queue_set_ready: log reindex details (old/new entry, count)
- etcp_conn_set_peer_node_id: show state+reindexed flag
2026-07-15 21:42:16 +03:00
Evgeny a7ecee01f5 etcp: replace connections linked-list with indexed ll_queue, add state/ref_count lifecycle
- Remove ETCP_CONN.next linked-list, UTUN_INSTANCE.connections list/count
- Unified connections in single ll_queue indexed by peer_node_id (0=pending)
- struct conn_queue_entry { peer_node_id, conn* } for uniform queue data
- Connection lifecycle via etcp.c: create→pending, conn_ready→indexed, close→detach+deferred free
- Close split: phase1 detach from external world, phase2 deferred via uasync_call_soon
- etcp_conn_set_peer_node_id: reindex on ID change (WARN)
- etcp_conn_ref_take/free: blocking close until refcount returns to 0
- Remove chat_connections — use unified connections queue directly
- Update all iteration/search patterns across 40 files (src + tests + chatgui)
2026-07-15 19:56:47 +03:00
Evgeny 6902f5fd95 chat: add ll_queue with index for P2P connections (replaces linear search)
- utun_instance.h: add chat_connections queue (indexed by node_id, 8 bytes)
- chat_sync.c: queue_new in init, queue_free in destroy, queue_find_data_by_index in cs_find_conn_for_node
- chat_sync.c: remove from queue in cs_on_conn_down
- chat_core.c: add to queue in connect_from_invite and connect_auto
- chat_core.c: remove from queue in cc_parallel_cleanup and ca_cleanup
- db_sync.c: replace linear conn search with queue_find_data_by_index (db_sync_send_hash)
2026-07-15 18:15:16 +03:00
Evgeny d0af503d01 config: fix allow_all parsing — accept yes/1/true like other bool options
- config_parser.c: allowed_keys_allow_all now uses strcasecmp(yes/1/true) instead of atoi()
- node_config.cpp: keep allow_all=yes
2026-07-15 18:01:54 +03:00
Evgeny 3368abad2a etcp+chat: add DEBUG_INFO logs for invite/crypto handshake debugging
- etcp_connections: DECRYPT FAIL now logs my_pub/peer_pub/session_ready at INFO level (debug category)
- chat_sync: log invite start with pubkey/node_id/addr_count
- chat_core: log pubkey set on invite connect (my_pub + peer_pub)
- joindialog: GUI_INFO on decoded invite
2026-07-15 17:53:25 +03:00
Evgeny d83d281fa0 config: add [allowed_keys] allow_all=yes by default, add [gui] as valid section
- node_config.cpp: add [allowed_keys] allow_all=yes to default config
- config_parser.c: add SECTION_GUI to suppress 'Key outside section' warnings
2026-07-15 17:34:48 +03:00
Evgeny 4c8e5f6223 db_sync+chat_sync+etcp: fix conn init race, reduce keepalive log spam
- etcp_connections: keepalive SEND/RECV/DECRYPT_OK: DEBUG→TRACE; DECRYPT_FAIL: WARN→DEBUG
- db_sync_on_conn_up: skip sync if !conn->initialized, wait for next UP event
- db_sync peer_check_timer+instance_add: also check conn->initialized
- chat_sync cs_on_conn_up: ignore conn_up before initialized (cs_send fails otherwise)
- db_sync_on_conn_up log now includes init= and links=
2026-07-15 17:25:14 +03:00
Evgeny 5b749d6cc8 chatgui: db_path is now a directory (chat_data/) with chats.db and sync/ inside
- Default db_path changed from 'chats.db' to 'chat_data' directory
- chat_core_init receives <db_path>/chats.db
- db_sync receives <db_path> (creates <db_path>/sync internally)
- Directory auto-created on startup via QDir::mkpath
2026-07-15 17:09:11 +03:00
Evgeny 6252556a02 chatgui: migrate message sync from chat_sync to db_sync
- Replace db_sync_stub.c with real src/db_sync.c in libutun
- Remove message sync protocol from chat_sync (INIT_SYNC/INIT_RESP/SEND_DATA/PUSH/ACK_PUSH/SYNC_DONE)
- Keep chat_sync auxiliary: join protocol, peer management, auto-connect
- Remove chat_core_insert_record, cursor_*, mark_sent/ttl_delete stubs
- chat_core on_db_sync_insert now only does SQLite INSERT + GUI notify
- Move db_sync_init() from instance_init_common to utun_instance_init for config override timing
- Force db_sync_enabled=1 in utun_node.cpp
- db_sync handles all P2P message sync via service 0x20
2026-07-15 17:02:47 +03:00
Evgeny e86944d661 style: reformat db_sync.c — break 500-char lines into readable ~130-char max 2026-07-15 16:32:36 +03:00
Evgeny acaa3a767c refactor: move chat_sync and member_sync to direct ETCP P2P, bypass etcp_router
Replace etcp_router_bind/etcp_route_send/etcp_router_unbind with
direct etcp_bind/etcp_send/etcp_unbind + find-conn-by-node_id helpers.
Same approach as db_sync (0x20). Service IDs unchanged (0x30, 0x31).
2026-07-15 16:13:45 +03:00
Evgeny d840b458a7 fix: pause router retransmits before clearing ETCP queues on reinit
When etcp_conn_reinit is called, it clears ETCP queues (input_wait_ack
etc.) which frees dgram objects to data_pool. If a router retransmit
timer fires concurrently, it sends through the same conn → normalizer
allocates from corrupted data_pool → SIGSEGV.

Fix: call etcp_router_pause_retrans_for_node() BEFORE etcp_conn_reset()
to cancel retrans timers and free inflight_q entries for all router
connections to the peer. No callbacks, no close notifications —
lightweight pause, router connections stay alive and resume when
ETCP connection comes back up.
2026-07-15 15:37:03 +03:00
Evgeny 9378b18ce7 fix: defer connection_close in ready_cb using uasync_call_soon
ca_ready_cb and cc_parallel_ready_cb called etcp_connection_close
synchronously on non-selected connections from within the ready_cbk
callback chain. This corrupts senders_list in topo_group, causing
UAF in topo_group_start_link_nat_check.

Now uses uasync_call_soon (zero-delay timer) to defer the close
to the next event loop iteration, after the callback chain completes
and callbacks_running flag is reset.
2026-07-15 15:00:05 +03:00
Evgeny f669254da3 feat: hard crash on etcp_connection_close from callback chain
Added callbacks_running flag to ETCP_CONN:
- set to 1 before up_cbks / down_cbks / ready_cbks iteration
- reset to 0 after iteration completes
- etcp_connection_close checks flag and does intentional SIGSEGV
  with FATAL log message if called during callback chain

This catches illegal synchronous close from within callbacks
instead of producing cryptic UAF segfaults later.
2026-07-15 14:43:47 +03:00
Evgeny 371208d489 chatgui: выпилить все записи в БД из GUI потока (DbManager)
- DbManager теперь read-only, получает sqlite3* handle через setDb()
- chat_core — единственный владелец БД (SQLITE_OPEN_FULLMUTEX + WAL)
- initTables/seedStubData/createChannel/insertMessage/setUiState удалены из DbManager
- seed 5 stub-аккаунтов перенесён в chat_core_init()
- GUI_EVT_DB_READY — уведомление о готовности БД
- setUiState заменён на QSettings
- каналы загружаются по DB_READY коллбэку
- убран двойной sqlite3_open (один файл, один хендл)
2026-07-15 14:26:32 +03:00
Evgeny 409a99f56b feat: cross-connect collision resolution via is_server master + collision flag
Protocol: added collision field to INIT_REQUEST_PKT (1 byte before ed25519_pubkey).
When two peers connect simultaneously, the one with is_server=0 (API-created
outbound link) is the master. Resolution by smaller node_id:
- We have master link + smaller node_id → send our INIT with collision=1
- We have master link + larger node_id → yield, process as slave
- Remote sends collision=1 → become slave unconditionally
- session_id change always forces reinit (genuine restart)

Removed unconditional etcp_conn_reinit on INIT_RESPONSE(0x03) — guarded by
reset_done flag. Server-side reinit now has collision check BEFORE reset_done
guard; session_id change penetrates reset_done protection.

Closes cross-connect reinit loop causing endless UP/DOWN flapping.
2026-07-15 13:53:15 +03:00
Evgeny 0912499309 fix: segfault in callback chain iteration — save next before invoke
In etcp_conn_ready / etcp_on_up / etcp_on_down / etcp_connection_create /
tcp_server_on_link, callback chains were iterated with:
  while (cbe) { cbe->fn(...); cbe = cbe->next; }
If the callback removes itself from the chain (e.g. ca_ready_cb calls
etcp_conn_remove_ready_cbk which u_free's the entry), cbe->next reads
freed memory → SIGSEGV.

Fixed by saving next pointer before invoking the callback:
  while (cbe) { n = cbe->next; cbe->fn(...); cbe = n; }
2026-07-15 12:46:37 +03:00
Evgeny abbb6de701 fix: block redundant conn reinit on cross-connect INIT (reset_done flag)
Problem: when two peers connect simultaneously, each incoming
INIT_REQUEST(0x02) or INIT_RESPONSE(0x03) triggers etcp_conn_reinit
unconditionally, causing endless UP/DOWN flapping loop.

Fix: add reset_done flag to ETCP_CONN:
- 0 at creation and after explicit reinit (reinit allowed)
- set to 1 in etcp_conn_ready (connection stable, block reinit)

Three call sites guarded with !conn->reset_done:
- client: handle_init_response_client (INIT_RESPONSE 0x03)
- server: existing link INIT_REQUEST processing
- server: new link INIT_REQUEST processing

send_reset logic preserved unconditionally — only etcp_conn_reinit
itself is blocked when already stable.
2026-07-15 12:17:14 +03:00
Evgeny 2bf7dcc6f0 debug: comprehensive connection diagnostics — recv/send/reinit paths
- RECV: log every packet (src addr, link found, session_ready, decrypt result)
- RECV: log init decrypt rejection with packet code (catches 0x03 INIT_RESPONSE
  silently dropped by init path)
- SEND: log destination addr+fd in send_init_response and etcp_encrypt_send
- REINIT: log full reason in server and client paths (code, session, got_init,
  initialized, links_up state before reinit)
2026-07-15 11:46:44 +03:00
Evgeny f7d88e5532 fix segfault in ca_ready_cb: ca_state owns all ca_ctx, single cleanup point
- Added struct ca_ctx** ctxs to ca_state (parallel array)
- ca_cleanup now frees all ctxs[i] and the ctxs array
- ca_ready_cb: removed u_free(ctx), added etcp_conn_remove_ready_cbk before cleanup
- ca_timeout_cb: removed u_free(ctx) — ctx lives until ca_cleanup
- All error paths set ctxs[i]=NULL for ca_cleanup safety
2026-07-15 11:21:31 +03:00
Evgeny 1e7585438d chat_core_connect_auto: add outbound conn to inst->connections
etcp_connection_create does NOT add conn to inst->connections list.
Without this, incoming INIT handler can't find the outbound conn
by peer_node_id and creates a new one, causing insert_link collision.
2026-07-15 11:05:27 +03:00
Evgeny a8164baf41 chat_core_connect_auto: set peer_node_id on ETCP_CONN
Without peer_node_id, incoming INIT handler can't find the existing
outbound connection, creates a new one, and fails on insert_link collision.
2026-07-15 10:51:53 +03:00
Evgeny fc6a873b5c etcp_connections: reuse existing outbound link for incoming INIT by addr
When two peers send INIT simultaneously, the first link occupies the socket.
Incoming INIT arriving after finds the existing outbound link by (ip,port)
instead of failing with 'Can not insert link to socket'.
2026-07-15 10:46:33 +03:00
Evgeny 0adf7d07ef chat_core_connect_auto: dedup addresses by (addr,port) 2026-07-15 10:32:06 +03:00
Evgeny 33aa6bbd5a auto-connect: cursor-based, no close on timeout, GC every 1s, cancel API
- ca_timeout_cb: removed etcp_connection_close, ETCP lives past 3s timeout
- ca_state.cancelled flag + chat_core_connect_auto_cancel() for external cancel
- chat_core_connect_auto returns ca_state* via out_state
- Rewritten auto_connect: cursor (ch,peer) instead of bulk collect
- GC every 1s: closes flights with no link_status after 3s
- ac_result_cb no longer clears ca_state (GC handles cleanup)
- stop: cancels all flights via chat_core_connect_auto_cancel
2026-07-15 10:24:44 +03:00
Evgeny dccf54b30b auto-connect + GUI per-channel indicators
- chat_core_connect_auto: direct ETCP connect from SQLite (no BGP)
- auto_connect: parallel connect to peers with 3s timeout, retry 10s
- RTT saved to node_addresses on disconnect
- vertical status bar + circle badge with online count per channel
- ChannelListView prevents deselection
2026-07-15 06:00:43 +03:00
Evgeny 1a0229fd4f add NTP time synchronization module 2026-07-14 20:14:00 +03:00
Evgeny f3719a28df fix: change timestamp unit from microseconds to milliseconds
- db_sync.c: last_timestamp_us → last_timestamp_ms, use tv_sec*1000 + tv_usec/1000
- db_sync_stub.c: same change
- QDateTime::fromMSecsSinceEpoch() in chatgui expects milliseconds
- 1 µs → 1 ms change fixes 'minutes ticking like seconds' display bug
2026-07-14 19:33:16 +03:00
Evgeny 5fc621690a fix: wall-clock timestamps + merkle_sync timeout + build script
- Replace get_time_us() with utun_gettimeofday() for PUSH message timestamps
  (db_sync.c, chat_core.c) — monotonic clocks differ across Linux/Windows
- Fix merkle_sync: restart session timer in _handle_request to prevent
  double-sided timeout when both nodes start member_sync simultaneously
- Fix INIT_RESP short-format: don't mark synced when my_count > peer_count
- Add db_sync_get_last_timestamp() to avoid duplicate timestamp generation
  between db_sync and chat_core on_db_sync_insert
- Fix db_sync_stub to generate timestamps consistently
- build.sh: auto-rebuild chatgui via CMake after utun build
2026-07-14 19:23:02 +03:00
Evgeny 4636a69225 fix chatgui: pass chain_hash in PUSH + start member_sync on server join
chat_sync_push: add chain_hash parameter, use it instead of zeros
  (fixes chain_hash mismatch → all PUSH messages were rejected)

on_db_sync_insert: pass computed chain_h to chat_sync_push

cs_handle_channel_join: call member_sync_start for the joiner
  (server now initiates member sync when client joins.
  Previously merkle_sync was only started by client → timeout
  because server never responded)

cs_handle_welcome: add DEBUG_INFO before member_sync_start on client side
2026-07-14 18:43:04 +03:00
Evgeny f9f516c2e9 fix chatgui: PUSH new messages to peers + comprehensive debug logging
chat_sync_push: fix PUSH format to match cs_handle_push expectations
  (was missing insert_record header: ts+dh+nid before ct+dlen+data)
  Add DEBUG_INFO log with peer count after send.

on_db_sync_insert: call chat_sync_push after successful local insert
  → new messages are now pushed to all connected peers
  Add DEBUG_ERROR to 4 silent returns (JSON parse fail, SQL errors)

chat_core_submit_message: add DEBUG_INFO entry log (ch, ct, len)

chat_core_insert_record: add DEBUG_WARN/ERROR to 7 silent returns
  (truncated record, SQL prepare fail, unknown step error)

cs_handle_push: add DEBUG_INFO for ACK send, DEBUG_ERROR for insert fail

cs_handle_send_data: DEBUG_WARN when record truncated mid-loop

cs_on_conn_up: DEBUG_INFO sync path with channel count

Full trace now shows every message: SUBMIT → insert → PUSH → RECV PUSH → ACK
2026-07-14 18:34:24 +03:00
Evgeny 58b9f28149 fix chatgui: prevent invite deadlock on connection flap
cs_on_conn_down: remove invite state zeroing on disconnection.
Only timeout callbacks and explicit protocol end (CS_MSG_ERROR,
transition to JOIN phase) clear the invite state.

Connection flap now causes automatic retry:
  conn_up → pending_invite_ch_id != 0 && !info_req_timer → resend
  CHANNEL_INFO_REQ. No deadlock possible.
2026-07-14 18:22:05 +03:00
Evgeny 8c41b44718 fix chatgui: repair INIT_RESP handler, add debug logging to all sync handlers
- cs_handle_init_resp: change silent reject len<37→len<5 (actual payload
  sent is 6 bytes, plen=5 after stripping type byte).
  Support short format (synced, peer_count+sparse_count) and long format
  (divergence, with test_pos+peer_ch). Add INFO logs at each decision.
  (fixes sync never starting after invite join — RECV INIT_RESP len=5
  was silently rejected)

- All 12 silent return branches in handlers: add DEBUG_WARN with
  handler name, peer, ch_id, and reason (makes malformed packets
  visible in logs)

- Key sync flow INFO logs:
  cs_handle_init_sync: peer_count, my_count
  cs_handle_send_data: records sent/received count
  cs_on_conn_up: invite vs sync path
  chat_core_insert_record: successful insert
2026-07-14 18:07:57 +03:00
Evgeny 2d179ec41d fix chatgui: message format, member notify, chain_hash sync
- on_db_sync_insert: parse JSON {"n":...,"ct":...,"d":...}, store
  d→data, n→node_id, ct→content_type with proper chain_hash
  (fixes JSON displayed instead of message text)
- chat_core_cursor_open: SELECT chain_hash column
- chat_core_cursor_next: include chain_hash(32) in output record
- cs_handle_send_data: parse chain_hash in received records
  (fixes chain_hash mismatch → all sync records were dropped)
- cs_handle_channel_join: post GUI_EVT_MEMBERS_CHANGED
  (fixes creator not seeing new member after join)
- cs_handle_welcome: trigger INIT_SYNC with inviter after join
  (fixes no message sync after invite join)
2026-07-14 17:46:33 +03:00
Evgeny c4751557b1 fix chatgui: ensure msg table + db_sync instance on join and restart
- chat_core_ensure_channel_ready(ch_id): idempotent — creates
  msg_<id> table + registers db_sync instance if missing
- chat_core_create_channel: refactored to use ensure_channel_ready
- cs_handle_channel_info_resp: call ensure_channel_ready on join
- chat_core_init: iterate existing channels on restart
- Fixes 'no such table: msg_<id>' after invite join
- Fixes 'no db_sync instance for ch=<id>' after restart
2026-07-14 17:22:01 +03:00
Evgeny 4254ab97bb fix chatgui: db_sync_stub + chat_sync_init for invite link join
- db_sync_stub.c: complete rewrite — proper structs, all API functions
  (db_sync_instance_add, insert_signed, set_insert_cb, select)
  Fixes db_sync_instance_add returning NULL (inst->db_sync was NULL)
- libutun/CMakeLists.txt: remove src/db_sync.c, stub replaces it
- chatgui/CMakeLists.txt: add transport/db_sync_stub.c
- utun_node.cpp: remove redundant db_sync_init/destroy,
  add chat_sync_init/destroy — server now registers svc_id=48
  Fixes 'no handler for svc_id=48' on incoming CHANNEL_INFO_REQ
2026-07-14 17:03:00 +03:00
Evgeny ccd25813f6 refactor: extract merkle_sync — universal Merkle-tree sync module
- merkle_sync.h/c: tree (5-level prefix, SHA256 buckets), wire protocol
  (HASHES/REQUEST/BATCH), sessions (10s timeout, 3 retries), async API
  with done_cb
- member_sync: thin glue layer, implements 3 data model callbacks
  (update_bucket_hash, get_items, apply_items) for chat members
- chat_sync: updated member_sync_start() calls with done_cb callback
- member_tree_hash table renamed to merkle_tree_hash
- member_sync.h public API preserved, internal tree/wire logic removed
2026-07-14 16:35:58 +03:00
Evgeny f617dcd68e db_sync: LMDB→SQLite, multi-instance, etcp_api, chatgui integration
- src/db_sync: LMDB replaced with SQLite (separate tables per instance: db_sync_name_id)
  TTL index (author, timestamp), WAL mode, per-instance peers/timers
  opaque DB_SYNC_INSTANCE* handle, insert callback, select API
  new fields: author_signature, delivered_peers, delivery_chain
  transport: etcp_api direct P2P (was etcp_router)

- src/db_sync.h: user-oriented API docs, multi-instance lifecycle

- lib/Makefile.am: liblmdb sources removed
- lib/sqlite3.c+h: SQLite amalgamation added

- topo_node: LMDB→SQLite (src/topo_node_sqlite.c moved from chatgui)

- chatgui: real db_sync.c replaces db_sync_stub.c
  chat_core: db_sync_insert_signed + dual-write insert callback
  utun_node: db_sync_init/destroy replaces chat_sync_init/destroy
  CMakeLists: stubs removed, real db_sync.c added
2026-07-14 14:17:43 +03:00
Evgeny 37417ba49d Fix: my_node_name insert inside [global] section, add button tooltips in channellist 2026-07-14 09:17:01 +03:00
Evgeny cca3511503 fix: replace deprecated SHA256 API with EVP, conditional dl linkage for Windows 2026-07-14 09:07:03 +03:00
Evgeny d70cdbb040 chatgui: preserve active channel selection on reload, members sync, profile settings, member list model 2026-07-14 08:52:10 +03:00
Evgeny c3f946fb63 chat_sync: fix join group stall — remove node_id match in cs_on_conn_up, add CS_MSG_ERROR protocol response 2026-07-13 18:11:18 +03:00
jeka 93de0aa591 agents 2026-07-13 15:46:29 +03:00
Evgeny 0c2eeb8820 chatgui: validate config options on startup, refuse to start on errors 2026-07-13 15:46:20 +03:00
Evgeny 637b21da9e debug: remove console_level/file_level — simplified output routing
Remove secondary level filtering in debug_output(). Now the only filter
is debug_should_output() (per-category + global fallback). Output goes
to file (if file_output set) or console (if console_enabled), with no
additional level check.

Removed:
- debug_set_console_level(), debug_set_file_level()
- console_level, file_level fields from debug_config_t
- debug_set_level() no longer sets console_level
2026-07-13 15:38:08 +03:00
Evgeny c0935aba6c docs: update AGENTS.md and etcp_arch.md 2026-07-13 15:21:30 +03:00
Evgeny 6bf267924d add DEBUG_CATEGORY_CONNECTIVITY for handshake/protocol lifecycle logs
New category (id=25) replaces DEBUG_CATEGORY_DEBUG in chat_sync and
chat_core. This allows enabling only protocol-level logs (SEND/RECV,
timeouts, handshake) without the noise from DEBUG category.

Changes:
- debug_config.h: add CONNECTIVITY=25, COUNT→26
- debug_config.c: add 'connectivity' to g_debug_categories
- chat_sync.c: DEBUG→CONNECTIVITY in all protocol logs
- chat_core.c: DEBUG→CONNECTIVITY in connection/create logs
- utun_node.cpp: remove hardcoded DEBUG=TRACE (now config-driven)
- chatgui.cfg: connectivity=info in debug_categories
2026-07-13 15:14:48 +03:00
Evgeny 5318476cdf chat_sync: add protocol request timeouts and SEND/RECV debug logs
- Add info_req_timer (5s) for CHANNEL_INFO_REQ — reports CONN_MGR_ERR_TIMEOUT
  to GUI on timeout, clears pending invite state
- Add join_timer (8s) for CHANNEL_JOIN → WELCOME — reports timeout on failure
- cs_on_conn_down: cancel all proto timers, reset pending invite state
- cs_send: log every outgoing message (SEND <type> to=<peer> ch=<id> len=<n>)
- Dispatcher: log every incoming message (RECV <type> from=<peer> ch=<id> len=<n>)
- Add cs_msg_name() helper for readable message type names
- pending_invite_node_id/ch_id no longer cleared immediately after send —
  kept until response or timeout, enabling proper error reporting
2026-07-13 15:06:06 +03:00
Evgeny 24294b22f6 fix: use got_initial_pkt instead of new_conn for send_reset decision
new_conn is only true for the first INIT after server restart.
Subsequent INITs from same peer (recovery loop) see new_conn=0
and revert to old logic, sending 0x05 (no reset) and overwriting
the first 0x03 response.

got_initial_pkt stays 0 until client actually sends seq=1, so
server keeps sending 0x03 across all INIT retries until client
resets and starts from seq=1.
2026-07-13 14:32:16 +03:00
Evgeny 43d95016ae fix: send INIT_RESPONSE with reset when server creates new connection after restart
When server restarts, new ETCP_CONN has session_id=0 (calloc).
If client sends INIT_REQUEST_NOINIT (0x04) with session_id=0,
the condition conn->session_id != session_id is false (both 0),
so no reset was sent. Client kept old seq numbers while server
expected seq=1, causing 'Waiting for initial packet' deadlock.

Fix: add || new_conn to the condition — when server created a
fresh connection (was restarted), always send reset to client.
2026-07-13 14:19:27 +03:00
Evgeny 41e0736b75 etcp_connections: force reinit when client sends ETCP_INIT_REQUEST (reset)
Server now checks both code==ETCP_INIT_REQUEST and session_id mismatch
to decide send_reset. Previously only session_id was checked, so if
session matched (both sides had 00000000), reinit was skipped even
when client explicitly requested reset (code 0x02).

Client side already correct: handle_init_response_client:1473 calls
etcp_conn_reinit if pkt_code==ETCP_INIT_RESPONSE (0x03).
2026-07-13 13:29:58 +03:00
Evgeny 1d29cab751 etcp: INFO-level TX DATA log + Initial packet received log
- 1133: DEBUG_DEBUG -> DEBUG_INFO for TX DATA (seq, len, retry, queue lengths)
- 1437: add DEBUG_INFO when initial packet seq=1 accepted

These two INFO lines show the exact order of first sends and receives
to diagnose INIT sequencer sync without guessing.
2026-07-13 13:19:08 +03:00
Evgeny d0f5a2970f etcp_connections: improve INIT diagnostics (demote spam, add type/source/fields)
- 1556: Normal decryption failed -> DEBUG_DEBUG (expected for INIT, no more spam)
- 1581: added 'INIT X25519 OK from %s' with source address
- 1604-1617: INFO log for decrypted packet type (PING/PONG/INIT) with peer_id+src
- 1662-1668: expanded INIT accepted log with all fields:
  peer_id, mtu, link_id, socket_id, type, only_local, src_ip:src_port,
  session_id, actual src (UDP) — shows NAT mismatch
2026-07-13 11:01:50 +03:00
Evgeny 03701f525f conn_mgr: add missing sc_init_ctx before sc_set_peer_public_key
Without sc_init_ctx, crypto_ctx.initialized==0, sc_set_peer_public_key returns
SC_ERR_NOT_INITIALIZED, and all subsequent sc_encrypt (INIT, etc) fail with
'encryption failed' error 2. Fixed in cm_start_phase_direct and
reverse-connect path (both places that create ETCP_CONN for conn_mgr).
2026-07-13 10:03:15 +03:00
Evgeny 89b1a01448 invite+clipboard+conn_mgr: sock_meta for alien nodes, clipboard fix, bg_ping fix
- chat_core_connect_from_invite: add TOPO_SOCKMETA4 (id=0, UNKNOWN, UNKNOWN)
  so cm_has_direct_ip/cm_start_phase_direct can match socket_id=0 addr
- conn_mgr: cm_bg_ping_timer_cb reads real addr from nq->node->v4_addrs
  instead of NULL; added cm_bg_ping_noop_cb per required etcp_send_ping_to_socket
- invitedialog: set both QClipboard::Clipboard and Selection on Linux/X11
2026-07-13 09:37:46 +03:00
Evgeny e3b32cf4d3 conn_mgr: fix all etcp_route_send calls (dgram was NULL) + improve error diagnostics
- conn_mgr.c: 5 call sites used queue_entry_new(data_size) with data[] instead
  of dgram — all etcp_route_send calls silently dropped. Fixed by switching to
  dgram-based allocation (queue_entry_new(0)+u_malloc) matching all other callers.
- DIRCECT_REQ reuses already-allocated pkt as qe->dgram (no extra memcpy).
- etcp_router.c:1078 — 'empty entry' now shows dgram=%p len=%u dst=0x... force=%d
- etcp_connections.c:1139 — 'bad args' now shows all arg pointers to identify NULL
- etcp_connections.c:1145 — 'no socket' now shows addr_family=%d
- conn_mgr.c:633 — 'no candidates' now notes direct+reverse failed
2026-07-13 08:49:20 +03:00
Evgeny 2d93dd66c8 chatgui: fix Windows build - replace netinet/in.h with socket_compat.h 2026-07-13 08:08:27 +03:00
Evgeny 85a91a1320 chatgui: invite link connect flow + GUI_ERROR always-on diagnostics
- debug_ui.h: remove NDEBUG guard so GUI_ERROR works in Release builds
- invite_link.h/cpp: add error field to InviteData with specific failure reason
- joindialog: show exact decode error in QLabel instead of generic message
- chat_sync: implement invite connect flow (CHANNEL_INFO_REQ/RESP, JOIN, WELCOME)
- topo_node_sqlite: node/address table init and lookup
- utun_node: getInviteAddresses fallback from ETCP sockets
- minor fixes in mainwindow, chat_core, gui_bridge
2026-07-13 08:06:19 +03:00
Evgeny 102816d0dc node_config: load() returns true on parse success; main: specific diagnostics per missing field 2026-07-13 07:52:49 +03:00
Evgeny c65940f499 Copy Qt platforms/qwindows.dll plugin alongside executable 2026-07-13 07:44:06 +03:00
Evgeny 79bdafc9a7 Optimize recursive DLL resolution: scan only newly added DLLs 2026-07-13 07:43:25 +03:00
Evgeny cb1021f231 Recursive DLL dependency resolution via objdump for build.sh windeployqt fallback 2026-07-13 07:36:51 +03:00
Evgeny 58503907b5 Use objdump -p for smart DLL dependency discovery as windeployqt fallback 2026-07-13 07:33:49 +03:00
Evgeny 666507d3fe Move Qt DLL deployment from CMake to build.sh (windeployqt + manual fallback) 2026-07-13 07:30:56 +03:00
Evgeny f0f20c158b Simplify windeployqt: use PATH, add diagnostics, --no-compiler-runtime 2026-07-13 07:23:35 +03:00
Evgeny ed5ccd1ffb Add windeployqt post-build step for automatic Qt DLL deployment on Windows 2026-07-13 07:19:37 +03:00
Evgeny 65611790fa Fix enterEvent signature for Qt5/Qt6 compatibility 2026-07-13 07:06:58 +03:00
Evgeny e74240f0dd Add my_ed25519_privkey field to UTUN_INSTANCE struct 2026-07-13 07:01:42 +03:00
Evgeny e438e19f9e Add cross-platform utun_mkdir() wrapper for Windows compatibility 2026-07-13 06:44:18 +03:00
Evgeny de7841f33a chatgui: create group, unified logging, fix uasync_post epoll bug
- Add CreateGroupDialog with group name input
- Generate X25519+Ed25519 keypairs, random uint64 channel_id,
  Ed25519 signature, write to DB via uasync thread
- chat_core_create_channel(): create msg table, topo_node_sqlite_channel_put()
- GUI_EVT_CHANNEL_UPDATED notification to reload channel list
- Per-category debug config: debug_level= + debug_categories= in [gui]
- Unified logging via debug_config: all threads write to same file/console
- debug_enable_console(1) to mirror file output to stderr
- DEBUG_LEVEL_DISABLED=-1: truly disable category, NONE=0 fallback to global
- Fix uasync_post under epoll: process_posted_tasks was never called
- Fix MainWindow constructor parameter ordering (dbPath vs debugFile)
- debug_ui.h: append mode ('a' instead of 'w')
2026-07-12 17:39:20 +03:00
Evgeny a9a6b6bb10 TOPO_GROUP: add group_type support (UTUN/CHAT), mismatch detection, topo_groups_create_group()
- CHAT groups don't send subnets and skip route insert/delete
- Group type mismatch sends ERR_GROUP_MISMATCH to peer
- New topo_groups_create_group() public API for creating typed groups
2026-07-11 22:16:39 +03:00
Evgeny 3f874a80ab TOPO_GROUP: embed struct ll_entry as first member (direct queue embedding, hash index) 2026-07-09 19:50:17 +03:00
Evgeny 1f4749a1b9 Убрал имена функций из DEBUG-сообщений (__FUNCTION__ уже добавляет)
Оставлены метки-подсказки: NODEINFO, ACK_SEND, ETCP_SEND, DUMP,
router, conn_mgr, db_sync, probe, ctrl_out, socks_proxy и т.д.
2026-07-09 19:23:49 +03:00
Evgeny 28e5950d29 TOPO_BGP -> TOPO_GROUP: multi-group support
- Renamed TOPO_BGP to TOPO_GROUP with topo_group_id (64bit)
- Added TOPO_GROUPS container (ll_queue group_list) in UTUN_INSTANCE
- Memory pools moved from TOPO_GROUP to TOPO_GROUPS (instance-level)
- Default group: TOPO_GROUP_UTUN = 0x8000000000000000
- topo_groups_get_default() searches by group_id, not just first entry
- All topo_bgp_* renamed to topo_group_*; function signatures updated
- Files: topo_bgp.h/c -> topo_group.h/c
2026-07-09 18:08:49 +03:00
Evgeny 3c3d9a65d9 refactor: переименование route_node/route_bgp в topo_node/topo_bgp
Файлы: route_node.* route_node_lmdb.* route_bgp.* -> topo_node.* topo_node_lmdb.* topo_bgp.*

Структуры:
- ROUTE_BGP -> TOPO_BGP, ROUTE_BGP_CONN_ITEM -> TOPO_BGP_CONN_ITEM
- NODEINFO -> TOPO_NODE, NODEINFO_Q -> TOPO_NODEQ, NODEINFO_PATH -> TOPO_NODEPATH
- NODEINFO_ROUTES -> TOPO_NODESUBNETS, NODE_CONNECTIVITY -> TOPO_CONNECTIVITY
- wire: NODEINFO_MSG -> TOPOMSG_NODE, NODEINFO_IPV4_* -> TOPOMSG_*4 и т.д.
- memory: NI_IPV4_* -> TOPO_*4, NI_IPV6_* -> TOPO_*6
- ROUTE_SUBCMD_* -> TOPO_SUBCMD_*, ETCP_ID_ROUTE_ENTRY -> ETCP_ID_TOPO_ENTRY

Функции:
- route_bgp_* -> topo_bgp_*
- nodeinfo_* -> topo_node_*
- route_node_* -> topo_node_*
- route_node_lmdb_* -> topo_node_lmdb_*

Смысл: отделение механизма формирования топологии узлов от IP-роутинга.
2026-07-09 17:01:56 +03:00
Evgeny c7ac42eb5f route_node: refactor NODEINFO — separate protocol/memory structs, linked lists with memory_pool
- NODEINFO_MSG (protocol, packed): added flags byte + group_id field
- NODEINFO (memory): ref_count, linked list heads instead of inline arrays
- NODEINFO_ROUTES: separate struct with linked list subnet heads
- NODEINFO_Q: node*, routes*, tranzit_data*, hop_list* as separate mallocs
- 6 memory_pools in ROUTE_BGP for NI_* linked list items
- ni_list_count() universal counter via _ni_head cast
- nodeinfo_serialize/deserialize for protocol <-> memory conversion
- NODEINFO_FLAG_SEND_SUBNETS controls subnet data in protocol
- group_id defaults: utun=NODEINFO_GROUP_UTUN(1) with subnets

Fixes:
- deserialization: data+2 instead of data+sizeof(BGP_NODEINFO_PACKET)
- memset: start after ll_entry to preserve ll.size
- hop_src: subtract hop_count*8 to point at correct dynamic offset
- same-ver branch: remove_path(conn) instead of remove_path_by_hop(peer)
- double-free in route_bgp_remove_conn/process_withdraw
- conn_mgr_add_alien_node: rewritten for new structures
- all test files updated for new API
2026-07-09 16:20:48 +03:00
Evgeny 62954ca0a9 chatgui: replace MsgClient/ChatPropagator/crypto with chat_core, update DB/messaging/layout 2026-07-08 21:15:45 +03:00
Evgeny 1889d23240 metrics: RTT/loss histograms + Ed25519 signing + peer exchange via INIT handshake + file logging to stats/<peer>.dat
- etcp_metrics struct: 12-bucket RTT histogram, 11-bucket loss histogram, 10-min snapshots (>=200 pkts)
- Total counters: packets/bytes sent/rcvd, losses (retrans + probe gap detection)
- CSV append to <stats_dir>/<peer_id>.dat every 10 min
- Ed25519 pubkey exchange in INIT handshake (V3), always present
- Signed metrics exchange via ETCP_SECTION_METRICS (0x0A)
- Received metrics saved as from_<peer_id>.dat with signature hex
- on_metrics_rcvd callback in ETCP_BINDINGS
- stats_dir derived from config file path, mkdir'd at instance init
- Simplified versioning: ETCP_INIT_REQ_SIZE / ETCP_INIT_RESP_SIZE via sizeof
2026-07-07 17:16:45 +03:00
Evgeny f5c93266dc security: hardening flags + control_server fixes (F-001, F-002)
- Add --hardened flag to build.sh with FORTIFY, stack protector, PIE, RELRO
- Add --enable-hardening to configure.ac
- F-001: fix %s -> %.32s to prevent stack leak from non-null-terminated network string
- F-002: add pre-check recv_len >= MAX before buf_space computation to prevent size_t underflow
2026-07-07 07:47:37 +03:00
jeka 861d81815f gui update db and connection 2026-07-06 21:57:52 +03:00
jeka 9dafb4c31e gui update db and connection 2026-07-06 21:52:46 +03:00
Evgeny 19a4505a03 chat_sync: новый протокол синхронизации чата вместо ChatPropagator
- gui_bridge.h/cpp — кросс-поточные асинхронные коллбэки uasync↔GUI
- chat_sync.h/c — протокол синхронизации (INIT_SYNC/RESP, SEND_DATA, PUSH/ACK, SYNC_DONE)
- DbManager полностью переписан: per-channel таблицы msg_<ch> + peers_<ch>,
  4 ключа в channels, signature, protocol/rtt в node_addresses, sync-методы
- ChatPropagator удалён, заменён на chat_sync
- db_sync_stub.c — заглушки для сборки chatgui без LMDB-версии db_sync
- db_schema.md актуализирован под новую схему
2026-07-06 16:25:52 +03:00
Evgeny 46f5ff7eca test_etcp_router: fix flaky keepalive timeout under dummynet congestion
Root cause: 80kbps congestion causes 9+ second backpressure during push
phase. No packets reach server → keepalive timeout (2s default) fires
→ ETCP link down → BGP cleanup → etcp_router_conn_restart()
  - frees send_q (up to 64 = ROUTER_MAX_SEND_Q_PACKETS lost)
  - sends 9-byte restart notification to service handler (conn=NULL)
  - resets seq to 0

Fixes:
1. Set keepalive_timeout=60000, keepalive_adaptive=0 in test configs
   to prevent spurious timeouts during intentional congestion testing
2. Handle 9-byte restart notification in srv_handler:
   when conn==NULL && len==9 → reset g_expected_seq=0 gracefully
2026-07-06 12:28:54 +03:00
Evgeny 7628f77b93 chatgui: integrate UtunNode + NodeConfig, replace MsgClient
- Add NodeConfig: simplified INI config manager for embedded uTun node
  - X25519 key generation via OpenSSL EVP
  - Random node_id generation
  - addServer/removeServer, addClient/removeClient API
  - Auto-creates config with default server on first run
- Integrate UtunNode into MainWindow:
  - Creates NodeConfig in ~/.config/chatgui/node.conf
  - Starts embedded uTun node (no TUN, no msg_transport)
  - In separate thread with uasync_poll event loop
- Replace MsgClient (TCP IPC) with UtunNode (direct calls):
  - ChatPropagator now uses UtunNode for send/recv
  - send(uint64_t, QByteArray) replaces sendTo
  - messageReceived signal replaces received signal
- MainWindow cleanup: remove MsgClient, add UtunNode+NodeConfig
2026-07-06 11:59:01 +03:00
Evgeny 118ab04788 chatgui: add UtunNode C++ wrapper for embedded uTun node
- UtunNode runs uTun instance in a dedicated std::thread
- no TUN, no msg_transport — direct etcp_route_send/recv calls
- Thread-safe cross-thread communication via Qt signals
- C callback (etcp_recv_fn) stores UtunNode* via thread_local
- etcp_router_bind on service ID 0x11 for chat messages
- uasync_poll(100ms) loop runs until stop() called
2026-07-06 11:51:41 +03:00
Evgeny bfb83f7459 chatgui: add CMake build for libutun (C library linked into GUI)
- Add tools/chatgui/libutun/CMakeLists.txt — builds libuasync (from lib/*.c)
  and libutun (from src/*.c except utun.c) as static libraries
- All C code compiled as C99, linked into chatgui C++ binary
- Include paths and defs match autotools build
- chatgui CMakeLists.txt: add_subdirectory(libutun), link utun library
- Fix zxing-cpp docs/ directory issue
2026-07-06 11:48:56 +03:00
Evgeny 797c0736a7 headers: add extern "C" guards for C++ interop (chatgui)
Added #ifdef __cplusplus / extern "C" { ... } / #endif to ~65 headers
in src/ and lib/, enabling the C library to be linked into C++ code (chatgui).
Fixed packet_dump.h (added missing header guard) and etcp_bbr.h
(#pragma once guard).
2026-07-06 11:44:29 +03:00
Evgeny 8756f561be build: extract libutun.a from src/ + add db_sync module + etcp API, BBR improvements
- Add libutun.a (noinst) containing all src/*.c except utun.c
- utun binary now links libutun.a + libuasync.a
- tests/Makefile.am simplified: all tests link libutun.a instead of
  manual .o file lists (removed ~70 lines of boilerplate)
- Add db_sync module: distributed content-addressed DB with LMDB
  backend, peer-to-peer sync via etcp_router (svc 0x20)
- etcp API: etcp_send_in_order(), etcp_clear_inflight_queues(),
  verify_packet_signature(), BBR getters
- BBR: adaptive threshold (1/4 of pipe-min instead of hardcoded)
- route_bgp/etcp_connect: STCP transport support in NODEINFO
- conn_mgr: NODEINFO_ADDRS doc for nat_type/transport/proto fields
- Cleanup: removed AUDIT_FINDINGS.txt, filelist.txt, nat.txt
2026-07-06 11:42:13 +03:00
jeka 7139969acd fix: Windows/MSYS2 build compatibility
- proxy/icmp_proxy.c: define struct icmp/ip + ICMP constants for _WIN32
- lwip_tcp_priv.h: use #include <winsock2.h> instead of #define htonl/htons
- lwip_tcp.h/c: rename poll field to pollfn (avoid WSAPoll macro conflict)
- tests: guard Unix-only headers with #ifndef _WIN32, skip fork/socketpair tests
2026-07-05 20:57:44 +03:00
Evgeny 59d307f98b conn_mgr.h: document in ll_queue style; config_parser/utun_instance updates 2026-07-05 19:08:13 +03:00
Evgeny 90ad64a106 stcp: add protocol field to NODEINFO addr structs, TCP support in etcp_connect, stcp traffic test
- route_node.h: add NODE_PROTO_UDP/NODE_PROTO_TCP constants, protocol field in NODEINFO_IPV4_ADDR/NODEINFO_IPV6_ADDR
- route_node.c: fill protocol for UDP sockets + TCP servers from config, update dumps
- etcp_connect.c: create stcp_link for TCP-capable addresses, ready/close callbacks, cleanup
- etcp_connections.c: store stcp_server_listen result in instance->stcp_server, fix socket init checks for TCP-only configs
- utun_instance.c: use stcp_link_server_destroy (high-level API) instead of stcp_server_destroy
- tests/test_stcp_traffic.c: 2 nodes via etcp_connect+TCP, 1MB bidirectional traffic (10-1800 byte pkts)
2026-07-05 14:49:35 +03:00
Evgeny 4914348557 etcp_router: транзитные очереди per (src,dst) с backpressure + унификация etcp_send
- TRANSIT_QUEUE: отдельная очередь на каждую пару src+dst узлов
- ll_queue с хеш-индексом для быстрого поиска transit-очередей
- backpressure через waiter на send_input_q (threshold=0, один пакет за вызов, round-robin)
- Очередь создаётся при первом пакете который не получается отправить напрямую, удаляется при опустошении
- Унифицирован etcp_send: единый путь queue_data_put(send_input_q) для UDP (normalizer->input) и STCP (tx_queue)
- Убран STCP-ветвления из router_forward_transit/drain_cb
- Исправлено перекрытие ll.data и tq->q в TRANSIT_QUEUE (добавлены явные поля src_node_id/dst_node_id)
- Фикс лика dgram в tx_queue_cb/client_tx_queue_cb (добавлен queue_dgram_free)
- transit_queues живут внутри ETCP_CONN, инициализируются лениво, очищаются в etcp_connection_close (до pn_deinit) и stcp_link_close
2026-07-05 12:57:41 +03:00
Evgeny 55fa5c7759 fix: add retry_timer fallback in tcp_proxy_server for backpressure stall
waiter-based backpressure (commit ba14e31e) could starve HTTPS CONNECT
worker under concurrent 4-worker load — send_q never drained below threshold
fast enough for the waiter to fire before curl timed out (exit=28).

Add retry_timer (500ms, force=1) alongside etcp_router_on_send_ready
waiter: waiter gives instant wake when send_q drains, timer guarantees
delivery via force=1 if waiter hasn't fired in time.

Also improve socks_proxy DEBUG diagnostics in backpressure branches.
2026-07-04 19:32:31 +03:00
Evgeny 0b5a710fe6 etcp_connect + BGP_READY callbacks + router reconnect test
- etcp_connect.c: async connect API with EARLY/LATE/BGP_READY callbacks
- etcp_connect.h: declaration (via etcp_api.h)
- connect_deliver: fix to preserve unfired callback nodes
- etcp_set_routing_exchange_state: fires bgp_ready_cbk on TABLE_COMPLETE
- route_bgp: TABLE_COMPLETE handler + route_bgp_send_table_complete
- etcp.c: connections_count tracking, cleanup on close
- test_etcp_router_reconnect: a-b-c topology with b restart (etcp_connect)
- test_etcp_connect: unit tests for async connect API
- etcp_router: inflight/retrans/close improvements
- Various proxy/NAT/router cleanups
2026-07-04 18:51:01 +03:00
Evgeny f1f533a552 move CLOSE/RST flags from hdr->seq to hdr->flags
- ROUTER_SEQ_CLOSE_FLAG (0x80000000) → ROUTER_FLAG_CLOSE (0x02 in flags)
- ROUTER_SEQ_RST_FLAG (0x40000000) → ROUTER_FLAG_RST (0x40 in flags, already existed)
- router_send_one_flags: uint32_t seq_flags → uint8_t flag_bits
- router_send_to: uint32_t seq_flags → uint8_t flag_bits, hdr->seq=0, hdr->flags=flag_bits
- etcp_router_recv_cb: check hdr->flags instead of hdr->seq for CLOSE/RST
2026-07-03 20:38:11 +03:00
Evgeny 03490cfc00 refactor: extract etcp_router_recv_cb into 5 functions + fix send_q threshold
- router_verify_signature — Ed25519 signature verification
- router_handle_ack — ACK processing + drain on send_blocked
- router_forward_transit — packet forwarding
- router_check_peer_restart — session restart + RST detection
- router_handle_data_packet — reorder/assembly/dedup
- free_entry helper
- etcp_router_recv_cb: 198 → 42 lines
- ROUTER_MAX_SEND_Q_PACKETS threshold: MAX/2 → MAX-1 (min wait)
2026-07-03 19:27:36 +03:00
Evgeny ba14e31e09 fix: remove retry_timer, use waiter-based round-robin on send_q for fairness
- tcp_proxy_server: drain_cb uses etcp_router_on_send_ready instead of retry_timer
- socks_proxy: on_read_cb + tx_waiter_cb use etcp_router_on_send_ready instead of waiter_register
- Result: FIFO round-robin via ll_queue waiters on send_q, all streams get fair share
- No timers — wakeup is event-driven via queue_data_get → check_waiters
- TCP backpressure: read_queue callback_suspended prevents over-reading
2026-07-03 16:23:56 +03:00
Evgeny 5b850ea5da test: add profiling (MB/s per file per worker) to socks_http_proxy 2026-07-03 15:54:09 +03:00
Evgeny 2fba5c31c4 fix: backpressure data loss in socks/http proxy + comprehensive test
socks_proxy (client side):
- tx_buf buffering on send_data failure in RELAY path, retry in tx_waiter_cb
- process_http_request: version_str null-term fix (memcpy(vlen)+explicit \0)
- process_http_request: assemble hdr+body+extra into heap pkt, atomic send
- socks_proxy_handle_etcp: alloc failure logs write_queue state + byte counters

tcp_proxy_server (exit node):
- read_queue_drain_cb: tx_buf buffering on backpressure, retry_timer (500ms, force=1)
- on_fin_cb: defer FIN when read_queue+tx_buf has pending data (dst_fin_deferred)
- handle_data + drain_cb: per-stream byte counters (sent/recv/relayed/drain_count)
- on_fin_cb/on_error_cb: detailed state logging (counters, queues, buffers)

test_socks_http_proxy:
- 4 workers: socks_ipv4, socks_domain (DNS resolution via SOCKS), http_proxy, https_connect
- 4 file sizes: 1KB, 100KB, 1MB, 10MB — deterministic content, byte-by-byte cmp -s
- HTTPS server with self-signed cert, POST/echo, HEAD, OPTIONS
- 51/51 tests PASS
2026-07-03 14:29:17 +03:00
jeka 4f168c93f6 1 2026-07-03 12:48:25 +03:00
Evgeny 6c938ed2ba test_etcp_router: dummynet congestion test + timer leak fixes
- test_etcp_router: full rewrite with dummynet proxy, congestion cycles
  (fill inflight to backpressure, push 20 packets through, release shaper)
  bitmap tracking (0 loss, 0 dup, 0 corrupt checks), timing diagnostics
- Makefile.am: add utun-dummynet.o to test_etcp_router LDADD
- conn_mgr: fix cm_exch_probe timer leak (NULL callback → retry_cb),
  clean exchange_pending in cm_entry_destroy + conn_mgr_destroy
- utun_instance: add stcp_server_destroy() call in utun_instance_destroy
2026-07-03 12:48:51 +03:00
Evgeny 2cb85cbc3d socks_proxy: поддержка HTTP-прокси (GET/POST/HEAD/OPTIONS/...) — не только CONNECT
process_http_request() теперь обрабатывает любые HTTP-методы:
- CONNECT — туннель как раньше, без изменений
- GET/POST/PUT/HEAD/OPTIONS/... — парсинг абсолютного URL, DNS,
  send_connect + пересборка строки с относительным путём + тело DATA

test_socks_http_proxy: +3-й worker (http_proxy через -x без --proxytunnel),
+проверки POST (echo), HEAD (200 OK), OPTIONS (Allow). Все 78 проверок — PASS.
2026-07-02 22:04:37 +03:00
Evgeny 754c08e9c2 etcpmon: widen sockets list by ~100px (15 chars), shift action section 2026-07-02 21:26:21 +03:00
Evgeny 71ccccc2d5 etcpmon: widen sockets list +15px, shift action section right 2026-07-02 21:20:37 +03:00
Evgeny c6d1ebeeaf debug: log etcpmon socket_list NAT values in send_socket_list 2026-07-02 20:40:08 +03:00
Evgeny eea0ed0a48 debug: add NAT category logging to self-detect, init_resp nat update, and NAT_INFO receive paths 2026-07-02 20:31:32 +03:00
Evgeny 0738dcfd5a fix: self-detection uses interface_addr instead of local_addr (works with 0.0.0.0 bind); don't overwrite nat_addr in handle_init_response_client if NAT already verified 2026-07-02 20:24:32 +03:00
Evgeny 4862c30261 add CFG_SERVER_TYPE_LOCAL (type=local) and UNKNOWN semantics: PRIVATE/LOCAL hidden from nodeinfo, PRIVATE blocks incoming INIT, LOCAL accepts; UNKNOWN treated as PUBLIC; self-detection of PUBLIC IP on incoming INIT; filter LOCAL from NAT checks and connectivity 2026-07-02 19:30:08 +03:00
Evgeny 3c5d82a6ee add cross-build-win.sh for Windows cross-compilation; gitignore build-win/ build artifacts only 2026-07-02 18:59:20 +03:00
Evgeny 389ef00da6 refactor: split socket meta 'type' into config_type and nat_type — separate config (PUBLIC/NAT/PRIVATE) from detected NAT (EIM/STRICT/DIRECT) across NODEINFO, conn_mgr, route_bgp, route_node, etcpmon; win32 compat fixes (memmem, SHUT_WR, tun test_mode, tcp_proxy guards) 2026-07-02 18:58:34 +03:00
Evgeny 487cb8718e fix: preserve newlines in config_updater insert/update — no extra newlines on key replacement, add leading nl when inserting at end without trailing nl 2026-07-02 17:37:17 +03:00
Evgeny 42d96f118d fix: swap queue_entry_free/queue_dgram_free order in 27 places — prevents UAF when dgram_free accesses freed entry
uasync: replace raw socket_node* handle with packed index — fixes UAF after socket_array realloc
tcp_io: defer u_free in tcp_conn_destroy via uasync_call_soon — prevents callback-chain UAF
tcp_io: guard read_cb/write_cb with NULL queue checks after deferred destroy
tcp_io: save read_queue to local before queue_data_put + NULL guard after
route_connectivity: linked-list probe_ctx — cancel all parallel probes before nq free
2026-07-02 17:14:38 +03:00
Evgeny 8962f58ea4 uasync: replace raw socket_node pointer with index-based handle — fix UAF after realloc 2026-07-02 17:08:24 +03:00
Evgeny 1f541d6432 tcp_io: save read_queue to local var before queue_data_put, add NULL guard after 2026-07-02 16:38:31 +03:00
Evgeny e66156eb49 tcp_io: deferred free in tcp_conn_destroy to prevent UAF in callback chain 2026-07-02 16:26:43 +03:00
Evgeny 2cd38988e6 AGENTS: add ASAN build instructions 2026-07-02 15:49:10 +03:00
Evgeny d58b532741 gitignore: add utun.pid 2026-07-02 15:48:05 +03:00
Evgeny 4716e54069 build: add --asan flag to build.sh, tcp_io: null sock/queues in destroy + guard read_cb/write_cb 2026-07-02 15:47:54 +03:00
Evgeny aa73e9d8cb route_connectivity: fix UAF in conn_probe_finish — break ctx->nq on cancel_node, always free ctx in finish 2026-07-02 15:02:11 +03:00
Evgeny 993c265125 control_server: add idle timeout (30s) on client socket + large output queue warning 2026-07-02 14:17:42 +03:00
Evgeny 5ca0bbf9e7 tcp_io: place write_monitor=0 before queue_resume_callback to prevent busy-loop when waiter_defer=1 2026-07-02 13:50:43 +03:00
Evgeny 0ef7ee597c tcp_io: fix write_cb busy-loop with deferred callback — turn off write_monitor after flush when waiter_defer=1 2026-07-02 13:43:19 +03:00
Evgeny 7d460ca156 fix: ASAN-detected UAFs and buffer overflows across 6 modules
- etcp_connections: fix link leak & UAF in etcp_socket_remove — remove_link inside etcp_link_close shifts array, skip NULL set and i++
- stcp_server: fix UAF in stcp_conn_process_recv — defer free via uasync_call_soon after stcp_conn_do_close
- etcp: save pkt_len before queue_data_put to output_queue (callback may free entry synchronously)
- socks_proxy: add pool bounds check before memcpy, UINT16_MAX truncation guard, freed flag
- tcp_proxy_server: reorder cleanup (cancel waiters before tcp_conn_destroy), UINT16_MAX guard
- test_route6_lib: increase nodes[] array to STRESS_NODES + STRESS_OPS
2026-07-02 13:35:22 +03:00
Evgeny b738b2ff1b etcp_router: add u_check guards at send_q/recv_q entry points to detect memory corruption 2026-07-02 12:57:21 +03:00
Evgeny cc5289e37c etcp_router: don't reset start_sent in reactive restart — prevents START-flag cascade on multi-hop restart detection 2026-07-02 12:28:31 +03:00
Evgeny 0c5f25c96d etcp_router: fix restart cascade — sync peer_sess_id on auto-create, remove sess_id increment on reactive restart 2026-07-02 12:23:12 +03:00
Evgeny b53758bdbe etcp_router: cancel timers in conn_restart, reset last_ack_sent_tb — fix use-after-free SIGSEGV 2026-07-02 11:34:57 +03:00
Evgeny f5b1c3e789 etcp_router: fix OOB drops after restart — add peer_sync_done to sync rx_seq with first incoming seq 2026-07-02 11:12:14 +03:00
Evgeny 0edd21e335 etcpmon: lift Router to y=1181 (tight to Queues), shrink window to 1400 2026-07-02 10:33:41 +03:00
Evgeny e0c38676e7 etcpmon: compress Queues h 300->280, lift Router y 1260->1240 2026-07-02 10:31:44 +03:00
Evgeny fefd2bcc52 etcpmon: fix layout — restore Debug Levels y=485, lift Links to y=710, compress Queues h=300 y=900 2026-07-02 10:29:58 +03:00
Evgeny 8427e18c98 control_server: fix infinite loop in router_conns aggregation — use read-only head->next traversal instead of get/put 2026-07-02 10:23:06 +03:00
Evgeny 7329fa65f1 etcp_router: add congestion counters, fix tx_seq rollback on etcp_send failure, add router metrics to etcpmon GUI
- etcp_router.h: add 9 per-conn counters (pkts_sent/err/rcvd, ack_sent/recv/stale, dup/oob/sign_fail)
- etcp_router.c: increment counters at 13 event points; fix tx_seq rollback on etcp_send failure (was causing phantom inflight=256 deadlock); add DEBUG_ERROR on all 5 ignored etcp_send returns; fix router_ack_do_send to update timestamps only on success
- etcpmon_protocol.h: add etcpmon_router_metrics (14 aggregated fields)
- control_server.c: iterate router_conns to fill aggregated router metrics
- etcpmon_gui: new 'Router Congestion' block with queue depths and all counters, Debug Levels shifted down, window 1300->1440
- test_etcp_router_unit: fix test_ack_timer for correct no-BGP-route behavior
2026-07-02 09:49:10 +03:00
Evgeny de8cce521a all 51 tests pass: fd=0 protection + clean socks_http_proxy test
- socket_compat: never close fd=0
- socks_http_proxy: rewritten cleanly, 2 workers, 25 reqs each, up to 100KB
- uasync_socket_race: dup2-based fd=0 reservation
- u_async: removed DIAG spam
- tcp_io: removed debug fprintf

Result: 51/51 tests pass.
2026-07-02 00:58:47 +03:00
Evgeny f06ebd2ab4 fix: protect fd 0 from close in socket_close_wrapper + dup2 reserve in tests
- socket_close_wrapper: never close sock==0 (prevents fd 0 reuse cycle)
- test_uasync_socket_race: dup2-based fd 0 reservation, pass 800/800
- tcp_io: SOCKET_INVALID before on_error (previous commit)
- Plus earlier fixes: uasync gen counter, ll_queue call_soon_id, pkt_normalizer

Result: test_uasync_socket_race PASSES, 50/51 tests pass
2026-07-02 00:50:36 +03:00
Evgeny a97a38d867 fix: set tc->sock=SOCKET_INVALID before on_error in handle_error
В одном epoll_wait batch три события для fd: EPOLLERR (освобождает tc),
EPOLLIN (read_cb), EPOLLOUT (write_cb). read_cb/write_cb guard проверяет
tc->sock==SOCKET_INVALID, но sock обнулялся ПОСЛЕ on_error.
Перенос sock=SOCKET_INVALID до on_error защищает от use-after-free.

Доказано: без фикса segfault на tcp_io.c:374 (write_cb → tc->connected).
С фиксом тест работает без крашей.
2026-07-02 00:43:37 +03:00
Evgeny aff3b2d00f fix: always remove socket from epoll in tcp_conn_handle_error
tcp_conn_handle_error гвардил uasync_remove_socket_t по tc->socket_id,
пропуская повторный вызов. В одном epoll_wait batch два события
(EPOLLERR + EPOLLOUT) на один fd: первое делает remove (socket_id=NULL),
второе проходит guard и dispatch'ит write_cb на освобождённом tc.

Фикс: всегда вызывать uasync_remove_socket_t + epoll_ctl(DEL),
независимо от socket_id.
2026-07-02 00:16:05 +03:00
Evgeny ccdbab95a8 fix: use-after-free in pkt_normalizer + ll_queue waiter cancel + uasync gen counter for epoll fd-reuse
- pkt_normalizer: save recvpart->len before queue_data_put (may free synchronously)
- ll_queue: save uasync_call_soon handle in queue_waiter_handle, cancel on free
- uasync: generation counter in epoll data.u64 to skip stale events after fd reuse
- uasync: keep fd_to_index mapping after socket_array_remove for inactive nodes
- test: test_uasync_socket_race — 4 children × 200 iterations, currently flaky
2026-07-01 23:51:31 +03:00
Evgeny a01e2f0899 tun_enabled config + socks/http stress test (50 reqs, 2 workers)
- config_parser: tun_enabled в [global] (по умолчанию yes), не дефолтить
  tun_ifname если TUN не нужен
- utun_instance: уважать tun_enabled, чистые дефолты tcp_proxy TUN
- socks_proxy: фикс дэдлока — reply сразу после CONNECT, без ожидания DATA
- utun.conf.sample: документирован tun_enabled, tun_ifname
- test_socks_http_proxy: 50 запросов (SOCKS+HTTP), 2 воркера параллельно,
  файлы 1k..100k, сверка через cmp, python http.server + curl
2026-07-01 22:14:37 +03:00
Evgeny dfdec06a17 build.sh: add missing ./configure step after autoreconf 2026-07-01 21:19:21 +03:00
Evgeny 393a641e18 socks/http proxy client: SOCKS5 + HTTP CONNECT через ETCP
Добавлен модуль socks_proxy.c/h — SOCKS5 и HTTP CONNECT proxy на клиенте.
Использует tcp_io для локальных TCP соединений, туннелит трафик через
существующий ETCP-протокол (ETCP_ID_TCP_PROXY/ETCP_ID_TCP_PROXY_CLIENT).

Конфиг [tcp_proxy_client]:
- socks_enabled/socks_addr — включить SOCKS5 сервер на указанном IP:PORT
- http_proxy_enabled/http_proxy_addr — включить HTTP CONNECT сервер
- Можно включать TUN, SOCKS и HTTP прокси одновременно или раздельно

Весь трефик идёт через exit node (tcp_proxy_server), менять exit не пришлось.
2026-07-01 20:33:11 +03:00
Evgeny 53d41d5c62 fix: use-after-free in route table — add route_delete before u_free in route_bgp_update_my_nodeinfo and route_bgp_process_nodeinfo, relax routes_overlap to allow hierarchical routes
Root cause: when bgp->local_node was reallocated (changed=1), old route
entries remained with stale v_node_info pointing to freed memory. The freed
memory got reused by a BGP peer's NODEINFO_Q, causing route_lookup to return
wrong node_id (b3b2b1b0afaeadac instead of local), which then failed at
etcp_route_send with 'no BGP route' and dropped packets to 10.23.2.1.

Changes:
- route_node.c: route_delete(old) before u_free + route_insert(new) after alloc
- utun_instance.c: remove duplicate route_insert (now atomic inside function)
- route_bgp.c: route_delete before freeing old peer NODEINFO_Q
- route_lib.c: routes_overlap now rejects only exact prefix+network duplicates
- tests/Makefile.am: add route_lib.o to test_route6_lib
2026-07-01 18:53:23 +03:00
Evgeny c60ba6eaec tun: tun_init_nat always uses TUN_MTU_DEFAULT (1500), ignore ETCP mtu from config 2026-07-01 17:57:36 +03:00
Evgeny c5123e4baf bgp: prevent sending NODEINFO to self + own NODEINFO early return
- route_bgp_process_nodeinfo: skip own node_id early, don't add to bgp->nodes
- broadcast loop: skip sending NODEINFO to connection where peer == node_id
- route_bgp_should_send_to: don't send node's NODEINFO to itself
- tests: lower debug level to ERROR, fix keepalive adaptive config
2026-07-01 17:49:27 +03:00
Evgeny 928c2fe057 tun: always use TUN_MTU_DEFAULT (1500) for TUN interface, ignore ETCP mtu from config 2026-07-01 17:35:53 +03:00
Evgeny b5b6875bc5 bgp: skip own NODEINFO in route_bgp_process_nodeinfo
When own NODEINFO is reflected back from a peer, do not add it
to bgp->nodes. Only update bgp->local_node which is sent separately.
This prevents route duplication and the 'local node in learned list'
inconsistency.
2026-07-01 14:48:38 +03:00
Evgeny 7c953c1929 bgp: add test_bgp_triangle + keepalive_adaptive config option
- keepalive_adaptive=0 disables adaptive keepalive period growth,
  keeping it fixed at 200ms for faster timeout detection in tests
- test_bgp_triangle: 5-node triangle topology test (6 phases):
  validates alternative path recording (fix#1), cascading node
  removal on full isolation, and full recovery
- config generation via utun_instance_create_from_str() with
  pre-generated keys — no temp files needed
2026-07-01 14:22:59 +03:00
Evgeny 1c09ad2e5a bgp: fix node removal when alternative paths exist
- route_bgp_process_nodeinfo: record alternative paths even when NODEINFO
  version hasn't changed (fixes nodes disappearing on link down despite
  having other active links)
- route_bgp_process_withdraw: only remove node when all paths gone (ret==1)
- route_bgp_remove_conn: broadcast withdraw for each removed node with
  correct wd_source
2026-07-01 13:37:35 +03:00
jjj 3284dffaaf 1 2026-07-01 13:02:32 +03:00
Evgeny c893285d71 action popup: use SW_RESTORE + SetForegroundWindow to bring hidden window to front 2026-07-01 12:46:19 +03:00
Evgeny 75028f278a action popup: re-show window on every action result if it was hidden 2026-07-01 12:42:30 +03:00
Evgeny 7fc1b5419f action popup: use ClearType Consolas 10pt font instead of ANSI_FIXED_FONT 2026-07-01 12:27:32 +03:00
Evgeny dee9889423 action popup: convert \n -> \r\n for Windows EDIT control line breaks 2026-07-01 12:24:48 +03:00
Evgeny 35871891db action nodes: send BGP node dump as text response to GUI, display in popup
- route_node_format_all(): format all BGP nodes to snprintf buffer
- control_server: 'nodes' action sends ETCPMON_RSP_ACTION_RESULT (0x89) with text
- etcpmon_protocol.h: new response type + struct, increase MAX_MSG_SIZE to 32K
- etcpmon_client: on_action_result callback, send_action via send_request with seq_id
- etcpmon_gui: ETCPMonActionResult popup window class with EDIT multiline,
  on_action_result creates popup (same size as main) or appends text + scrolls
2026-07-01 12:19:50 +03:00
Evgeny 373e4b33c7 route_node_dump_all: dump all BGP nodes to log via debug INFO
Add route_node_dump_all() in route_node.c that walks bgp->nodes list
and outputs per-node info: node_id, name, version, pubkeys, IPv4/IPv6
addrs/sockets/subnets, paths with hop list, connectivity state, conn_mgr.
Wire 'nodes' action in control_server to call this function.
2026-06-30 23:33:45 +03:00
Evgeny 090e2f28e4 1 2026-06-30 19:18:03 +03:00
Evgeny a2c7f1e6cb fix etcp: infinite loop in feed_dgram_to_asm due to uint16_t overflow in new_cap doubling 2026-06-30 16:21:43 +03:00
Evgeny 21da640ebe route_bgp: find_conn_for_node prefers live links_up path, fallback to best any 2026-06-30 14:59:26 +03:00
Evgeny 80177c9b3e fix: test_conn_mgr flakiness — wait for BGP nodeinfo, not just INIT links
Root cause: test1 used lks() (INIT links up) but conn_mgr needs BGP NODEINFO.
Nodeinfo arrives after INIT, so test1 must wait for it.
Also made result volatile to prevent loop optimization.
2026-06-30 00:34:06 +03:00
Evgeny 8f13f0ec4d refactor: extract handle_ping, handle_pong, send_init_response, handle_init_response_client from read callback
Reduce etcp_connections_read_callback_socket from ~600 to ~300 lines.
All 4 helpers are static file-scope functions, no logic changes.
2026-06-30 00:16:34 +03:00
Evgeny 5420e414ce fix: DIRECT detection read srс_ipv4/src_port before INIT RESPONSE overwrites the buffer
Both req (INIT_REQUEST) and resp (INIT_RESPONSE) point to pkt->data.
resp->peer_port at offset 23-24 overwrites req->src_ipv4[0..1] at same offset.
Save src_ipv4/src_port before building the response.
2026-06-29 23:58:46 +03:00
Evgeny 69112ffde1 feat: NAT_DIRECT detection during INIT handshake + refactor init packet structs
- Extract ETCP_INIT_REQUEST_PKT / ETCP_INIT_RESPONSE_PKT named structs
- Replace anonymous structs and manual offset parsing with typed struct access
- Add src_ipv4/src_port fields (V2) to INIT REQUEST for DIRECT detection
- Server compares reported client addr with observed src; match + public IP → NAT_TYPE_DIRECT
- Skip STUN NAT check for DIRECT links
- route_bgp: handle NAT_TYPE_DIRECT → NAT_VERIFIED_DIRECT in handle_nat_info
2026-06-29 23:15:38 +03:00
Evgeny 89d675f15d fix: drain input_send_q and input_wait_ack at end of etcp_conn_reset to ensure inflight queues are cleared on remote node restart 2026-06-29 20:17:53 +03:00
Evgeny 914f252b12 fix: remove send_blocked_inflight — per-link inflight gate was blocking input_send_q and ACK send, causing bidirectional deadlock 2026-06-29 20:04:20 +03:00
Evgeny a7b20f268f feat: conn_mgr, dummynet block-addr, indirect routing, alien nodes, nodeinfo refactor 2026-06-29 20:01:40 +03:00
Evgeny c0207ad41d fix: move nodeinfo_dyn_size declaration to route_bgp.h 2026-06-25 09:47:27 +03:00
Evgeny 8051dfe4e5 zxing-cpp: strip to core library only (remove tests/wrappers/examples/docs); .gitignore: fix /core to allow zxing-cpp/core/ dir 2026-06-25 09:37:45 +03:00
Evgeny 22c24d135e chatgui: invite link dialogs, zxing-cpp QR code, crypto transport, channel improvements; msg_transport: chat protocol support 2026-06-25 09:29:44 +03:00
Evgeny d8c8c1c5b0 control_server: node subscription protocol with output queue, push notifications on routing changes 2026-06-25 09:28:07 +03:00
Evgeny 231081dd2f docs: add chatgui animated emoji system and tdesktop-dev reference to AGENTS.md 2026-06-24 13:50:10 +03:00
Evgeny 62b3695a77 chatgui: rlottie animated emoji with hover-to-animate, drag-to-select
- Add LottieIcon class (TGS gzip decompress + rlottie C API render)
- Add AnimTimer singleton (33ms tick, activate/deactivate refcount)
- Add EmojiButton (hover-triggered animation in emoji picker)
- Add AnimatedEmoji support to EmojiData (5 TGS mappings)
- MessageDelegate: QTextLayout-based animated emoji overlay (text + reactions)
- ChatView: drag-to-select (click deselects, 5px drag starts selection)
- ChatView: hoveredIndexChanged signal for hover-triggered animations
- MessageList: wire AnimTimer tick to viewport update, hover → activate
- CMake: AUTORCC, rlottie, zlib dependencies
- Add 5 TGS animations as Qt resources
2026-06-24 13:45:25 +03:00
Evgeny 0e2b0e28d8 tools/chatgui: Makefile cmake wrapper, author name in bubble, emoji panel height ~1/1.7 2026-06-23 14:39:09 +03:00
Evgeny e50ca9978e tools/chatgui: input bar with emoji panel, custom tab bar, Ctrl+Enter multiline, scroll fixes 2026-06-23 00:17:07 +03:00
Evgeny b667d6f352 tools/chatgui: Qt5 GUI chat app stub with tray, channel list, message bubbles 2026-06-22 22:48:27 +03:00
Evgeny 6717c863e7 etcp_router: Ed25519 digital signature for service messages
- Add ed25519_public_key[32] to NODEINFO struct for pubkey distribution
- Add ed25519_public_key[32] to ROUTE_BGP (derived from X25519 privkey at init)
- Add ROUTER_FLAG_SIGNED (0x08) to SVC_ROUTE_HDR flags byte
- router_send_one_flags: when is_signed, sign [hdr][payload] and append 64-byte sig
- etcp_router_recv_cb: verify Ed25519 signature when ROUTER_FLAG_SIGNED set
- New API: etcp_router_conn_send_signed()
- Drop signed packets if sender node not in routing table or no Ed25519 key
- 5 unit tests in test_etcp_router_unit.c covering OK/tampered/unknown/no-key/short
2026-06-22 15:53:24 +03:00
Evgeny 2bbdf8195f fix: TASSERT return value in test_etcp_stcp.c and test_stcp_link.c 2026-06-18 22:11:25 +03:00
Evgeny 73f707ef9b crypto: migrate ECDH(secp256r1) to X25519, drop TinyCrypt
- SC_PUBKEY_SIZE: 64 -> 32 bytes, SC_PUBKEY_ENC_SIZE: 72 -> 40
- sc_set_peer_public_key uses EVP_PKEY_derive with X25519
- sc_generate_keypair via EVP_PKEY_keygen(EVP_PKEY_X25519)
- sc_compute_public_key_from_private via EVP_PKEY_new_raw_private_key
- sc_validate_key: check non-zero (X25519 has no invalid keys)
- sc_obfuscate_pubkey: XOR now 32 bytes
- Remove all TinyCrypt code (~860 lines), always use OpenSSL
- Remove TinyCrypt from build system, drop test_crypto/test_ecc_encrypt
- Update all test keys (128 hex -> 64 hex)
- Fix bbr_integration.c split-line pubkeys
2026-06-17 21:28:07 +03:00
Evgeny 34683f0500 fix: update last_recv_local_time on INIT packet receive to prevent spurious keepalive timeout
last_recv_local_time was not set in the server INIT processing path,
causing the keepalive timer to see a stale timestamp and trigger a
false link-down → route_bgp_remove_conn() right after reconnect.
Routes could be deleted before the first normal packet from the
client updated the timestamp via process_decrypted.
2026-06-17 15:05:20 +03:00
Evgeny 150e9977df stcp: unified transport — etcp_send/bind over TCP via stcp_link
- stcp_link.h/c: manage TCP connections through STCP protocol
  stcp_server_listen, stcp_link_connect/send/close
  rx dispatch goes directly through inst->api_bindings (single table)
- etcp_api.c: etcp_send checks conn->transport_link for TCP path
  etcp_bind stores only in api_bindings (no duplication)
- etcp_connections.c: init_sockets creates stcp_server for TCP
  init_connections creates stcp_link_connect for TCP clients
  tcp_server_on_link creates minimal ETCP_CONN + fires callbacks
- etcp.h: ETCP_CONN.transport_link field (NULL for UDP, set for TCP)
- config_parser: CFG_SERVER.transport field, parsing transport=tcp|udp
- utun_instance.h: struct stcp_server*
- Tests: test_stcp_link (handshake, wrong key)
         test_etcp_stcp (etcp_send/bind over TCP)
- Renamed: transport.h/c → stcp_link.h/c (cleaner, no bindings table)
- 46/46 tests pass
2026-06-16 01:19:58 +03:00
1852 changed files with 1069972 additions and 36808 deletions

No files matched your search

+18 -2
View File
@@ -1,5 +1,9 @@
# Build directory (out-of-tree build)
/build/
build-win/
tools/chatgui/build/
tools/chatgui/ffmpeg/ffmpeg_build/
tools/lightsout/build/
# Build outputs
*.o
@@ -8,7 +12,8 @@
*.a
*.so
*.so.*
utun
/utun
/src/utun
test_serialize
linux-x64-deb
@@ -25,6 +30,7 @@ index.html
# Autotools
Makefile
!tools/etcpmon/Makefile
Makefile.in
/aclocal.m4
/autom4te.cache/
@@ -59,7 +65,7 @@ Makefile.in
.cache/
# Debug outputs
core
/core
vgcore.*
callgrind.out.*
@@ -74,6 +80,7 @@ logs/
# Test binaries (executables without extension), but track source/config files
tests/test_*
!tests/test_*.c
!tests/test_*.h
!tests/test_*.conf
!tests/test_*.json
tests/bench_*
@@ -83,6 +90,10 @@ tests/bench_*
*.log
utun.log*
# logreceiver artifacts
tools/logreceiver/logreceiver_output.log*
tools/logreceiver/build/
# Local configs
*.cfg
!utun_test.cfg
@@ -96,6 +107,9 @@ utun.conf
tools/bping/bping
tools/proxy/udp_proxy
# Reference projects (not part of utun)
tools/tdesktop-dev/
# Debug/temp artifacts
*.bak
*.debug
@@ -109,3 +123,5 @@ build_win.log
/wintun.dll
src/lwip_orig/
lib/timeout/
utun.pid
.check_time
-771
View File
@@ -1,771 +0,0 @@
<div align="center">
![OpenAgents Control Hero](docs/images/hero-image.png)
# OpenAgents Control (OAC)
### Control your AI patterns. Get repeatable results.
**AI agents that learn YOUR coding patterns and generate matching code every time.**
🎯 **Pattern Control** - Define your patterns once, AI uses them forever
✋ **Approval Gates** - Review and approve before execution
🔁 **Repeatable Results** - Same patterns = Same quality code
📝 **Editable Agents** - Full control over AI behavior
👥 **Team-Ready** - Everyone uses the same patterns
**Multi-language:** TypeScript • Python • Go • Rust • Any language*
**Model Agnostic:** Claude • GPT • Gemini • Local models
[![GitHub stars](https://img.shields.io/github/stars/darrenhinde/OpenAgentsControl?style=flat-square&logo=github&labelColor=black&color=ffcb47)](https://github.com/darrenhinde/OpenAgentsControl/stargazers)
[![X Follow](https://img.shields.io/twitter/follow/DarrenBuildsAI?style=flat-square&logo=x&labelColor=black&color=1DA1F2)](https://x.com/DarrenBuildsAI)
[![License: MIT](https://img.shields.io/badge/License-MIT-3fb950?style=flat-square&labelColor=black)](https://opensource.org/licenses/MIT)
[![Last Commit](https://img.shields.io/github/last-commit/darrenhinde/OpenAgentsControl?style=flat-square&labelColor=black&color=8957e5)](https://github.com/darrenhinde/OpenAgentsControl/commits/main)
[🚀 Quick Start](#-quick-start) • [💻 Show Me Code](#-example-workflow) • [🗺️ Roadmap](https://github.com/darrenhinde/OpenAgentsControl/projects) • [💬 Community](https://nextsystems.ai)
</div>
---
> **Built on [OpenCode](https://opencode.ai)** - An open-source AI coding framework. OAC extends OpenCode with specialized agents, context management, and team workflows.
---
## The Problem
Most AI agents are like hiring a developer who doesn't know your codebase. They write generic code. You spend hours rewriting, refactoring, and fixing inconsistencies. Tokens burned. Time wasted. No actual work done.
**Example:**
```typescript
// What AI gives you (generic)
export async function POST(request: Request) {
const data = await request.json();
return Response.json({ success: true });
}
// What you actually need (your patterns)
export async function POST(request: Request) {
const body = await request.json();
const validated = UserSchema.parse(body); // Your Zod validation
const result = await db.users.create(validated); // Your Drizzle ORM
return Response.json(result, { status: 201 }); // Your response format
}
```
## The Solution
**OpenAgentsControl teaches agents your patterns upfront.** They understand your coding standards, your architecture, your security requirements. They propose plans before implementing. They execute incrementally with validation.
**The result:** Production-ready code that ships without heavy rework.
### What Makes AOC Different
**🎯 Context-Aware (Your Secret Weapon)**
Agents load YOUR patterns before generating code. Code matches your project from the start. No refactoring needed.
**📝 Editable Agents (Not Baked-In Plugins)**
Full control over agent behavior. Edit markdown files directly—no compilation, no vendor lock-in. Change workflows, add constraints, customize for your team.
**✋ Approval Gates (Human-Guided AI)**
Agents ALWAYS request approval before execution. Propose → Approve → Execute. You stay in control. No "oh no, what did the AI just do?" moments.
**⚡ Token Efficient (MVI Principle)**
Minimal Viable Information design. Only load what's needed, when it's needed. Context files <200 lines, lazy loading, faster responses.
**👥 Team-Ready (Repeatable Patterns)**
Store YOUR coding patterns once. Entire team uses same standards. Commit context to repo. New developers inherit team patterns automatically.
**🔄 Model Agnostic**
Use any AI model (Claude, GPT, Gemini, local). No vendor lock-in.
**Full-stack development:** AOC handles both frontend and backend work. The agents coordinate to build complete features from UI to database.
---
## 🆚 Quick Comparison
| Feature | OpenAgentsControl | Cursor/Copilot | Aider | Oh My OpenCode |
|---------|-------------------|----------------|-------|----------------|
| **Learn Your Patterns** | ✅ Built-in context system | ❌ No pattern learning | ❌ No pattern learning | ⚠️ Manual setup |
| **Approval Gates** | ✅ Always required | ⚠️ Optional (default off) | ❌ Auto-executes | ❌ Fully autonomous |
| **Token Efficiency** | ✅ MVI principle (80% reduction) | ❌ Full context loaded | ❌ Full context loaded | ❌ High token usage |
| **Team Standards** | ✅ Shared context files | ❌ Per-user settings | ❌ No team support | ⚠️ Manual config per user |
| **Edit Agent Behavior** | ✅ Markdown files you edit | ❌ Proprietary/baked-in | ⚠️ Limited prompts | ✅ Config files |
| **Model Choice** | ✅ Any model, any provider | ⚠️ Limited options | ⚠️ OpenAI/Claude only | ✅ Multiple models |
| **Execution Speed** | ⚠️ Sequential with approval | Fast | Fast | ✅ Parallel agents |
| **Error Recovery** | ✅ Human-guided validation | ⚠️ Auto-retry (can loop) | ⚠️ Auto-retry | ✅ Self-correcting |
| **Best For** | Production code, teams | Quick prototypes | Solo developers | Power users, complex projects |
**Use AOC when:**
- ✅ You have established coding patterns
- ✅ You want code that ships without refactoring
- ✅ You need approval gates for quality control
- ✅ You care about token efficiency and costs
**Use others when:**
- **Cursor/Copilot:** Quick prototypes, don't care about patterns
- **Aider:** Simple file edits, no team coordination
- **Oh My OpenCode:** Need autonomous execution with parallel agents (speed over control)
> **Full comparison:** [Read detailed analysis →](https://github.com/darrenhinde/OpenAgentsControl/discussions/116)
---
## 🚀 Quick Start
**Prerequisites:** [OpenCode CLI](https://opencode.ai/docs) (free, open-source) • Bash 3.2+ • Git
### Step 1: Install
**One command:**
```bash
curl -fsSL https://raw.githubusercontent.com/darrenhinde/OpenAgentsControl/main/install.sh | bash -s developer
```
<sub>The installer will set up OpenCode CLI if you don't have it yet.</sub>
**Or interactive:**
```bash
curl -fsSL https://raw.githubusercontent.com/darrenhinde/OpenAgentsControl/main/install.sh -o install.sh
bash install.sh
```
### Step 2: Start Building
```bash
opencode --agent OpenAgent
> "Create a user authentication system"
```
### Step 3: Approve & Ship
**What happens:**
1. Agent analyzes your request
2. Proposes a plan (you approve)
3. Executes step-by-step with validation
4. Delegates to specialists when needed
5. Ships production-ready code
**That's it.** Works immediately with your default model. No configuration required.
---
## 💡 The Context System: Your Secret Weapon
**The problem with AI code:** It doesn't match your patterns. You spend hours refactoring.
**The AOC solution:** Teach your patterns once. Agents load them automatically. Code matches from the start.
### How It Works
```
Your Request
↓
ContextScout discovers relevant patterns
↓
Agent loads YOUR standards
↓
Code generated using YOUR patterns
↓
Ships without refactoring ✅
```
### Add Your Patterns (10-15 Minutes)
```bash
/add-context
```
**Answer 6 simple questions:**
1. What's your tech stack? (Next.js + TypeScript + PostgreSQL + Tailwind)
2. Show an API endpoint example (paste your code)
3. Show a component example (paste your code)
4. What naming conventions? (kebab-case, PascalCase, camelCase)
5. Any code standards? (TypeScript strict, Zod validation, etc.)
6. Any security requirements? (validate input, parameterized queries, etc.)
**Result:** Agents now generate code matching your exact patterns. No refactoring needed.
### The MVI Advantage: Token Efficiency
**MVI (Minimal Viable Information)** = Only load what's needed, when it's needed.
**Traditional approach:**
- Loads entire codebase context
- Large token overhead per request
- Slow responses, high costs
**AOC approach:**
- Loads only relevant patterns
- Context files <200 lines (quick to load)
- Lazy loading (agents load what they need)
- 80% of tasks use isolation context (minimal overhead)
**Real benefits:**
- **Efficiency:** Lower token usage vs loading entire codebase
- **Speed:** Faster responses with smaller context
- **Quality:** Code matches your patterns (no refactoring)
### For Teams: Repeatable Patterns
**The team problem:** Every developer writes code differently. Inconsistent patterns. Hard to maintain.
**The AOC solution:** Store team patterns in `.opencode/context/project/`. Commit to repo. Everyone uses same standards.
**Example workflow:**
```bash
# Team lead adds patterns once
/add-context
# Answers questions with team standards
# Commit to repo
git add .opencode/context/
git commit -m "Add team coding standards"
git push
# All team members now use same patterns automatically
# New developers inherit standards on day 1
```
**Result:** Consistent code across entire team. No style debates. No refactoring PRs.
---
## 📖 How It Works
### The Core Idea
**Most AI tools:** Generic code → You refactor
**OpenAgentsControl:** Your patterns → AI generates matching code
### The Workflow
```
1. Add Your Context (one time)
↓
2. ContextScout discovers relevant patterns
↓
3. Agent loads YOUR standards
↓
4. Agent proposes plan (using your patterns)
↓
5. You approve
↓
6. Agent implements (matches your project)
↓
7. Code ships (no refactoring needed)
```
### Key Benefits
**🎯 Context-Aware**
ContextScout discovers relevant patterns. Agents load YOUR standards before generating code. Code matches your project from the start.
**🔁 Repeatable**
Same patterns → Same results. Configure once, use forever. Perfect for teams.
**⚡ Token Efficient (80% Reduction)**
MVI principle: Only load what's needed. 8,000 tokens → 750 tokens. Massive cost savings.
**✋ Human-Guided**
Agents propose plans, you approve before execution. Quality gates prevent mistakes. No auto-execution surprises.
**📝 Transparent & Editable**
Agents are markdown files you can edit. Change workflows, add constraints, customize behavior. No vendor lock-in.
### What Makes This Special
**1. ContextScout - Smart Pattern Discovery**
Before generating code, ContextScout discovers relevant patterns from your context files. Ranks by priority (Critical → High → Medium). Prevents wasted work.
**2. Editable Agents - Full Control**
Unlike Cursor/Copilot where behavior is baked into plugins, AOC agents are markdown files. Edit them directly:
```bash
nano .opencode/agent/core/opencoder.md # local project install
# Or: nano ~/.config/opencode/agent/core/opencoder.md # global install
# Add project rules, change workflows, customize behavior
```
**3. ExternalScout - Live Documentation** 🆕
Working with external libraries? ExternalScout fetches current documentation:
- Gets live docs from official sources (npm, GitHub, docs sites)
- No outdated training data - always current
- Automatically triggered when agents detect external dependencies
- Supports frameworks, APIs, libraries, and more
**4. Approval Gates - No Surprises**
Agents ALWAYS request approval before:
- Writing/editing files
- Running bash commands
- Delegating to subagents
- Making any changes
You stay in control. Review plans before execution.
**5. MVI Principle - Token Efficiency**
Files designed for quick loading:
- Concepts: <100 lines
- Guides: <150 lines
- Examples: <80 lines
Result: Lower token usage vs loading entire codebase.
**6. Team Patterns - Repeatable Results**
Store patterns in `.opencode/context/project/`. Commit to repo. Entire team uses same standards. New developers inherit patterns automatically.
---
## 🎯 Which Agent Should I Use?
### OpenAgent (Start Here)
**Best for:** Learning the system, general tasks, quick implementations
```bash
opencode --agent OpenAgent
> "Create a user authentication system" # Building features
> "How do I implement authentication in Next.js?" # Questions
> "Create a README for this project" # Documentation
> "Explain the architecture of this codebase" # Analysis
```
**What it does:**
- Loads your patterns via ContextScout
- Proposes plan (you approve)
- Executes with validation
- Delegates to specialists when needed
**Perfect for:** First-time users, simple features, learning the workflow
### OpenCoder (Production Development)
**Best for:** Complex features, multi-file refactoring, production systems
```bash
opencode --agent OpenCoder
> "Create a user authentication system" # Full-stack features
> "Refactor this codebase to use dependency injection" # Multi-file refactoring
> "Add real-time notifications with WebSockets" # Complex implementations
```
**What it does:**
- **Discover:** ContextScout finds relevant patterns
- **Propose:** Detailed implementation plan
- **Approve:** You review and approve
- **Execute:** Incremental implementation with validation
- **Validate:** Tests, type checking, code review
- **Ship:** Production-ready code
**Perfect for:** Production code, complex features, team development
### SystemBuilder (Custom AI Systems)
**Best for:** Building complete custom AI systems tailored to your domain
```bash
opencode --agent SystemBuilder
> "Create a customer support AI system"
```
Interactive wizard generates orchestrators, subagents, context files, workflows, and commands.
**Perfect for:** Creating domain-specific AI systems
---
## 🛠️ What's Included
### 🤖 Main Agents
- **OpenAgent** - General tasks, questions, learning (start here)
- **OpenCoder** - Production development, complex features
- **SystemBuilder** - Generate custom AI systems
### 🔧 Specialized Subagents (Auto-delegated)
- **ContextScout** - Smart pattern discovery (your secret weapon)
- **TaskManager** - Breaks complex features into atomic subtasks
- **CoderAgent** - Focused code implementations
- **TestEngineer** - Test authoring and TDD
- **CodeReviewer** - Code review and security analysis
- **BuildAgent** - Type checking and build validation
- **DocWriter** - Documentation generation
- **ExternalScout** - Fetches live docs for external libraries (no outdated training data) **NEW!**
- Plus category specialists: frontend, devops, copywriter, technical-writer, data-analyst
### ⚡ Productivity Commands
- `/add-context` - Interactive wizard to add your patterns
- `/commit` - Smart git commits with conventional format
- `/test` - Testing workflows
- `/optimize` - Code optimization
- `/context` - Context management
- And 7+ more productivity commands
### 📚 Context System (MVI Principle)
Your coding standards automatically loaded by agents:
- **Code quality** - Your patterns, security, standards
- **UI/design** - Design system, component patterns
- **Task management** - Workflow definitions
- **External libraries** - Integration guides (18+ libraries supported)
- **Project-specific** - Your team's patterns
**Key features:**
- 80% token reduction via MVI
- Smart discovery via ContextScout
- Lazy loading (only what's needed)
- Team-ready (commit to repo)
- Version controlled (track changes)
### How Context Resolution Works
ContextScout discovers context files using a **local-first** approach:
```
1. Check local: .opencode/context/core/navigation.md
↓ Found? → Use local for everything. Done.
↓ Not found?
2. Check global: ~/.config/opencode/context/core/navigation.md
↓ Found? → Use global for core/ files only.
↓ Not found? → Proceed without core context.
```
**Key rules:**
- **Local always wins** — if you installed locally, global is never checked
- **Global fallback is only for `core/`** (standards, workflows, guides) — universal files that are the same across projects
- **Project intelligence is always local** — your tech stack, patterns, and naming conventions live in `.opencode/context/project-intelligence/` and are never loaded from global
- **One-time check** — ContextScout resolves the core location once at startup (max 2 glob checks), not per-file
**Common setups:**
| Setup | Core files from | Project intelligence from |
|-------|----------------|--------------------------|
| Local install (`bash install.sh developer`) | `.opencode/context/core/` | `.opencode/context/project-intelligence/` |
| Global install + `/add-context` | `~/.config/opencode/context/core/` | `.opencode/context/project-intelligence/` |
| Both local and global | `.opencode/context/core/` (local wins) | `.opencode/context/project-intelligence/` |
---
## 💻 Example Workflow
```bash
opencode --agent OpenCoder
> "Create a user dashboard with authentication and profile settings"
```
**What happens:**
**1. Discover (~1-2 min)** - ContextScout finds relevant patterns
- Your tech stack (Next.js + TypeScript + PostgreSQL)
- Your API pattern (Zod validation, error handling)
- Your component pattern (functional, TypeScript, Tailwind)
- Your naming conventions (kebab-case files, PascalCase components)
**2. Propose (~2-3 min)** - Agent creates detailed implementation plan
```
## Proposed Implementation
**Components:**
- user-dashboard.tsx (main page)
- profile-settings.tsx (settings component)
- auth-guard.tsx (authentication wrapper)
**API Endpoints:**
- /api/user/profile (GET, POST)
- /api/auth/session (GET)
**Database:**
- users table (Drizzle schema)
- sessions table (Drizzle schema)
All code will follow YOUR patterns from context.
Approve? [y/n]
```
**3. Approve** - You review and approve the plan (human-guided)
**4. Execute (~10-15 min)** - Incremental implementation with validation
- Implements one component at a time
- Uses YOUR patterns for every file
- Validates after each step (type check, lint)
- *This is the longest step - generating quality code takes time*
**5. Validate (~2-3 min)** - Tests, type checking, code review
- Delegates to TestEngineer for tests
- Delegates to CodeReviewer for security check
- Ensures production quality
**6. Ship** - Production-ready code
- Code matches your project exactly
- No refactoring needed
- Ready to commit and deploy
**Total time: ~15-25 minutes** for a complete feature (guided, with approval gates)
### 💡 Pro Tips
**After finishing a feature:**
- Run `/add-context --update` to add new patterns you discovered
- Update your context with new libraries, conventions, or standards
- Keep your patterns fresh as your project evolves
**Working with external libraries?**
- **ExternalScout** automatically fetches current documentation
- No more outdated training data - gets live docs from official sources
- Works with npm packages, APIs, frameworks, and more
---
## ⚙️ Advanced Configuration
### Model Configuration (Optional)
**By default, all agents use your OpenCode default model.** Configure models per agent only if you want different agents to use different models.
**When to configure:**
- You want faster agents to use cheaper models (e.g., Haiku/Flash)
- You want complex agents to use smarter models (e.g., Opus/GPT-5)
- You want to test different models for different tasks
**How to configure:**
Edit agent files directly:
```bash
nano .opencode/agent/core/opencoder.md # local project install
# Or: nano ~/.config/opencode/agent/core/opencoder.md # global install
```
Change the model in the frontmatter:
```yaml
---
description: "Development specialist"
model: anthropic/claude-sonnet-4-5 # Change this line
---
```
Browse available models at [models.dev](https://models.dev/?search=open) or run `opencode models`.
### Update Context as You Go
Your project evolves. Your context should too.
```bash
/add-context --update
```
**What gets updated:**
- Tech stack, patterns, standards
- Version incremented (1.0 → 1.1)
- Updated date refreshed
**Example updates:**
- Add new library (Stripe, Twilio, etc.)
- Change patterns (new API format, component structure)
- Migrate tech stack (Prisma → Drizzle)
- Update security requirements
Agents automatically use updated patterns.
---
## 🎯 Is This For You?
### ✅ Use AOC if you:
- Build production code that ships without heavy rework
- Work in a team with established coding standards
- Want control over agent behavior (not black-box plugins)
- Care about token efficiency and cost savings
- Need approval gates for quality assurance
- Want repeatable, consistent results
- Use multiple AI models (no vendor lock-in)
### ⚠️ Skip AOC if you:
- Want fully autonomous execution without approval gates
- Prefer "just do it" mode over human-guided workflows
- Don't have established coding patterns yet
- Need multi-agent parallelization (use Oh My OpenCode instead)
- Want plug-and-play with zero configuration
### 🤔 Not Sure?
**Try this test:**
1. Ask your current AI tool to generate an API endpoint
2. Count how many minutes you spend refactoring it to match your patterns
3. If you're spending time on refactoring, AOC will save you that time
**Or ask yourself:**
- Do you have coding standards your team follows?
- Do you spend time refactoring AI-generated code?
- Do you want AI to follow YOUR patterns, not generic ones?
If you answered "yes" to any of these, AOC is for you.
---
## 🚀 Advanced Features
### Frontend Design Workflow
The **OpenFrontendSpecialist** follows a structured 4-stage design workflow:
1. **Layout** - ASCII wireframe, responsive structure planning
2. **Theme** - Design system selection, OKLCH colors, typography
3. **Animation** - Micro-interactions, timing, accessibility
4. **Implementation** - Single HTML file, semantic markup
### Task Management & Breakdown
The **TaskManager** breaks complex features into atomic, verifiable subtasks with smart agent suggestions and parallel execution support.
### System Builder
Build complete custom AI systems tailored to your domain in minutes. Interactive wizard generates orchestrators, subagents, context files, workflows, and commands.
---
## ❓ FAQ
### Getting Started
**Q: Does this work on Windows?**
A: Yes! Use Git Bash (recommended) or WSL.
**Q: What languages are supported?**
A: Agents are language-agnostic and adapt based on your project files. Primarily tested with TypeScript/Node.js. Python, Go, Rust, and other languages are supported but less battle-tested. The context system works with any language.
**Q: Do I need to add context?**
A: No, but it's highly recommended. Without context, agents write generic code. With context, they write YOUR code.
**Q: Can I use this without customization?**
A: Yes, it works out of the box. But you'll get the most value after adding your patterns (10-15 minutes with `/add-context`).
**Q: What models are supported?**
A: Any model from any provider (Claude, GPT, Gemini, local models). No vendor lock-in.
### For Teams
**Q: How do I share context with my team?**
A: Commit `.opencode/context/project/` to your repo. Team members automatically use same patterns.
**Q: How do we ensure everyone follows the same standards?**
A: Add team patterns to context once. All agents load them automatically. Consistent code across entire team.
**Q: Can different projects have different patterns?**
A: Yes! Use project-specific context (`.opencode/` in project root) to override global patterns.
### Technical
**Q: How does token efficiency work?**
A: MVI principle: Only load what's needed, when it's needed. Context files <200 lines (scannable in 30s). ContextScout discovers relevant patterns. Lazy loading prevents context bloat. 80% of tasks use isolation context (minimal overhead).
**Q: What's ContextScout?**
A: Smart pattern discovery agent. Finds relevant context files before code generation. Ranks by priority. Prevents wasted work.
**Q: Can I edit agent behavior?**
A: Yes! Agents are markdown files. Edit them directly: `nano .opencode/agent/core/opencoder.md` (local) or `nano ~/.config/opencode/agent/core/opencoder.md` (global)
**Q: How do approval gates work?**
A: Agents ALWAYS request approval before execution (write/edit/bash). You review plans before implementation. No surprises.
**Q: How do I update my context?**
A: Run `/add-context --update` anytime your patterns change. Agents automatically use updated patterns.
### Comparison
**Q: How is this different from Cursor/Copilot?**
A: AOC has editable agents (not baked-in), approval gates (not auto-execute), context system (YOUR patterns), and MVI token efficiency.
**Q: How is this different from Aider?**
A: AOC has team patterns, context system, approval workflow, and smart pattern discovery. Aider is file-based only.
**Q: How does this compare to Oh My OpenCode?**
A: Both are built on OpenCode. AOC focuses on **control & repeatability** (approval gates, pattern control, team standards). Oh My OpenCode focuses on **autonomy & speed** (parallel agents, auto-execution). [Read detailed comparison →](https://github.com/darrenhinde/OpenAgentsControl/discussions/116)
**Q: When should I NOT use AOC?**
A: If you want fully autonomous execution without approval gates, or if you don't have established coding patterns yet.
### Setup
**Q: What bash version do I need?**
A: Bash 3.2+ (macOS default works). Run `bash scripts/tests/test-compatibility.sh` to check.
**Q: Do I need to install plugins/tools?**
A: No, they're optional. Only install if you want Telegram notifications or Gemini AI features.
**Q: Where should I install - globally or per-project?**
A: Local (`.opencode/` in your project) is recommended — patterns are committed to git and shared with your team. Global (`~/.config/opencode/`) is good for personal defaults across all projects. The installer asks you to choose. See [OpenCode Config Docs](https://opencode.ai/docs/config/) for how configs merge.
---
## 🗺️ Roadmap & What's Coming
**This is only the beginning!** We're actively developing new features and improvements every day.
### 🚀 See What's Coming Next
Check out our [**Project Board**](https://github.com/darrenhinde/OpenAgentsControl/projects) to see:
- 🔨 **In Progress** - Features being built right now
- 📋 **Planned** - What's coming soon
- 💡 **Ideas** - Future enhancements under consideration
- ✅ **Recently Shipped** - Latest improvements
### 🎯 Current Focus Areas
- **Plugin System** - npm-based plugin architecture for easy distribution
- **Performance Improvements** - Faster agent execution and context loading
- **Enhanced Context Discovery** - Smarter pattern recognition
- **Multi-language Support** - Better Python, Go, Rust support
- **Team Collaboration** - Shared context and team workflows
- **Documentation** - More examples, tutorials, and guides
### 💬 Have Ideas?
We'd love to hear from you!
- 💡 [**Submit Feature Requests**](https://github.com/darrenhinde/OpenAgentsControl/issues/new?labels=enhancement)
- 🐛 [**Report Bugs**](https://github.com/darrenhinde/OpenAgentsControl/issues/new?labels=bug)
- 💬 [**Join Discussions**](https://github.com/darrenhinde/OpenAgentsControl/discussions)
**Star the repo** ⭐ to stay updated with new releases!
---
## 🤝 Contributing
We welcome contributions!
1. Follow the established naming conventions and coding standards
2. Write comprehensive tests for new features
3. Update documentation for any changes
4. Ensure security best practices are followed
See: [Contributing Guide](docs/contributing/CONTRIBUTING.md) • [Code of Conduct](docs/contributing/CODE_OF_CONDUCT.md)
---
## 💬 Community & Support
<div align="center">
**Join the community and stay updated with the latest AI development workflows!**
[![YouTube](https://img.shields.io/badge/YouTube-Darren_Builds_AI-red?style=for-the-badge&logo=youtube&logoColor=white)](https://youtube.com/@DarrenBuildsAI)
[![Community](https://img.shields.io/badge/Community-NextSystems.ai-blue?style=for-the-badge&logo=discourse&logoColor=white)](https://nextsystems.ai)
[![X/Twitter](https://img.shields.io/badge/Follow-@DarrenBuildsAI-1DA1F2?style=for-the-badge&logo=x&logoColor=white)](https://x.com/DarrenBuildsAI)
[![Buy Me A Coffee](https://img.shields.io/badge/Support-Buy_Me_A_Coffee-FFDD00?style=for-the-badge&logo=buy-me-a-coffee&logoColor=black)](https://buymeacoffee.com/darrenhinde)
**📺 Tutorials & Demos** • **💬 Join Waitlist** • **🐦 Latest Updates** • **☕ Support Development**
*Your support helps keep this project free and open-source!*
</div>
---
## License
This project is licensed under the MIT License.
---
**Made with ❤️ by developers, for developers. Star the repo if this saves you refactoring time!**
@@ -1,116 +0,0 @@
---
name: BuildAgent
description: Type check and build validation agent
mode: subagent
temperature: 0.1
permission:
bash:
"tsc": "allow"
"mypy": "allow"
"go build": "allow"
"cargo check": "allow"
"cargo build": "allow"
"npm run build": "allow"
"yarn build": "allow"
"pnpm build": "allow"
"python -m build": "allow"
"*": "deny"
edit:
"**/*": "deny"
write:
"**/*": "deny"
task:
contextscout: "allow"
"*": "deny"
---
# BuildAgent
> **Mission**: Validate type correctness and build success — always grounded in project build standards discovered via ContextScout.
<rule id="context_first">
ALWAYS call ContextScout BEFORE running build checks. Load build standards, type-checking requirements, and project conventions first. This ensures you run the right commands for this project.
</rule>
<rule id="read_only">
Read-only agent. NEVER modify any code. Detect errors and report them — fixes are someone else's job.
</rule>
<rule id="detect_language_first">
ALWAYS detect the project language before running any commands. Never assume TypeScript or any other language.
</rule>
<rule id="report_only">
Report errors clearly with file paths and line numbers. If no errors, report success. That's it.
</rule>
<system>Build validation gate within the development pipeline</system>
<domain>Type checking and build validation — language detection, compiler errors, build failures</domain>
<task>Detect project language → run type checker → run build → report results</task>
<constraints>Read-only. No code modifications. Bash limited to build/type-check commands only.</constraints>
<tier level="1" desc="Critical Operations">
- @context_first: ContextScout ALWAYS before build checks
- @read_only: Never modify code — report only
- @detect_language_first: Identify language before running commands
- @report_only: Clear error reporting with paths and line numbers
</tier>
<tier level="2" desc="Build Workflow">
- Detect project language (package.json, requirements.txt, go.mod, Cargo.toml)
- Run appropriate type checker
- Run appropriate build command
- Report results
</tier>
<tier level="3" desc="Quality">
- Error message clarity
- Actionable error descriptions
- Build time reporting
</tier>
<conflict_resolution>Tier 1 always overrides Tier 2/3. If language detection is ambiguous → report ambiguity, don't guess. If a build command isn't in the allowed list → report that, don't try alternatives.</conflict_resolution>
---
## 🔍 ContextScout — Your First Move
**ALWAYS call ContextScout before running any build checks.** This is how you understand the project's build conventions, expected type-checking setup, and any custom build configurations.
### When to Call ContextScout
Call ContextScout immediately when ANY of these triggers apply:
- **Before any build validation** — always, to understand project conventions
- **Project doesn't match standard configurations** — custom build setups need context
- **You need type-checking standards** — what level of strictness is expected
- **Build commands aren't obvious** — verify what the project actually uses
### How to Invoke
```
task(subagent_type="ContextScout", description="Find build standards", prompt="Find build validation guidelines, type-checking requirements, and build command conventions for this project. I need to know what build tools and configurations are expected.")
```
### After ContextScout Returns
1. **Read** every file it recommends (Critical priority first)
2. **Verify** expected build commands match what you detect in the project
3. **Apply** any custom build configurations or strictness requirements
---
# OpenCode Agent Configuration
# Metadata (id, name, category, type, version, author, tags, dependencies) is stored in:
# .opencode/config/agent-metadata.json
---
## What NOT to Do
- ❌ **Don't skip ContextScout** — build validation without project standards = running wrong commands
- ❌ **Don't modify any code** — report errors only, fixes are not your job
- ❌ **Don't assume the language** — always detect from project files first
- ❌ **Don't skip type-check** — run both type check AND build, not just one
- ❌ **Don't run commands outside the allowed list** — stick to approved build tools only
- ❌ **Don't give vague error reports** — include file paths, line numbers, and what's expected
---
# OpenCode Agent Configuration
# Metadata (id, name, category, type, version, author, tags, dependencies) is stored in:
# .opencode/config/agent-metadata.json
<context_first>ContextScout before any validation — understand project conventions first</context_first>
<detect_first>Language detection before any commands — never assume</detect_first>
<read_only>Report errors, never fix them — clear separation of concerns</read_only>
<actionable_reporting>Every error includes path, line, and what's expected — developers can fix immediately</actionable_reporting>
@@ -1,253 +0,0 @@
---
name: CoderAgent
description: Executes coding subtasks in sequence, ensuring completion as specified
mode: subagent
temperature: 0
permission:
bash:
"*": "deny"
"bash .opencode/skills/task-management/router.sh complete*": "allow"
"bash .opencode/skills/task-management/router.sh status*": "allow"
edit:
"**/*.env*": "deny"
"**/*.key": "deny"
"**/*.secret": "deny"
"node_modules/**": "deny"
".git/**": "deny"
task:
contextscout: "allow"
externalscout: "allow"
TestEngineer: "allow"
---
# CoderAgent
> **Mission**: Execute coding subtasks precisely, one at a time, with full context awareness and self-review before handoff.
<rule id="context_first">
ALWAYS call ContextScout BEFORE writing any code. Load project standards, naming conventions, and security patterns first. This is not optional — it's how you produce code that fits the project.
</rule>
<rule id="external_scout_mandatory">
When you encounter ANY external package or library (npm, pip, etc.) that you need to use or integrate with, ALWAYS call ExternalScout for current docs BEFORE implementing. Training data is outdated — never assume how a library works.
</rule>
<rule id="self_review_required">
NEVER signal completion without running the Self-Review Loop (Step 6). Every deliverable must pass type validation, import verification, anti-pattern scan, and acceptance criteria check.
</rule>
<rule id="task_order">
Execute subtasks in the defined sequence. Do not skip or reorder. Complete one fully before starting the next.
</rule>
<system>Subtask execution engine within the OpenAgents task management pipeline</system>
<domain>Software implementation — coding, file creation, integration</domain>
<task>Implement atomic subtasks from JSON definitions, following project standards discovered via ContextScout</task>
<constraints>Limited bash access for task status updates only. Sequential execution. Self-review mandatory before handoff.</constraints>
<tier level="1" desc="Critical Operations">
- @context_first: ContextScout ALWAYS before coding
- @external_scout_mandatory: ExternalScout for any external package
- @self_review_required: Self-Review Loop before signaling done
- @task_order: Sequential, no skipping
</tier>
<tier level="2" desc="Core Workflow">
- Read subtask JSON and understand requirements
- Load context files (standards, patterns, conventions)
- Implement deliverables following acceptance criteria
- Update status tracking in JSON
</tier>
<tier level="3" desc="Quality">
- Modular, functional, declarative code
- Clear comments on non-obvious logic
- Completion summary (max 200 chars)
</tier>
<conflict_resolution>
Tier 1 always overrides Tier 2/3. If context loading conflicts with implementation speed → load context first. If ExternalScout returns different patterns than expected → follow ExternalScout (it's live docs).
</conflict_resolution>
---
## 🔍 ContextScout — Your First Move
**ALWAYS call ContextScout before writing any code.** This is how you get the project's standards, naming conventions, security patterns, and coding conventions that govern your output.
### When to Call ContextScout
Call ContextScout immediately when ANY of these triggers apply:
- **Task JSON doesn't include all needed context_files** — gaps in standards coverage
- **You need naming conventions or coding style** — before writing any new file
- **You need security patterns** — before handling auth, data, or user input
- **You encounter an unfamiliar project pattern** — verify before assuming
### How to Invoke
```
task(subagent_type="ContextScout", description="Find coding standards for [feature]", prompt="Find coding standards, security patterns, and naming conventions needed to implement [feature]. I need patterns for [concrete scenario].")
```
### After ContextScout Returns
1. **Read** every file it recommends (Critical priority first)
2. **Apply** those standards to your implementation
3. If ContextScout flags a framework/library → call **ExternalScout** for live docs (see below)
---
# OpenCode Agent Configuration
# Metadata (id, name, category, type, version, author, tags, dependencies) is stored in:
# .opencode/config/agent-metadata.json
---
## Workflow
### Step 1: Read Subtask JSON
```
Location: .tmp/tasks/{feature}/subtask_{seq}.json
```
Read the subtask JSON to understand:
- `title` — What to implement
- `acceptance_criteria` — What defines success
- `deliverables` — Files/endpoints to create
- `context_files` — Standards to load (lazy loading)
- `reference_files` — Existing code to study
### Step 2: Load Reference Files
**Read each file listed in `reference_files`** to understand existing patterns, conventions, and code structure before implementing. These are the source files and project code you need to study — not standards documents.
This step ensures your implementation is consistent with how the project already works.
### Step 3: Discover Context (ContextScout)
**ALWAYS do this.** Even if `context_files` is populated, call ContextScout to verify completeness:
```
task(subagent_type="ContextScout", description="Find context for [subtask title]", prompt="Find coding standards, patterns, and conventions for implementing [subtask title]. Check for security patterns, naming conventions, and any relevant guides.")
```
Load every file ContextScout recommends. Apply those standards.
### Step 4: Check for External Packages
Scan your subtask requirements. If ANY external library is involved:
```
task(subagent_type="ExternalScout", description="Fetch [Library] docs", prompt="Fetch current docs for [Library]: [what I need to know]. Context: [what I'm building]")
```
### Step 5: Update Status to In Progress
Use `edit` (NOT `write`) to patch only the status fields — preserving all other fields like `acceptance_criteria`, `deliverables`, and `context_files`:
Find `"status": "pending"` and replace with:
```json
"status": "in_progress",
"agent_id": "coder-agent",
"started_at": "2026-01-28T00:00:00Z"
```
**NEVER use `write` here** — it would overwrite the entire subtask definition.
### Step 6: Implement Deliverables
For each item in `deliverables`:
- Create or modify the specified file
- Follow acceptance criteria exactly
- Apply all standards from ContextScout
- Use API patterns from ExternalScout (if applicable)
- Write tests if specified in acceptance criteria
### Step 7: Self-Review Loop (MANDATORY)
**Run ALL checks before signaling completion. Do not skip any.**
#### Check 1: Type & Import Validation
- Scan for mismatched function signatures vs. usage
- Verify all imports/exports exist (use `glob` to confirm file paths)
- Check for missing type annotations where acceptance criteria require them
- Verify no circular dependencies introduced
#### Check 2: Anti-Pattern Scan
Use `grep` on your deliverables to catch:
- `console.log` — debug statements left in
- `TODO` or `FIXME` — unfinished work
- Hardcoded secrets, API keys, or credentials
- Missing error handling: `async` functions without `try/catch` or `.catch()`
- `any` types where specific types were required
#### Check 3: Acceptance Criteria Verification
- Re-read the subtask's `acceptance_criteria` array
- Confirm EACH criterion is met by your implementation
- If ANY criterion is unmet → fix before proceeding
#### Check 4: ExternalScout Verification
- If you used any external library: confirm your usage matches the documented API
- Never rely on training-data assumptions for external packages
#### Self-Review Report
Include this in your completion summary:
```
Self-Review: ✅ Types clean | ✅ Imports verified | ✅ No debug artifacts | ✅ All acceptance criteria met | ✅ External libs verified
```
If ANY check fails → fix the issue. Do not signal completion until all checks pass.
### Step 8: Mark Complete and Signal
Update subtask status and report completion to orchestrator:
**8.1 Update Subtask Status** (REQUIRED for parallel execution tracking):
```bash
# Mark this subtask as completed using task-cli.ts
bash .opencode/skills/task-management/router.sh complete {feature} {seq} "{completion_summary}"
```
Example:
```bash
bash .opencode/skills/task-management/router.sh complete auth-system 01 "Implemented JWT authentication with refresh tokens"
```
**8.2 Verify Status Update**:
```bash
bash .opencode/skills/task-management/router.sh status {feature}
```
Confirm your subtask now shows: `status: "completed"`
**8.3 Signal Completion to Orchestrator**:
Report back with:
- Self-Review Report (from Step 7)
- Completion summary (max 200 chars)
- List of deliverables created
- Confirmation that subtask status is marked complete
Example completion report:
```
✅ Subtask {feature}-{seq} COMPLETED
Self-Review: ✅ Types clean | ✅ Imports verified | ✅ No debug artifacts | ✅ All acceptance criteria met | ✅ External libs verified
Deliverables:
- src/auth/service.ts
- src/auth/middleware.ts
- src/auth/types.ts
Summary: Implemented JWT authentication with refresh tokens and error handling
```
**Why this matters for parallel execution**:
- Orchestrator monitors subtask status to detect when entire parallel batch is complete
- Without status update, orchestrator cannot proceed to next batch
- Status marking is the signal that enables parallel workflow progression
---
# OpenCode Agent Configuration
# Metadata (id, name, category, type, version, author, tags, dependencies) is stored in:
# .opencode/config/agent-metadata.json
---
## Principles
- Context first, code second. Always.
- One subtask at a time. Fully complete before moving on.
- Self-review is not optional — it's the quality gate.
- External packages need live docs. Always.
- Functional, declarative, modular. Comments explain why, not what.
-108
View File
@@ -1,108 +0,0 @@
---
name: CodeReviewer
description: Code review, security, and quality assurance agent
mode: subagent
temperature: 0.1
permission:
bash:
"*": "deny"
edit:
"**/*": "deny"
write:
"**/*": "deny"
task:
contextscout: "allow"
---
# CodeReviewer
> **Mission**: Perform thorough code reviews for correctness, security, and quality — always grounded in project standards discovered via ContextScout.
<rule id="context_first">
ALWAYS call ContextScout BEFORE reviewing any code. Load code quality standards, security patterns, and naming conventions first. Reviewing without standards = meaningless feedback.
</rule>
<rule id="read_only">
Read-only agent. NEVER use write, edit, or bash. Provide review notes and suggested diffs — do NOT apply changes.
</rule>
<rule id="security_priority">
Security vulnerabilities are ALWAYS the highest priority finding. Flag them first, with severity ratings. Never bury security issues in style feedback.
</rule>
<rule id="output_format">
Start with: "Reviewing..., what would you devs do if I didn't check up on you?" Then structured findings by severity.
</rule>
<system>Code quality gate within the development pipeline</system>
<domain>Code review — correctness, security, style, performance, maintainability</domain>
<task>Review code against project standards, flag issues by severity, suggest fixes without applying them</task>
<constraints>Read-only. No code modifications. Suggested diffs only.</constraints>
<tier level="1" desc="Critical Operations">
- @context_first: ContextScout ALWAYS before reviewing
- @read_only: Never modify code — suggest only
- @security_priority: Security findings first, always
- @output_format: Structured output with severity ratings
</tier>
<tier level="2" desc="Review Workflow">
- Load project standards and review guidelines
- Analyze code for security vulnerabilities
- Check correctness and logic
- Verify style and naming conventions
</tier>
<tier level="3" desc="Quality Enhancements">
- Performance considerations
- Maintainability assessment
- Test coverage gaps
- Documentation completeness
</tier>
<conflict_resolution>Tier 1 always overrides Tier 2/3. Security findings always surface first regardless of other issues found.</conflict_resolution>
---
## 🔍 ContextScout — Your First Move
**ALWAYS call ContextScout before reviewing any code.** This is how you get the project's code quality standards, security patterns, naming conventions, and review guidelines.
### When to Call ContextScout
Call ContextScout immediately when ANY of these triggers apply:
- **No review guidelines provided in the request** — you need project-specific standards
- **You need security vulnerability patterns** — before scanning for security issues
- **You need naming convention or style standards** — before checking code style
- **You encounter unfamiliar project patterns** — verify before flagging as issues
### How to Invoke
```
task(subagent_type="ContextScout", description="Find code review standards", prompt="Find code review guidelines, security scanning patterns, code quality standards, and naming conventions for this project. I need to review [feature/file] against established standards.")
```
### After ContextScout Returns
1. **Read** every file it recommends (Critical priority first)
2. **Apply** those standards as your review criteria
3. Flag deviations from team standards as findings
---
# OpenCode Agent Configuration
# Metadata (id, name, category, type, version, author, tags, dependencies) is stored in:
# .opencode/config/agent-metadata.json
---
## What NOT to Do
- ❌ **Don't skip ContextScout** — reviewing without project standards = generic feedback that misses project-specific issues
- ❌ **Don't apply changes** — suggest diffs only, never modify files
- ❌ **Don't bury security issues** — they always surface first regardless of severity mix
- ❌ **Don't review without a plan** — share what you'll inspect before diving in
- ❌ **Don't flag style issues as critical** — match severity to actual impact
- ❌ **Don't skip error handling checks** — missing error handling is a correctness issue
---
# OpenCode Agent Configuration
# Metadata (id, name, category, type, version, author, tags, dependencies) is stored in:
# .opencode/config/agent-metadata.json
<context_first>ContextScout before any review — standards-blind reviews are useless</context_first>
<security_first>Security findings always surface first — they have the highest impact</security_first>
<read_only>Suggest, never apply — the developer owns the fix</read_only>
<severity_matched>Flag severity matches actual impact, not personal preference</severity_matched>
<actionable>Every finding includes a suggested fix — not just "this is wrong"</actionable>
@@ -1,126 +0,0 @@
---
name: TestEngineer
description: Test authoring and TDD agent
mode: subagent
temperature: 0.1
permission:
bash:
"npx vitest *": "allow"
"npx jest *": "allow"
"pytest *": "allow"
"npm test *": "allow"
"npm run test *": "allow"
"yarn test *": "allow"
"pnpm test *": "allow"
"bun test *": "allow"
"go test *": "allow"
"cargo test *": "allow"
"rm -rf *": "ask"
"sudo *": "deny"
"*": "deny"
edit:
"**/*.env*": "deny"
"**/*.key": "deny"
"**/*.secret": "deny"
task:
contextscout: "allow"
externalscout: "allow"
---
# TestEngineer
> **Mission**: Author comprehensive tests following TDD principles — always grounded in project testing standards discovered via ContextScout.
<rule id="context_first">
ALWAYS call ContextScout BEFORE writing any tests. Load testing standards, coverage requirements, and TDD patterns first. Tests without standards = tests that don't match project conventions.
</rule>
<rule id="positive_and_negative">
EVERY testable behavior MUST have at least one positive test (success case) AND one negative test (failure/edge case). Never ship with only positive tests.
</rule>
<rule id="arrange_act_assert">
ALL tests must follow the Arrange-Act-Assert pattern. Structure is non-negotiable.
</rule>
<rule id="mock_externals">
Mock ALL external dependencies and API calls. Tests must be deterministic — no network, no time flakiness.
</rule>
<system>Test quality gate within the development pipeline</system>
<domain>Test authoring — TDD, coverage, positive/negative cases, mocking</domain>
<task>Write comprehensive tests that verify behavior against acceptance criteria, following project testing conventions</task>
<constraints>Deterministic tests only. No real network calls. Positive + negative required. Run tests before handoff.</constraints>
<tier level="1" desc="Critical Operations">
- @context_first: ContextScout ALWAYS before writing tests
- @positive_and_negative: Both test types required for every behavior
- @arrange_act_assert: AAA pattern in every test
- @mock_externals: All external deps mocked — deterministic only
</tier>
<tier level="2" desc="TDD Workflow">
- Propose test plan with behaviors to test
- Request approval before implementation
- Implement tests following AAA pattern
- Run tests and report results
</tier>
<tier level="3" desc="Quality">
- Edge case coverage
- Lint compliance before handoff
- Test comments linking to objectives
- Determinism verification (no flaky tests)
</tier>
<conflict_resolution>Tier 1 always overrides Tier 2/3. If test speed conflicts with positive+negative requirement → write both. If a test would use real network → mock it.</conflict_resolution>
---
## 🔍 ContextScout — Your First Move
**ALWAYS call ContextScout before writing any tests.** This is how you get the project's testing standards, coverage requirements, TDD patterns, and test structure conventions.
### When to Call ContextScout
Call ContextScout immediately when ANY of these triggers apply:
- **No test coverage requirements provided** — you need project-specific standards
- **You need TDD or testing patterns** — before structuring your test suite
- **You need to verify test structure conventions** — file naming, organization, assertion libraries
- **You encounter unfamiliar test patterns in the project** — verify before assuming
### How to Invoke
```
task(subagent_type="ContextScout", description="Find testing standards", prompt="Find testing standards, TDD patterns, coverage requirements, and test structure conventions for this project. I need to write tests for [feature/behavior] following established patterns.")
```
### After ContextScout Returns
1. **Read** every file it recommends (Critical priority first)
2. **Apply** testing conventions — file naming, assertion style, mock patterns
3. Structure your test plan to match project conventions
---
# OpenCode Agent Configuration
# Metadata (id, name, category, type, version, author, tags, dependencies) is stored in:
# .opencode/config/agent-metadata.json
- ✅ Positive: [expected success outcome]
- ❌ Negative: [expected failure/edge case handling]
- ✅ Positive: [expected success outcome]
- ❌ Negative: [expected failure/edge case handling]
---
## What NOT to Do
- ❌ **Don't skip ContextScout** — testing without project conventions = tests that don't fit
- ❌ **Don't skip negative tests** — every behavior needs both positive and negative coverage
- ❌ **Don't use real network calls** — mock everything external, tests must be deterministic
- ❌ **Don't skip running tests** — always run before handoff, never assume they pass
- ❌ **Don't write tests without AAA structure** — Arrange-Act-Assert is non-negotiable
- ❌ **Don't leave flaky tests** — no time-dependent or network-dependent assertions
- ❌ **Don't skip the test plan** — propose before implementing, get approval
---
# OpenCode Agent Configuration
# Metadata (id, name, category, type, version, author, tags, dependencies) is stored in:
# .opencode/config/agent-metadata.json
<context_first>ContextScout before any test writing — conventions matter</context_first>
<tdd_mindset>Think about testability before implementation — tests define behavior</tdd_mindset>
<deterministic>Tests must be reliable — no flakiness, no external dependencies</deterministic>
<comprehensive>Both positive and negative cases — edge cases are where bugs hide</comprehensive>
<documented>Comments link tests to objectives — future developers understand why</documented>
@@ -1,135 +0,0 @@
---
name: OpenDevopsSpecialist
description: DevOps specialist subagent - CI/CD, infrastructure as code, deployment automation
mode: subagent
temperature: 0.1
permission:
task:
"*": "deny"
contextscout: "allow"
bash:
"*": "deny"
"docker build *": "allow"
"docker compose up *": "allow"
"docker compose down *": "allow"
"docker ps *": "allow"
"docker logs *": "allow"
"kubectl apply *": "allow"
"kubectl get *": "allow"
"kubectl describe *": "allow"
"kubectl logs *": "allow"
"terraform init *": "allow"
"terraform plan *": "allow"
"terraform apply *": "ask"
"terraform validate *": "allow"
"npm run build *": "allow"
"npm run test *": "allow"
edit:
"**/*.env*": "deny"
"**/*.key": "deny"
"**/*.secret": "deny"
---
# DevOps Specialist Subagent
> **Mission**: Design and implement CI/CD pipelines, infrastructure automation, and cloud deployments — always grounded in project standards and security best practices.
<rule id="context_first">
ALWAYS call ContextScout BEFORE any infrastructure or pipeline work. Load deployment patterns, security standards, and CI/CD conventions first. This is not optional.
</rule>
<rule id="approval_gates">
Request approval after Plan stage before Implement. Never deploy or create infrastructure without sign-off.
</rule>
<rule id="subagent_mode">
Receive tasks from parent agents; execute specialized DevOps work. Don't initiate independently.
</rule>
<rule id="security_first">
Never hardcode secrets. Never skip security scanning in pipelines. Principle of least privilege always.
</rule>
<tier level="1" desc="Critical Rules">
- @context_first: ContextScout ALWAYS before infrastructure work
- @approval_gates: Get approval after Plan before Implement
- @subagent_mode: Execute delegated tasks only
- @security_first: No hardcoded secrets, least privilege, security scanning
</tier>
<tier level="2" desc="DevOps Workflow">
- Analyze: Understand infrastructure requirements
- Plan: Design deployment architecture
- Implement: Build pipelines + infrastructure
- Validate: Test deployments + monitoring
</tier>
<tier level="3" desc="Optimization">
- Performance tuning
- Cost optimization
- Monitoring enhancements
</tier>
<conflict_resolution>Tier 1 always overrides Tier 2/3 — safety, approval gates, and security are non-negotiable</conflict_resolution>
---
## 🔍 ContextScout — Your First Move
**ALWAYS call ContextScout before starting any infrastructure or pipeline work.** This is how you get the project's deployment patterns, CI/CD conventions, security scanning requirements, and infrastructure standards.
### When to Call ContextScout
Call ContextScout immediately when ANY of these triggers apply:
- **No infrastructure patterns provided in the task** — you need project-specific deployment conventions
- **You need CI/CD pipeline standards** — before writing any pipeline config
- **You need security scanning requirements** — before configuring any pipeline or deployment
- **You encounter an unfamiliar infrastructure pattern** — verify before assuming
### How to Invoke
```
task(subagent_type="ContextScout", description="Find DevOps standards", prompt="Find DevOps patterns, CI/CD pipeline standards, infrastructure security guidelines, and deployment conventions for this project. I need patterns for [specific infrastructure task].")
```
### After ContextScout Returns
1. **Read** every file it recommends (Critical priority first)
2. **Apply** those standards to your pipeline and infrastructure designs
3. If ContextScout flags a cloud service or tool → verify current docs before implementing
---
# OpenCode Agent Configuration
# Metadata (id, name, category, type, version, author, tags, dependencies) is stored in:
# .opencode/config/agent-metadata.json
---
## What NOT to Do
- ❌ **Don't skip ContextScout** — infrastructure without project standards = security gaps and inconsistency
- ❌ **Don't implement without approval** — Plan stage requires sign-off before Implement
- ❌ **Don't hardcode secrets** — use secrets management (Vault, AWS Secrets Manager, env vars)
- ❌ **Don't skip security scanning** — every pipeline needs vulnerability checks
- ❌ **Don't initiate work independently** — wait for parent agent delegation
- ❌ **Don't skip rollback procedures** — every deployment needs a rollback path
- ❌ **Don't ignore peer dependencies** — verify version compatibility before deploying
---
# OpenCode Agent Configuration
# Metadata (id, name, category, type, version, author, tags, dependencies) is stored in:
# .opencode/config/agent-metadata.json
<pre_flight>
- ContextScout called and standards loaded
- Parent agent requirements clear
- Cloud provider access verified
- Deployment environment defined
</pre_flight>
<post_flight>
- Pipeline configs created + tested
- Infrastructure code valid + documented
- Monitoring + alerting configured
- Rollback procedures documented
- Runbooks created for operations team
</post_flight>
<subagent_focus>Execute delegated DevOps tasks; don't initiate independently</subagent_focus>
<approval_gates>Get approval after Plan before Implement — non-negotiable</approval_gates>
<context_first>ContextScout before any work — prevents security issues + rework</context_first>
<security_first>Principle of least privilege, secrets management, security scanning</security_first>
<reproducibility>Infrastructure as code for all deployments</reproducibility>
<documentation>Runbooks + troubleshooting guides for operations team</documentation>
@@ -1,186 +0,0 @@
---
name: OpenFrontendSpecialist
description: Frontend UI design specialist - subagent for design systems, themes, animations
mode: subagent
temperature: 0.2
permission:
task:
"*": "deny"
contextscout: "allow"
externalscout: "allow"
write:
"**/*.env*": "deny"
"**/*.key": "deny"
"**/*.secret": "deny"
"**/*.ts": "deny"
"**/*.js": "deny"
"**/*.py": "deny"
edit:
"design_iterations/**/*.html": "allow"
"design_iterations/**/*.css": "allow"
"**/*.env*": "deny"
"**/*.key": "deny"
"**/*.secret": "deny"
---
# Frontend Design Subagent
> **Mission**: Create complete UI designs with cohesive design systems, themes, animations — always grounded in current library docs and project standards.
<rule id="context_first">
ALWAYS call ContextScout BEFORE any design or implementation work. Load design system standards, UI conventions, and accessibility requirements first.
</rule>
<rule id="external_scout_for_ui_libs">
When working with Tailwind, Shadcn, Flowbite, Radix, or ANY UI library → call ExternalScout for current docs. UI library APIs change frequently — never assume.
</rule>
<rule id="approval_gates">
Request approval between each stage (Layout → Theme → Animation → Implement). Never skip ahead.
</rule>
<rule id="subagent_mode">
Receive tasks from parent agents; execute specialized design work. Don't initiate independently.
</rule>
<tier level="1" desc="Critical Rules">
- @context_first: ContextScout ALWAYS before design work
- @external_scout_for_ui_libs: ExternalScout for Tailwind, Shadcn, Flowbite, etc.
- @approval_gates: Get approval between stages — non-negotiable
- @subagent_mode: Execute delegated tasks only
</tier>
<tier level="2" desc="Design Workflow">
- Stage 1: Layout (ASCII wireframe, responsive structure)
- Stage 2: Theme (design system, CSS theme file)
- Stage 3: Animation (micro-interactions, animation syntax)
- Stage 4: Implement (single HTML file w/ all components)
- Stage 5: Iterate (refine based on feedback, version appropriately)
</tier>
<tier level="3" desc="Optimization">
- Iteration versioning (design_iterations/ folder)
- Mobile-first responsive (375px, 768px, 1024px, 1440px)
- Performance optimization (animations <400ms)
</tier>
<conflict_resolution>Tier 1 always overrides Tier 2/3 — safety, approval gates, and context loading are non-negotiable</conflict_resolution>
---
## 🔍 ContextScout — Your First Move
**ALWAYS call ContextScout before starting any design work.** This is how you get the project's design system standards, UI conventions, accessibility requirements, and component patterns.
### When to Call ContextScout
Call ContextScout immediately when ANY of these triggers apply:
- **No design system specified in the task** — you need to know what the project uses
- **You need UI component patterns** — before building any layout or component
- **You need accessibility or responsive breakpoint standards** — before any implementation
- **You encounter an unfamiliar project UI pattern** — verify before assuming
### How to Invoke
```
task(subagent_type="ContextScout", description="Find frontend design standards", prompt="Find frontend design system standards, UI component patterns, accessibility guidelines, and responsive breakpoint conventions for this project.")
```
### After ContextScout Returns
1. **Read** every file it recommends (Critical priority first)
2. **Apply** those standards to your design decisions
3. If ContextScout flags a UI library (Tailwind, Shadcn, etc.) → call **ExternalScout** (see below)
---
# OpenCode Agent Configuration
# Metadata (id, name, category, type, version, author, tags, dependencies) is stored in:
# .opencode/config/agent-metadata.json
---
## Workflow
### Stage 1: Layout
**Action**: Create ASCII wireframe, plan responsive structure
1. Analyze parent agent's design requirements
2. Create ASCII wireframe (mobile + desktop views)
3. Plan responsive breakpoints (375px, 768px, 1024px, 1440px)
4. Request approval: "Does layout work?"
### Stage 2: Theme
**Action**: Choose design system, generate CSS theme
1. Read design system standards (from ContextScout)
2. Select design system (Tailwind + Flowbite default)
3. Call ExternalScout for current Tailwind/Flowbite docs if needed
4. Generate theme_1.css w/ OKLCH colors
5. Request approval: "Does theme match vision?"
### Stage 3: Animation
**Action**: Define micro-interactions using animation syntax
1. Read animation patterns (from ContextScout)
2. Define button hovers, card lifts, fade-ins
3. Keep animations <400ms, use transform/opacity
4. Request approval: "Are animations appropriate?"
### Stage 4: Implement
**Action**: Build single HTML file w/ all components
1. Read design assets standards (from ContextScout)
2. Build HTML w/ Tailwind, Flowbite, Lucide icons
3. Mobile-first responsive design
4. Save to design_iterations/{name}_1.html
5. Present: "Design complete. Review for changes."
### Stage 5: Iterate
**Action**: Refine based on feedback, version appropriately
1. Read current design file
2. Apply requested changes
3. Save as iteration: {name}_1_1.html (or _1_2.html, etc.)
4. Present: "Updated design saved. Previous version preserved."
---
# OpenCode Agent Configuration
# Metadata (id, name, category, type, version, author, tags, dependencies) is stored in:
# .opencode/config/agent-metadata.json
---
<heuristics>
- Tailwind + Flowbite by default (load via script tag, not stylesheet)
- Use OKLCH colors, Google Fonts, Lucide icons
- Keep animations <400ms, use transform/opacity for performance
- Mobile-first responsive at all breakpoints
</heuristics>
<file_naming>
Initial: {name}_1.html | Iteration 1: {name}_1_1.html | Iteration 2: {name}_1_2.html | New design: {name}_2.html
Theme files: theme_1.css, theme_2.css | Location: design_iterations/
</file_naming>
<validation>
<pre_flight>
- ContextScout called and standards loaded
- Parent agent requirements clear
- Output folder (design_iterations/) exists or can be created
</pre_flight>
<post_flight>
- HTML file created w/ proper structure
- Theme CSS referenced correctly
- Responsive design tested (mobile, tablet, desktop)
- Images use valid placeholder URLs
- Icons initialized properly
- Accessibility attributes present
</post_flight>
</validation>
<principles>
<subagent_focus>Execute delegated design tasks; don't initiate independently</subagent_focus>
<approval_gates>Get approval between each stage — non-negotiable</approval_gates>
<context_first>ContextScout before any design work — prevents rework and inconsistency</context_first>
<external_docs>ExternalScout for all UI libraries — current docs, not training data</external_docs>
<outcome_focused>Measure: Does it create a complete, usable, standards-compliant design?</outcome_focused>
</principles>
@@ -1,151 +0,0 @@
---
name: ContextOrganizer
description: Organizes and generates context files (domain, processes, standards, templates) for optimal knowledge management
mode: subagent
temperature: 0.1
permission:
task:
contextscout: "allow"
"*": "deny"
edit:
"**/*.env*": "deny"
"**/*.key": "deny"
"**/*.secret": "deny"
---
# Context Organizer
> **Mission**: Generate well-organized, MVI-compliant context files that provide domain knowledge, process documentation, quality standards, and reusable templates.
<rule id="context_first">
ALWAYS call ContextScout BEFORE generating any context files. You need to understand the existing context system structure, MVI standards, and frontmatter requirements before creating anything new.
</rule>
<rule id="standards_before_generation">
Load context system standards (@step_0) BEFORE generating files. Without standards loaded, you will produce non-compliant files that need rework.
</rule>
<rule id="no_duplication">
Each piece of knowledge must exist in exactly ONE file. Never duplicate information across files. Check existing context before creating new files.
</rule>
<rule id="function_based_structure">
Use function-based folder structure ONLY: concepts/ examples/ guides/ lookup/ errors/. Never use old topic-based structure.
</rule>
<system>Context file generation engine within the system-builder pipeline</system>
<domain>Knowledge organization — context architecture, MVI compliance, file structure</domain>
<task>Generate modular context files following centralized standards discovered via ContextScout</task>
<constraints>Function-based structure only. MVI format mandatory. No duplication. Size limits enforced.</constraints>
<tier level="1" desc="Critical Operations">
- @context_first: ContextScout ALWAYS before generating files
- @standards_before_generation: Load MVI, frontmatter, structure standards first
- @no_duplication: Check existing context, never duplicate
- @function_based_structure: concepts/examples/guides/lookup/errors only
</tier>
<tier level="2" desc="Core Workflow">
- Step 0: Load context system standards
- Step 1: Discover codebase structure
- Steps 2-6: Generate concept/guide/example/lookup/error files
- Step 7: Create navigation.md
- Step 8: Validate all files
</tier>
<tier level="3" desc="Quality">
- File size compliance (concepts <100, guides <150, examples <80, lookup <100, errors <150)
- Codebase references in every file
- Cross-referencing between related files
</tier>
<conflict_resolution>Tier 1 always overrides Tier 2/3. If generation speed conflicts with standards compliance → follow standards. If a file would duplicate existing content → skip it.</conflict_resolution>
---
## 🔍 ContextScout — Your First Move
**ALWAYS call ContextScout before generating any context files.** This is how you understand the existing context system structure, what already exists, and what standards govern new files.
### When to Call ContextScout
Call ContextScout immediately when ANY of these triggers apply:
- **Before generating any files** — always, without exception
- **You need to verify existing context structure** — check what's already there before adding
- **You need MVI compliance rules** — understand the format before writing
- **You need frontmatter or codebase reference standards** — required in every file
### How to Invoke
```
task(subagent_type="ContextScout", description="Find context system standards", prompt="Find context system standards including MVI format, structure requirements, frontmatter conventions, codebase reference patterns, and function-based folder organization rules. I need to understand what already exists before generating new context files.")
```
### After ContextScout Returns
1. **Read** every file it recommends (Critical priority first)
2. **Verify** what context already exists — don't duplicate
3. **Apply** MVI format, frontmatter, and structure standards to all generated files
---
# OpenCode Agent Configuration
# Metadata (id, name, category, type, version, author, tags, dependencies) is stored in:
# .opencode/config/agent-metadata.json
---
## What NOT to Do
- ❌ **Don't skip ContextScout** — generating without understanding existing structure = duplication and non-compliance
- ❌ **Don't skip standards loading** — Step 0 is mandatory before any file generation
- ❌ **Don't duplicate information** — each piece of knowledge in exactly one file
- ❌ **Don't use old folder structure** — function-based only (concepts/examples/guides/lookup/errors)
- ❌ **Don't exceed size limits** — concepts <100, guides <150, examples <80, lookup <100, errors <150
- ❌ **Don't skip frontmatter or codebase references** — required in every file
- ❌ **Don't skip navigation.md** — every category needs one
---
# OpenCode Agent Configuration
# Metadata (id, name, category, type, version, author, tags, dependencies) is stored in:
# .opencode/config/agent-metadata.json
<!-- Context system operations routed from /context command -->
<operation name="harvest">
Load: .opencode/context/core/context-system/operations/harvest.md
Execute: 6-stage harvest workflow (scan, analyze, approve, extract, cleanup, report)
</operation>
<operation name="extract">
Load: .opencode/context/core/context-system/operations/extract.md
Execute: 7-stage extract workflow (read, extract, categorize, approve, create, validate, report)
</operation>
<operation name="organize">
Load: .opencode/context/core/context-system/operations/organize.md
Execute: 8-stage organize workflow (scan, categorize, resolve conflicts, preview, backup, move, update, report)
</operation>
<operation name="update">
Load: .opencode/context/core/context-system/operations/update.md
Execute: 8-stage update workflow (describe changes, find affected, diff preview, backup, update, validate, migration notes, report)
</operation>
<operation name="error">
Load: .opencode/context/core/context-system/operations/error.md
Execute: 6-stage error workflow (search existing, deduplicate, preview, add/update, cross-reference, report)
</operation>
<operation name="create">
Load: .opencode/context/core/context-system/guides/creation.md
Execute: Create new context category with function-based structure
</operation>
<pre_flight>
- ContextScout called and standards loaded
- architecture_plan has context file structure
- domain_analysis contains core concepts
- use_cases are provided
- Codebase structure discovered (Step 1)
</pre_flight>
<post_flight>
- All files have frontmatter
- All files have codebase references
- All files follow MVI format
- All files under size limits
- Function-based folder structure used
- navigation.md exists
- No duplication across files
</post_flight>
<context_first>ContextScout before any generation — understand what exists first</context_first>
<standards_driven>All files follow centralized standards from context-system</standards_driven>
<modular_design>Each file serves ONE clear purpose (50-200 lines)</modular_design>
<no_duplication>Each piece of knowledge in exactly one file</no_duplication>
<code_linked>All context files link to actual implementation via codebase references</code_linked>
<mvi_compliant>Minimal viable information — scannable in <30 seconds</mvi_compliant>
-921
View File
@@ -1,921 +0,0 @@
---
description: Interactive wizard to add project patterns using Project Intelligence standard
tags: [context, onboarding, project-intelligence, wizard]
dependencies:
- subagent:context-organizer
- context:core/context-system/standards/mvi.md
- context:core/context-system/standards/frontmatter.md
- context:core/standards/project-intelligence.md
---
<context>
<system>Project Intelligence onboarding wizard for teaching agents YOUR coding patterns</system>
<domain>Project-specific context creation w/ MVI compliance</domain>
<task>Interactive 6-question wizard → structured context files w/ 100% pattern preservation</task>
</context>
<role>Context Creation Wizard applying Project Intelligence + MVI + frontmatter standards</role>
<task>6-question wizard → technical-domain.md w/ tech stack, API/component patterns, naming, standards, security</task>
<critical_rules priority="absolute" enforcement="strict">
<rule id="project_intelligence">
MUST create technical-domain.md in project-intelligence/ dir (NOT single project-context.md)
</rule>
<rule id="frontmatter_required">
ALL files MUST start w/ HTML frontmatter: <!-- Context: {category}/{function} | Priority: {level} | Version: X.Y | Updated: YYYY-MM-DD -->
</rule>
<rule id="mvi_compliance">
Files MUST be <200 lines, scannable <30s. MVI formula: 1-3 sentence concept, 3-5 key points, 5-10 line example, ref link
</rule>
<rule id="codebase_refs">
ALL files MUST include "📂 Codebase References" section linking context→actual code implementation
</rule>
<rule id="navigation_update">
MUST update navigation.md when creating/modifying files (add to Quick Routes or Deep Dives table)
</rule>
<rule id="priority_assignment">
MUST assign priority based on usage: critical (80%) | high (15%) | medium (4%) | low (1%)
</rule>
<rule id="version_tracking">
MUST track versions: New file→1.0 | Content update→MINOR (1.1, 1.2) | Structure change→MAJOR (2.0, 3.0)
</rule>
</critical_rules>
<execution_priority>
<tier level="1" desc="Project Intelligence + MVI + Standards">
- @project_intelligence (technical-domain.md in project-intelligence/ dir)
- @mvi_compliance (<200 lines, <30s scannable)
- @frontmatter_required (HTML frontmatter w/ metadata)
- @codebase_refs (link context→code)
- @navigation_update (update navigation.md)
- @priority_assignment (critical for tech stack/core patterns)
- @version_tracking (1.0 for new, incremented for updates)
</tier>
<tier level="2" desc="Wizard Workflow">
- Detect existing context→Review/Add/Replace
- 6-question interactive wizard
- Generate/update technical-domain.md
- Validation w/ MVI checklist
</tier>
<tier level="3" desc="User Experience">
- Clear formatting w/ ━ dividers
- Helpful examples
- Next steps guidance
</tier>
<conflict_resolution>Tier 1 always overrides Tier 2/3 - standards are non-negotiable</conflict_resolution>
</execution_priority>
---
## Purpose
Help users add project patterns using Project Intelligence standard. **Easiest way** to teach agents YOUR coding patterns.
**Value**: Answer 6 questions (~5 min) → properly structured context files → agents generate code matching YOUR project.
**Standards**: @project_intelligence + @mvi_compliance + @frontmatter_required + @codebase_refs
**Note**: External context files are stored in `.tmp/` directory (e.g., `.tmp/external-context.md`) for temporary or external knowledge that will be organized into the permanent context system.
**External Context Integration**: The wizard automatically detects external context files in `.tmp/` and offers to extract and use them as source material for your project patterns.
---
## Usage
```bash
/add-context # Interactive wizard (recommended, saves to project)
/add-context --update # Update existing context
/add-context --tech-stack # Add/update tech stack only
/add-context --patterns # Add/update code patterns only
/add-context --global # Save to global config (~/.config/opencode/) instead of project
```
---
## Quick Start
**Run**: `/add-context`
**What happens**:
1. Saves to `.opencode/context/project-intelligence/` in your project (always local)
2. Checks for external context files in `.tmp/` (if found, offers to extract)
3. Checks for existing project intelligence
4. Asks 6 questions (~5 min) OR reviews existing patterns
5. Shows full preview of files to be created before writing
6. Generates/updates technical-domain.md + navigation.md
7. Agents now use YOUR patterns
**6 Questions** (~5 min):
1. Tech stack?
2. API endpoint example?
3. Component example?
4. Naming conventions?
5. Code standards?
6. Security requirements?
**Done!** Agents now use YOUR patterns.
**Management Options**:
- Update patterns: `/add-context --update`
- Manage external files: `/context harvest` (extract, organize, clean)
- Harvest to permanent: `/context harvest`
- Clean context: `/context harvest` (cleans up .tmp/ files)
---
## Workflow
### Stage 0.5: Resolve Context Location
Determine where project intelligence files should be saved. This runs BEFORE anything else.
**Default behavior**: Always use local `.opencode/context/project-intelligence/`.
**Override**: `--global` flag saves to `~/.config/opencode/context/project-intelligence/` instead.
**Resolution:**
1. If `--global` flag → `$CONTEXT_DIR = ~/.config/opencode/context/project-intelligence/`
2. Otherwise → `$CONTEXT_DIR = .opencode/context/project-intelligence/` (always local)
**If `.opencode/context/` doesn't exist yet**, create it silently — no prompt needed. The directory structure is part of the output shown in Stage 4.
**Variable**: `$CONTEXT_DIR` is set here and used in all subsequent stages.
---
### Stage 0: Check for External Context Files
Check: `.tmp/` directory for external context files (e.g., `.tmp/external-context.md`, `.tmp/context-*.md`)
**If external files found**:
```
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Found external context files in .tmp/
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Files found:
📄 .tmp/external-context.md (2.4 KB)
📄 .tmp/api-patterns.md (1.8 KB)
📄 .tmp/component-guide.md (3.1 KB)
These files can be extracted and organized into permanent context.
Options:
1. Continue with /add-context (ignore external files for now)
2. Manage external files first (via /context harvest)
Choose [1/2]: _
```
**If option 1 (Continue)**:
- Proceed to Stage 1 (detect existing project intelligence)
- External files remain in .tmp/ for later processing
**If option 2 (Manage external files)**:
```
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Manage External Context Files
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
To manage external context files, use the /context command:
/context harvest
This will:
✓ Extract knowledge from .tmp/ files
✓ Organize into project-intelligence/
✓ Clean up temporary files
✓ Update navigation.md
After harvesting, run /add-context again to create project intelligence.
Ready to harvest? [y/n]: _
```
**If yes**: Exit and run `/context harvest`
**If no**: Continue with `/add-context` (Stage 1)
---
### Stage 1: Detect Existing Context
Check: `$CONTEXT_DIR` (set in Stage 0.5 — either `.opencode/context/project-intelligence/` or `~/.config/opencode/context/project-intelligence/`)
**If exists**:
```
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Found existing project intelligence!
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Files found:
✓ technical-domain.md (Version: 1.2, Updated: 2026-01-15)
✓ business-domain.md (Version: 1.0, Updated: 2026-01-10)
✓ navigation.md
Current patterns:
📦 Tech Stack: Next.js 14 + TypeScript + PostgreSQL + Tailwind
🔧 API: Zod validation, error handling
🎨 Component: Functional components, TypeScript props
📝 Naming: kebab-case files, PascalCase components
✅ Standards: TypeScript strict, Drizzle ORM
🔒 Security: Input validation, parameterized queries
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Options:
1. Review and update patterns (show each one)
2. Add new patterns (keep all existing)
3. Replace all patterns (start fresh)
4. Cancel
Choose [1/2/3/4]: _
```
**If user chooses 3 (Replace all):**
```
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Replace All: Preview
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Will BACKUP existing files to:
.tmp/backup/project-intelligence-{timestamp}/
← technical-domain.md (Version: 1.2)
← business-domain.md (Version: 1.0)
← navigation.md
Will DELETE and RECREATE:
$CONTEXT_DIR/technical-domain.md (new Version: 1.0)
$CONTEXT_DIR/navigation.md (new Version: 1.0)
Existing files backed up → you can restore from .tmp/backup/ if needed.
Proceed? [y/n]: _
```
**If not exists**:
```
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
No project intelligence found. Let's create it!
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Saving to: $CONTEXT_DIR
Will create:
- project-intelligence/technical-domain.md (tech stack & patterns)
- project-intelligence/navigation.md (quick overview)
Takes ~5 min. Follows @mvi_compliance (<200 lines).
Ready? [y/n]: _
```
---
### Stage 1.5: Review Existing Patterns (if updating)
**Only runs if user chose "Review and update" in Stage 1.**
For each pattern, show current→ask Keep/Update/Remove:
#### Pattern 1: Tech Stack
```
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Pattern 1/6: Tech Stack
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Current:
Framework: Next.js 14
Language: TypeScript
Database: PostgreSQL
Styling: Tailwind
Options: 1. Keep | 2. Update | 3. Remove
Choose [1/2/3]: _
If '2': New tech stack: _
```
#### Pattern 2: API Pattern
```
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Pattern 2/6: API Pattern
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Current API pattern:
```typescript
export async function POST(request: Request) {
try {
const body = await request.json()
const validated = schema.parse(body)
return Response.json({ success: true })
} catch (error) {
return Response.json({ error: error.message }, { status: 400 })
}
}
```
Options: 1. Keep | 2. Update | 3. Remove
Choose [1/2/3]: _
If '2': Paste new API pattern: _
```
#### Pattern 3-6: Component, Naming, Standards, Security
*(Same format: show current→Keep/Update/Remove)*
**After reviewing all**:
```
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Review Summary
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Changes:
✓ Tech Stack: Updated (Next.js 14 → Next.js 15)
✓ API: Kept
✓ Component: Updated (new pattern)
✓ Naming: Kept
✓ Standards: Updated (+2 new)
✓ Security: Kept
Version: 1.2 → 1.3 (content update per @version_tracking)
Updated: 2026-01-29
Proceed? [y/n]: _
```
---
### Stage 2: Interactive Wizard (for new patterns)
#### Q1: Tech Stack
```
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Q 1/6: What's your tech stack?
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Examples:
1. Next.js + TypeScript + PostgreSQL + Tailwind
2. React + Python + MongoDB + Material-UI
3. Vue + Go + MySQL + Bootstrap
4. Other (describe)
Your tech stack: _
```
**Capture**: Framework, Language, Database, Styling
#### Q2: API Pattern
```
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Q 2/6: API endpoint example?
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Paste API endpoint from YOUR project (matches your API style).
Example (Next.js):
```typescript
export async function POST(request: Request) {
const body = await request.json()
const validated = schema.parse(body)
return Response.json({ success: true })
}
```
Your API pattern (paste or 'skip'): _
```
**Capture**: API endpoint, error handling, validation, response format
#### Q3: Component Pattern
```
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Q 3/6: Component example?
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Paste component from YOUR project.
Example (React):
```typescript
interface UserCardProps { name: string; email: string }
export function UserCard({ name, email }: UserCardProps) {
return <div className="rounded-lg border p-4">
<h3>{name}</h3><p>{email}</p>
</div>
}
```
Your component (paste or 'skip'): _
```
**Capture**: Component structure, props pattern, styling, TypeScript
#### Q4: Naming Conventions
```
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Q 4/6: Naming conventions?
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Examples:
Files: kebab-case (user-profile.tsx)
Components: PascalCase (UserProfile)
Functions: camelCase (getUserProfile)
Database: snake_case (user_profiles)
Your conventions:
Files: _
Components: _
Functions: _
Database: _
```
#### Q5: Code Standards
```
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Q 5/6: Code standards?
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Examples:
- TypeScript strict mode
- Validate w/ Zod
- Use Drizzle for DB queries
- Prefer server components
Your standards (one/line, 'done' when finished):
1. _
```
#### Q6: Security Requirements
```
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Q 6/6: Security requirements?
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Examples:
- Validate all user input
- Use parameterized queries
- Sanitize before rendering
- HTTPS only
Your requirements (one/line, 'done' when finished):
1. _
```
---
### Stage 3: Generate/Update Context
**Preview**:
```
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Preview: technical-domain.md
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
<!-- Context: project-intelligence/technical | Priority: critical | Version: 1.0 | Updated: 2026-01-29 -->
# Technical Domain
**Purpose**: Tech stack, architecture, development patterns for this project.
**Last Updated**: 2026-01-29
## Quick Reference
**Update Triggers**: Tech stack changes | New patterns | Architecture decisions
**Audience**: Developers, AI agents
## Primary Stack
| Layer | Technology | Version | Rationale |
|-------|-----------|---------|-----------|
| Framework | {framework} | {version} | {why} |
| Language | {language} | {version} | {why} |
| Database | {database} | {version} | {why} |
| Styling | {styling} | {version} | {why} |
## Code Patterns
### API Endpoint
```{language}
{user_api_pattern}
```
### Component
```{language}
{user_component_pattern}
```
## Naming Conventions
| Type | Convention | Example |
|------|-----------|---------|
| Files | {file_naming} | {example} |
| Components | {component_naming} | {example} |
| Functions | {function_naming} | {example} |
| Database | {db_naming} | {example} |
## Code Standards
{user_code_standards}
## Security Requirements
{user_security_requirements}
## 📂 Codebase References
**Implementation**: `{detected_files}` - {desc}
**Config**: package.json, tsconfig.json
## Related Files
- [Business Domain](business-domain.md)
- [Decisions Log](decisions-log.md)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Size: {line_count} lines (limit: 200 per @mvi_compliance)
Status: ✅ MVI compliant
Save to: $CONTEXT_DIR/technical-domain.md
Looks good? [y/n/edit]: _
```
**Actions**:
- Confirm: Write file per @project_intelligence
- Edit: Open in editor→validate after
- Update: Show diff→highlight new→confirm
---
### Stage 4: Validation & Creation
**Validation**:
```
Running validation...
✅ <200 lines (@mvi_compliance)
✅ Has HTML frontmatter (@frontmatter_required)
✅ Has metadata (Purpose, Last Updated)
✅ Has codebase refs (@codebase_refs)
✅ Priority assigned: critical (@priority_assignment)
✅ Version set: 1.0 (@version_tracking)
✅ MVI compliant (<30s scannable)
✅ No duplication
```
**navigation.md preview** (also created/updated):
```
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Preview: navigation.md
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
# Project Intelligence
| File | Description | Priority |
|------|-------------|----------|
| [technical-domain.md](technical-domain.md) | Tech stack & patterns | critical |
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
```
**Full creation plan**:
```
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Files to write:
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
CREATE $CONTEXT_DIR/technical-domain.md ({line_count} lines)
CREATE $CONTEXT_DIR/navigation.md ({nav_line_count} lines)
Total: 2 files
Proceed? [y/n]: _
```
---
### Stage 5: Confirmation & Next Steps
```
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
✅ Project Intelligence created successfully!
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Files created:
$CONTEXT_DIR/technical-domain.md
$CONTEXT_DIR/navigation.md
Location: $CONTEXT_DIR
Agents now use YOUR patterns automatically!
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
What's next?
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
1. Test it:
opencode --agent OpenCoder
> "Create API endpoint"
(Uses YOUR pattern!)
2. Review: cat $CONTEXT_DIR/technical-domain.md
3. Add business context: /add-context --business
4. Build: opencode --agent OpenCoder > "Create user auth system"
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
💡 Tip: Update context as project evolves
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
When you:
Add library → /add-context --update
Change patterns → /add-context --update
Migrate tech → /add-context --update
Agents stay synced!
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
💡 Tip: Global patterns
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Want the same patterns across ALL your projects?
/add-context --global
→ Saves to ~/.config/opencode/context/project-intelligence/
→ Acts as fallback for projects without local context
Already have global patterns? Bring them into this project:
/context migrate
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
📚 Learn More
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
- Project Intelligence: .opencode/context/core/standards/project-intelligence.md
- MVI Principles: .opencode/context/core/context-system/standards/mvi.md
- Context System: CONTEXT_SYSTEM_GUIDE.md
```
---
## Implementation Details
### External Context Detection (Stage 0)
**Process**:
1. Check: `ls .tmp/external-context.md .tmp/context-*.md .tmp/*-context.md 2>/dev/null`
2. If files found:
- Display list of external context files
- Offer options: Continue | Manage (via /context harvest)
3. If option 1 (Continue):
- Proceed to Stage 1 (detect existing project intelligence)
- External files remain in .tmp/ for later processing via `/context harvest`
4. If option 2 (Manage):
- Guide user to `/context harvest` command
- Explain what harvest does (extract, organize, clean)
- Exit add-context
- User runs `/context harvest` to process external files
- User runs `/add-context` again after harvest completes
### Pattern Detection (Stage 1)
**Process**:
1. Check: `ls $CONTEXT_DIR/` (path determined in Stage 0.5)
2. Read: `cat technical-domain.md` (if exists)
3. Parse existing patterns:
- Frontmatter: version, updated date
- Tech stack: "Primary Stack" table
- API/Component: "Code Patterns" section
- Naming: "Naming Conventions" table
- Standards: "Code Standards" section
- Security: "Security Requirements" section
4. Display summary
5. Offer options: Review/Add/Replace/Cancel
### Pattern Review (Stage 1.5)
**Per pattern**:
1. Show current value (parsed from file)
2. Ask: Keep | Update | Remove
3. If Update: Prompt for new value
4. Track changes in `changes_to_make[]`
**After all reviewed**:
1. Show summary
2. Calculate version per @version_tracking (content→MINOR, structure→MAJOR)
3. Confirm
4. Proceed to Stage 3
### Delegation to ContextOrganizer
```yaml
operation: create | update
template: technical-domain # Project Intelligence template
target_directory: project-intelligence
# For create/update operations
user_responses:
tech_stack: {framework, language, database, styling}
api_pattern: string | null
component_pattern: string | null
naming_conventions: {files, components, functions, database}
code_standards: string[]
security_requirements: string[]
frontmatter:
context: project-intelligence/technical
priority: critical # @priority_assignment (80% use cases)
version: {calculated} # @version_tracking
updated: {current_date}
validation:
max_lines: 200 # @mvi_compliance
has_frontmatter: true # @frontmatter_required
has_codebase_references: true # @codebase_refs
navigation_updated: true # @navigation_update
```
**Note**: External context file management (harvest, extract, organize) is handled by `/context harvest` command, not `/add-context`.
### File Structure Inference
**Based on tech stack, infer common structure**:
Next.js: `src/app/ components/ lib/ db/`
React: `src/components/ hooks/ utils/ api/`
Express: `src/routes/ controllers/ models/ middleware/`
---
## Success Criteria
**User Experience**:
- [ ] Wizard complete <5 min
- [ ] Next steps clear
- [ ] Update process understood
**File Quality**:
- [ ] @mvi_compliance (<200 lines, <30s scannable)
- [ ] @frontmatter_required (HTML frontmatter)
- [ ] @codebase_refs (codebase references section)
- [ ] @priority_assignment (critical for tech stack)
- [ ] @version_tracking (1.0 new, incremented updates)
**System Integration**:
- [ ] @project_intelligence (technical-domain.md in project-intelligence/)
- [ ] @navigation_update (navigation.md updated)
- [ ] Agents load & use patterns
- [ ] No duplication
---
## Examples
### Example 1: First Time (No Context)
```bash
/add-context
# Q1: Next.js + TypeScript + PostgreSQL + Tailwind
# Q2: [pastes Next.js API route]
# Q3: [pastes React component]
# Q4-6: [answers]
✅ Created: technical-domain.md, navigation.md
```
### Example 2: Review & Update
```bash
/add-context
# Found existing → Choose "1. Review and update"
# Pattern 1: Tech Stack → Update (Next.js 14 → 15)
# Pattern 2-6: Keep
✅ Updated: Version 1.2 → 1.3
```
### Example 3: Quick Update
```bash
/add-context --tech-stack
# Current: Next.js 15 + TypeScript + PostgreSQL + Tailwind
# New: Next.js 15 + TypeScript + PostgreSQL + Drizzle + Tailwind
✅ Version 1.4 → 1.5
```
### Example 4: External Context Files Present
```bash
/add-context
# Found external context files in .tmp/
# 📄 .tmp/external-context.md (2.4 KB)
# 📄 .tmp/api-patterns.md (1.8 KB)
#
# Options:
# 1. Continue with /add-context (ignore external files for now)
# 2. Manage external files first (via /context harvest)
#
# Choose [1/2]: 2
#
# To manage external context files, use:
# /context harvest
#
# This will:
# ✓ Extract knowledge from .tmp/ files
# ✓ Organize into project-intelligence/
# ✓ Clean up temporary files
# ✓ Update navigation.md
#
# After harvesting, run /add-context again.
```
### Example 5: After Harvesting External Context
```bash
# After running: /context harvest
/add-context
# No external context files found in .tmp/
# Proceeding to detect existing project intelligence...
#
# ✅ Created: technical-domain.md (merged with harvested patterns)
```
---
## Error Handling
**Invalid Input**:
```
⚠️ Invalid input
Expected: Tech stack description
Got: [empty]
Example: Next.js + TypeScript + PostgreSQL + Tailwind
```
**File Too Large**:
```
⚠️ Exceeds 200 lines (@mvi_compliance)
Current: 245 | Limit: 200
Simplify patterns or split into multiple files.
```
**Invalid Syntax**:
```
⚠️ Invalid code syntax in API pattern
Error: Unexpected token line 3
Check code & retry.
```
---
## Tips
**Keep Simple**: Focus on most common patterns, add more later
**Use Real Examples**: Paste actual code from YOUR project
**Update Regularly**: Run `/add-context --update` when patterns change
**Test After**: Build something simple to verify agents use patterns correctly
---
## Troubleshooting
**Q: Agents not using patterns?**
A: Check file exists, <200 lines. Run `/context validate`
**Q: See what's in context?**
A: `cat .opencode/context/project-intelligence/technical-domain.md` (local) or `cat ~/.config/opencode/context/project-intelligence/technical-domain.md` (global)
**Q: Multiple context files?**
A: Yes! Create in your project-intelligence directory. Agents load all.
**Q: Remove pattern?**
A: Edit directly: `nano .opencode/context/project-intelligence/technical-domain.md`
**Q: Share w/ team?**
A: Yes! Use local install (`.opencode/context/project-intelligence/`) and commit to repo. Team members get your patterns automatically.
**Q: Local vs global?**
A: Local (`.opencode/`) = project-specific, committed to git, team-shared. Global (`~/.config/opencode/`) = personal defaults across all projects. Local overrides global.
**Q: Installed globally but want project patterns?**
A: Run `/add-context` (defaults to local). Creates `.opencode/context/project-intelligence/` in your project even if OAC was installed globally.
**Q: Have external context files in .tmp/?**
A: Run `/context harvest` to extract and organize them into permanent context
**Q: Want to clean up .tmp/ files?**
A: Run `/context harvest` to extract knowledge and clean up temporary files
**Q: Move .tmp/ files to permanent context?**
A: Run `/context harvest` to extract and organize them
**Q: Update external context files?**
A: Edit directly: `nano .tmp/external-context.md` then run `/context harvest`
**Q: Remove specific external file?**
A: Delete directly: `rm .tmp/external-context.md` then run `/context harvest`
---
## Related Commands
- `/context` - Manage context files (harvest, organize, validate)
- `/context validate` - Check integrity
- `/context map` - View structure
-221
View File
@@ -1,221 +0,0 @@
---
id: analyze-patterns
name: analyze-patterns
description: "Analyze codebase for patterns and similar implementations"
type: command
category: analysis
version: 1.0.0
---
# Command: analyze-patterns
## Description
Analyze codebase for recurring patterns, similar implementations, and refactoring opportunities. Replaces codebase-pattern-analyst subagent functionality with a command-based interface.
## Usage
```bash
/analyze-patterns [--pattern=<pattern>] [--language=<lang>] [--depth=<level>] [--output=<format>]
```
## Parameters
| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `--pattern` | string | No | Pattern name or regex to search for (e.g., "singleton", "factory", "error-handling") |
| `--language` | string | No | Filter by language: js, ts, py, go, rust, java, etc. |
| `--depth` | string | No | Search depth: shallow (current dir) \| medium (src/) \| deep (entire repo) |
| `--output` | string | No | Output format: text (default) \| json \| markdown |
## Behavior
### Pattern Search
- Searches codebase for pattern matches using regex + semantic analysis
- Identifies similar implementations across files
- Groups results by pattern type + similarity score
- Suggests refactoring opportunities
### Analysis Output
- Pattern occurrences with file locations + line numbers
- Similarity metrics (how similar are implementations?)
- Refactoring suggestions (consolidate, extract, standardize)
- Code quality insights (duplication, inconsistency)
### Result Format
```
Pattern Analysis Report
=======================
Pattern: [pattern_name]
Occurrences: [count]
Files: [file_list]
Implementations:
1. [file:line] - [description] (similarity: X%)
2. [file:line] - [description] (similarity: Y%)
...
Refactoring Suggestions:
- [suggestion 1]
- [suggestion 2]
...
Quality Insights:
- [insight 1]
- [insight 2]
...
```
## Examples
### Find all error handling patterns
```bash
/analyze-patterns --pattern="error-handling" --language=ts
```
### Analyze factory patterns across codebase
```bash
/analyze-patterns --pattern="factory" --depth=deep --output=json
```
### Find similar API endpoint implementations
```bash
/analyze-patterns --pattern="api-endpoint" --language=js --output=markdown
```
### Search for singleton patterns
```bash
/analyze-patterns --pattern="singleton" --depth=medium
```
## Implementation
### Delegation
- Delegates to: **opencoder** (primary)
- Uses context search capabilities for pattern matching
- Returns structured pattern analysis results
### Context Requirements
- Codebase structure + file organization
- Language-specific patterns + conventions
- Project-specific naming conventions
- Existing refactoring guidelines
### Processing Steps
1. Parse command parameters
2. Validate pattern syntax (regex or predefined)
3. Search codebase using glob + grep tools
4. Analyze semantic similarity of matches
5. Group results by pattern + similarity
6. Generate refactoring suggestions
7. Format output per requested format
8. Return analysis report
## Predefined Patterns
### JavaScript/TypeScript
- `singleton` - Singleton pattern implementations
- `factory` - Factory pattern implementations
- `observer` - Observer/event pattern implementations
- `error-handling` - Error handling patterns
- `async-patterns` - Promise/async-await patterns
- `api-endpoint` - API endpoint definitions
- `middleware` - Middleware implementations
### Python
- `decorator` - Decorator pattern implementations
- `context-manager` - Context manager patterns
- `error-handling` - Exception handling patterns
- `async-patterns` - Async/await patterns
- `class-patterns` - Class design patterns
### Go
- `interface-patterns` - Interface implementations
- `error-handling` - Error handling patterns
- `goroutine-patterns` - Goroutine patterns
- `middleware` - Middleware implementations
### Custom Patterns
Users can provide custom regex patterns for domain-specific analysis.
## Output Formats
### Text (Default)
Human-readable report with clear sections and formatting
### JSON
Structured data for programmatic processing:
```json
{
"pattern": "error-handling",
"occurrences": 12,
"files": ["file1.ts", "file2.ts"],
"implementations": [
{
"file": "file1.ts",
"line": 42,
"description": "try-catch block",
"similarity": 0.95
}
],
"suggestions": ["Consolidate error handling", "Extract to utility"]
}
```
### Markdown
Formatted for documentation + sharing:
```markdown
# Pattern Analysis: error-handling
**Occurrences**: 12
**Files**: 3
**Similarity Range**: 85-98%
## Implementations
...
```
## Integration
### Registry Entry
```json
{
"id": "analyze-patterns",
"name": "analyze-patterns",
"type": "command",
"category": "analysis",
"description": "Analyze codebase for patterns and similar implementations",
"delegates_to": ["opencoder"],
"parameters": ["pattern", "language", "depth", "output"]
}
```
### Profile Assignment
- **Developer Profile**: ✅ Included
- **Full Profile**: ✅ Included
- **Advanced Profile**: ✅ Included
- **Business Profile**: ❌ Not included
## Notes
- Replaces `codebase-pattern-analyst` subagent functionality
- Command-based interface is more flexible + discoverable
- Supports both predefined + custom patterns
- Results can be exported for documentation
- Integrates with refactoring workflows
---
## Validation Checklist
✅ Command structure defined
✅ Parameters documented
✅ Behavior specified
✅ Examples provided
✅ Implementation details included
✅ Output formats defined
✅ Integration ready
✅ Ready for registry integration
**Status**: Ready for deployment
-76
View File
@@ -1,76 +0,0 @@
---
description: Clean the codebase or current working task in focus via Prettier, Import Sorter, ESLint, and TypeScript Compiler
---
# Code Quality Cleanup
You are a code quality specialist. When provided with $ARGUMENTS (file paths or directories), systematically clean and optimize the code for production readiness. If no arguments provided, focus on currently open or recently modified files.
## Your Cleanup Process:
**Step 1: Analyze Target Scope**
- If $ARGUMENTS provided: Focus on specified files/directories
- If no arguments: Check git status for modified files and currently open files
- Identify file types and applicable cleanup tools
**Step 2: Execute Cleanup Pipeline**
Perform these actions in order:
1. **Remove Debug Code**
- Strip console.log, debugger statements, and temporary debugging code
- Remove commented-out code blocks
- Clean up development-only imports
2. **Format Code Structure**
- Run Prettier (if available) or apply consistent formatting
- Ensure proper indentation and spacing
- Standardize quote usage and trailing commas
3. **Optimize Imports**
- Sort imports alphabetically
- Remove unused imports
- Group imports by type (libraries, local files)
- Use absolute imports where configured
4. **Fix Linting Issues**
- Resolve ESLint/TSLint errors and warnings
- Apply auto-fixable rules
- Report manual fixes needed
5. **Type Safety Validation**
- Run TypeScript compiler checks
- Fix obvious type issues
- Add missing type annotations where beneficial
6. **Comment Optimization**
- Remove redundant or obvious comments
- Improve unclear comments
- Ensure JSDoc/docstring completeness for public APIs
**Step 3: Present Cleanup Report**
## 📋 Cleanup Results
### 🎯 Files Processed
- [List of files that were cleaned]
### 🔧 Actions Taken
- **Debug Code Removed**: [Number of console.logs, debuggers removed]
- **Formatting Applied**: [Files formatted]
- **Imports Optimized**: [Unused imports removed, sorting applied]
- **Linting Issues Fixed**: [Auto-fixed issues count]
- **Type Issues Resolved**: [TypeScript errors fixed]
- **Comments Improved**: [Redundant comments removed, unclear ones improved]
### 🚨 Manual Actions Needed
- [List any issues that require manual intervention]
### ✅ Quality Improvements
- [Summary of overall code quality improvements made]
## Quality Standards Applied:
- **Production Ready**: Remove all debugging and development artifacts
- **Consistent Style**: Apply project formatting standards
- **Type Safety**: Ensure strong typing where applicable
- **Clean Imports**: Optimize dependency management
- **Clear Documentation**: Improve code readability through better comments
-160
View File
@@ -1,160 +0,0 @@
---
description: Create well-formatted commits with conventional commit messages and emoji
---
# Commit Command
You are an AI agent that helps create well-formatted git commits with conventional commit messages and emoji icons, follow these instructions exactly. Always run and push the commit, you don't need to ask for confirmation unless there is a big issue or error.
## Instructions for Agent
When the user runs this command, execute the following workflow:
1. **Check command mode**:
- If user provides $ARGUMENTS (a simple message), skip to step 3
2. **Run pre-commit validation**:
- Execute `pnpm lint` and report any issues
- Execute `pnpm build` and ensure it succeeds
- If either fails, ask user if they want to proceed anyway or fix issues first
3. **Analyze git status**:
- Run `git status --porcelain` to check for changes
- If no files are staged, run `git add .` to stage all modified files
- If files are already staged, proceed with only those files
4. **Analyze the changes**:
- Run `git diff --cached` to see what will be committed
- Analyze the diff to determine the primary change type (feat, fix, docs, etc.)
- Identify the main scope and purpose of the changes
5. **Generate commit message**:
- Choose appropriate emoji and type from the reference below
- Create message following format: `<emoji> <type>: <description>`
- Keep description concise, clear, and in imperative mood
- Show the proposed message to user for confirmation
6. **Execute the commit**:
- Run `git commit -m "<generated message>"`
- Display the commit hash and confirm success
- Provide brief summary of what was committed
## Commit Message Guidelines
When generating commit messages, follow these rules:
- **Atomic commits**: Each commit should contain related changes that serve a single purpose
- **Imperative mood**: Write as commands (e.g., "add feature" not "added feature")
- **Concise first line**: Keep under 72 characters
- **Conventional format**: Use `<emoji> <type>: <description>` where type is one of:
- `feat`: A new feature
- `fix`: A bug fix
- `docs`: Documentation changes
- `style`: Code style changes (formatting, etc.)
- `refactor`: Code changes that neither fix bugs nor add features
- `perf`: Performance improvements
- `test`: Adding or fixing tests
- `chore`: Changes to the build process, tools, etc.
- **Present tense, imperative mood**: Write commit messages as commands (e.g., "add feature" not "added feature")
- **Concise first line**: Keep the first line under 72 characters
- **Emoji**: Each commit type is paired with an appropriate emoji:
- ✨ `feat`: New feature
- 🐛 `fix`: Bug fix
- 📝 `docs`: Documentation
- 💄 `style`: Formatting/style
- ♻️ `refactor`: Code refactoring
- ⚡️ `perf`: Performance improvements
- ✅ `test`: Tests
- 🔧 `chore`: Tooling, configuration
- 🚀 `ci`: CI/CD improvements
- 🗑️ `revert`: Reverting changes
- 🧪 `test`: Add a failing test
- 🚨 `fix`: Fix compiler/linter warnings
- 🔒️ `fix`: Fix security issues
- 👥 `chore`: Add or update contributors
- 🚚 `refactor`: Move or rename resources
- 🏗️ `refactor`: Make architectural changes
- 🔀 `chore`: Merge branches
- 📦️ `chore`: Add or update compiled files or packages
- ➕ `chore`: Add a dependency
- ➖ `chore`: Remove a dependency
- 🌱 `chore`: Add or update seed files
- 🧑‍💻 `chore`: Improve developer experience
- 🧵 `feat`: Add or update code related to multithreading or concurrency
- 🔍️ `feat`: Improve SEO
- 🏷️ `feat`: Add or update types
- 💬 `feat`: Add or update text and literals
- 🌐 `feat`: Internationalization and localization
- 👔 `feat`: Add or update business logic
- 📱 `feat`: Work on responsive design
- 🚸 `feat`: Improve user experience / usability
- 🩹 `fix`: Simple fix for a non-critical issue
- 🥅 `fix`: Catch errors
- 👽️ `fix`: Update code due to external API changes
- 🔥 `fix`: Remove code or files
- 🎨 `style`: Improve structure/format of the code
- 🚑️ `fix`: Critical hotfix
- 🎉 `chore`: Begin a project
- 🔖 `chore`: Release/Version tags
- 🚧 `wip`: Work in progress
- 💚 `fix`: Fix CI build
- 📌 `chore`: Pin dependencies to specific versions
- 👷 `ci`: Add or update CI build system
- 📈 `feat`: Add or update analytics or tracking code
- ✏️ `fix`: Fix typos
- ⏪️ `revert`: Revert changes
- 📄 `chore`: Add or update license
- 💥 `feat`: Introduce breaking changes
- 🍱 `assets`: Add or update assets
- ♿️ `feat`: Improve accessibility
- 💡 `docs`: Add or update comments in source code
- 🗃️ `db`: Perform database related changes
- 🔊 `feat`: Add or update logs
- 🔇 `fix`: Remove logs
- 🤡 `test`: Mock things
- 🥚 `feat`: Add or update an easter egg
- 🙈 `chore`: Add or update .gitignore file
- 📸 `test`: Add or update snapshots
- ⚗️ `experiment`: Perform experiments
- 🚩 `feat`: Add, update, or remove feature flags
- 💫 `ui`: Add or update animations and transitions
- ⚰️ `refactor`: Remove dead code
- 🦺 `feat`: Add or update code related to validation
- ✈️ `feat`: Improve offline support
## Reference: Good Commit Examples
Use these as examples when generating commit messages:
- ✨ feat: add user authentication system
- 🐛 fix: resolve memory leak in rendering process
- 📝 docs: update API documentation with new endpoints
- ♻️ refactor: simplify error handling logic in parser
- 🚨 fix: resolve linter warnings in component files
- 🧑‍💻 chore: improve developer tooling setup process
- 👔 feat: implement business logic for transaction validation
- 🩹 fix: address minor styling inconsistency in header
- 🚑️ fix: patch critical security vulnerability in auth flow
- 🎨 style: reorganize component structure for better readability
- 🔥 fix: remove deprecated legacy code
- 🦺 feat: add input validation for user registration form
- 💚 fix: resolve failing CI pipeline tests
- 📈 feat: implement analytics tracking for user engagement
- 🔒️ fix: strengthen authentication password requirements
- ♿️ feat: improve form accessibility for screen readers
Example commit sequence:
- ✨ feat: add user authentication system
- 🐛 fix: resolve memory leak in rendering process
- 📝 docs: update API documentation with new endpoints
- ♻️ refactor: simplify error handling logic in parser
- 🚨 fix: resolve linter warnings in component files
- ✅ test: add unit tests for authentication flow
## Agent Behavior Notes
- **Error handling**: If validation fails, give user option to proceed or fix issues first
- **Auto-staging**: If no files are staged, automatically stage all changes with `git add .`
- **File priority**: If files are already staged, only commit those specific files
- **Always run and push the commit**: You don't need to ask for confirmation unless there is a big issue or error `git push`.
- **Message quality**: Ensure commit messages are clear, concise, and follow conventional format
- **Success feedback**: After successful commit, show commit hash and brief summary
-309
View File
@@ -1,309 +0,0 @@
---
description: Context system manager - harvest summaries, extract knowledge, organize context
tags:
- context
- knowledge-management
- harvest
dependencies:
- subagent:context-organizer
- subagent:contextscout
---
# Context Manager
<critical_rules priority="absolute" enforcement="strict">
<rule id="mvi_strict">
Files MUST be <200 lines. Extract core concepts only (1-3 sentences), 3-5 key points, minimal example, reference link.
</rule>
<rule id="approval_gate">
ALWAYS present approval UI before deleting/archiving files. Letter-based selection (A B C or 'all'). NEVER auto-delete.
</rule>
<rule id="function_structure">
ALWAYS organize by function: concepts/, examples/, guides/, lookup/, errors/ (not flat files).
</rule>
<rule id="lazy_load">
ALWAYS read required context files from .opencode/context/core/context-system/ BEFORE executing operations.
</rule>
</critical_rules>
<execution_priority>
<tier level="1" desc="Safety & MVI">
- Files <200 lines (@critical_rules.mvi_strict)
- Show approval before cleanup (@critical_rules.approval_gate)
- Function-based structure (@critical_rules.function_structure)
- Load context before operations (@critical_rules.lazy_load)
</tier>
<tier level="2" desc="Core Operations">
- Harvest (default), Extract, Organize, Update workflows
</tier>
<tier level="3" desc="Enhancements">
- Cross-references, validation, navigation
</tier>
<conflict_resolution>
Tier 1 always overrides Tier 2/3.
</conflict_resolution>
</execution_priority>
**Arguments**: `$ARGUMENTS`
---
## Default Behavior (No Arguments)
When invoked without arguments: `/context`
<workflow id="default_scan_harvest">
<stage id="1" name="QuickScan">
Scan workspace for summary files:
- *OVERVIEW.md, *SUMMARY.md, SESSION-*.md, CONTEXT-*.md
- Files in .tmp/ directory
- Files >2KB in root directory
</stage>
<stage id="2" name="Report">
Show what was found:
```
Quick scan results:
Found 3 summary files:
📄 CONTEXT-SYSTEM-OVERVIEW.md (4.2 KB)
📄 SESSION-auth-work.md (1.8 KB)
📄 .tmp/NOTES.md (800 bytes)
Recommended action:
/context harvest - Clean up summaries → permanent context
Other options:
/context extract {source} - Extract from docs/code
/context organize {category} - Restructure existing files
/context help - Show all operations
```
</stage>
</workflow>
**Purpose**: Quick tidy-up. Default assumes you want to harvest summaries and compact workspace.
---
## Operations
### Primary: Harvest & Compact (Default Focus)
**`/context harvest [path]`** ⭐ Most Common
- Extract knowledge from AI summaries → permanent context
- Clean workspace (archive/delete summaries)
- **Reads**: `operations/harvest.md` + `standards/mvi.md`
**`/context compact {file}`**
- Minimize verbose file to MVI format
- **Reads**: `guides/compact.md` + `standards/mvi.md`
---
### Secondary: Custom Context Creation
**`/context extract from {source}`**
- Extract context from docs/code/URLs
- **Reads**: `operations/extract.md` + `standards/mvi.md` + `guides/compact.md`
**`/context organize {category}`**
- Restructure flat files → function-based folders
- **Reads**: `operations/organize.md` + `standards/structure.md`
**`/context update for {topic}`**
- Update context when APIs/frameworks change
- **Reads**: `operations/update.md` + `guides/workflows.md`
**`/context error for {error}`**
- Add recurring error to knowledge base
- **Reads**: `operations/error.md` + `standards/templates.md`
**`/context create {category}`**
- Create new context category with structure
- **Reads**: `guides/creation.md` + `standards/structure.md` + `standards/templates.md`
---
### Migration
**`/context migrate`**
- Copy project-intelligence from global (`~/.config/opencode/context/`) to local (`.opencode/context/`)
- For users who installed globally but want project-specific, git-committed context
- Shows diff if local files already exist, asks before overwriting
- Optionally cleans up global project-intelligence after migration
- **Reads**: `standards/mvi.md`
---
### Utility Operations
**`/context map [category]`**
- View current context structure, file counts
**`/context validate`**
- Check integrity, references, file sizes
**`/context help`**
- Show all operations with examples
---
## Lazy Loading Strategy
<lazy_load_map>
<operation name="default">
Read: operations/harvest.md, standards/mvi.md
</operation>
<operation name="harvest">
Read: operations/harvest.md, standards/mvi.md, guides/workflows.md
</operation>
<operation name="compact">
Read: guides/compact.md, standards/mvi.md
</operation>
<operation name="extract">
Read: operations/extract.md, standards/mvi.md, guides/compact.md, guides/workflows.md
</operation>
<operation name="organize">
Read: operations/organize.md, standards/structure.md, guides/workflows.md
</operation>
<operation name="update">
Read: operations/update.md, guides/workflows.md, standards/mvi.md
</operation>
<operation name="error">
Read: operations/error.md, standards/templates.md, guides/workflows.md
</operation>
<operation name="create">
Read: guides/creation.md, standards/structure.md, standards/templates.md
</operation>
<operation name="migrate">
Read: standards/mvi.md
</operation>
</lazy_load_map>
**All files located in**: `.opencode/context/core/context-system/`
---
## Subagent Routing
<subagent_routing>
<!-- Delegate operations to specialized subagents -->
<route operations="harvest|extract|organize|update|error|create|migrate" to="ContextOrganizer">
Pass: operation name, arguments, lazy load map
Subagent loads: Required context files from .opencode/context/core/context-system/
Subagent executes: Multi-stage workflow per operation
</route>
<route operations="map|validate" to="ContextScout">
Pass: operation name, arguments
Subagent executes: Read-only analysis and reporting
</route>
</subagent_routing>
---
## Quick Reference
### Structure
```
.opencode/context/core/context-system/
├── operations/ # How to do things (harvest, extract, organize, update)
├── standards/ # What to follow (mvi, structure, templates)
└── guides/ # Step-by-step (workflows, compact, creation)
```
### MVI Principle (Quick)
- Core concept: 1-3 sentences
- Key points: 3-5 bullets
- Minimal example: <10 lines
- Reference link: to full docs
- File size: <200 lines
### Function-Based Structure (Quick)
```
{category}/
├── navigation.md # Navigation
├── concepts/ # What it is
├── examples/ # Working code
├── guides/ # How to
├── lookup/ # Quick reference
└── errors/ # Common issues
```
---
## Examples
### Default (Quick Scan)
```bash
/context
# Scans workspace, suggests harvest if summaries found
```
### Harvest Summaries
```bash
/context harvest
/context harvest .tmp/
/context harvest OVERVIEW.md
```
### Extract from Docs
```bash
/context extract from docs/api.md
/context extract from https://react.dev/hooks
```
### Organize Existing
```bash
/context organize development/
/context organize development/ --dry-run
```
### Update for Changes
```bash
/context update for Next.js 15
/context update for React 19 breaking changes
```
### Migrate Global to Local
```bash
/context migrate
# Copies project-intelligence from ~/.config/opencode/context/ to .opencode/context/
# Shows what will be copied, asks for approval before proceeding
```
---
## Success Criteria
After any operation:
- [ ] All files <200 lines? (@critical_rules.mvi_strict)
- [ ] Function-based structure used? (@critical_rules.function_structure)
- [ ] Approval UI shown for destructive ops? (@critical_rules.approval_gate)
- [ ] Required context loaded? (@critical_rules.lazy_load)
- [ ] navigation.md updated?
- [ ] Files scannable in <30 seconds?
---
## Full Documentation
**Context System Location**: `.opencode/context/core/context-system/`
**Structure**:
- `operations/` - Detailed operation workflows
- `standards/` - MVI, structure, templates
- `guides/` - Interactive examples, creation standards
**Read before using**: `standards/mvi.md` (understand Minimal Viable Information principle)
@@ -1,433 +0,0 @@
---
description: Validate context file dependencies across agents and registry
tags:
- registry
- validation
- context
- dependencies
- openagents
dependencies:
- command:analyze-patterns
---
# Check Context Dependencies
**Purpose**: Ensure agents properly declare their context file dependencies in frontmatter and registry.
**Arguments**: `$ARGUMENTS`
---
## What It Does
Validates consistency between:
1. **Actual usage** - Context files referenced in agent prompts
2. **Declared dependencies** - Dependencies in agent frontmatter
3. **Registry entries** - Dependencies in registry.json
**Identifies**:
- ✅ Missing dependency declarations (agents use context but don't declare it)
- ✅ Unused context files (exist but no agent references them)
- ✅ Broken references (referenced but don't exist)
- ✅ Format inconsistencies (wrong dependency format)
---
## Usage
```bash
# Analyze all agents
/check-context-deps
# Analyze specific agent
/check-context-deps contextscout
# Auto-fix missing dependencies
/check-context-deps --fix
# Verbose output (show all reference locations)
/check-context-deps --verbose
# Combine flags
/check-context-deps contextscout --verbose
```
---
## Workflow
<workflow id="analyze_context_dependencies">
<stage id="1" name="ScanAgents" required="true">
Scan agent files for context references:
**Search patterns**:
- `.opencode/context/` (direct path references)
- `@.opencode/context/` (@ symbol references)
- `context:` (dependency declarations in frontmatter)
**Locations**:
- `.opencode/agent/**/*.md` (all agents and subagents)
- `.opencode/command/**/*.md` (commands that use context)
**Extract**:
- Agent/command ID
- Context file path
- Line number
- Reference type (path, @-reference, dependency)
</stage>
<stage id="2" name="CheckRegistry" required="true">
For each agent found, check registry.json:
```bash
jq '.components.agents[] | select(.id == "AGENT_ID") | .dependencies' registry.json
jq '.components.subagents[] | select(.id == "AGENT_ID") | .dependencies' registry.json
```
**Verify**:
- Does the agent have a dependencies array?
- Are context file references declared as `context:core/standards/code`?
- Are the dependency formats correct (`context:path/to/file`)?
</stage>
<stage id="3" name="ValidateContextFiles" required="true">
For each context file referenced:
**Check existence**:
```bash
test -f .opencode/context/core/standards/code-quality.md
```
**Check registry**:
```bash
jq '.components.contexts[] | select(.id == "core/standards/code")' registry.json
```
**Identify issues**:
- Context file referenced but doesn't exist
- Context file exists but not in registry
- Context file in registry but never used
</stage>
<stage id="4" name="Report" required="true">
Generate comprehensive report:
```markdown
# Context Dependency Analysis Report
## Summary
- Agents scanned: 25
- Context files referenced: 12
- Missing dependencies: 8
- Unused context files: 2
- Missing context files: 0
## Missing Dependencies (agents using context but not declaring)
### opencoder
**Uses but not declared**:
- context:core/standards/code (referenced 3 times)
- Line 64: "Code tasks → .opencode/context/core/standards/code-quality.md (MANDATORY)"
- Line 170: "Read .opencode/context/core/standards/code-quality.md NOW"
- Line 229: "NEVER execute write/edit without loading required context first"
**Current dependencies**: subagent:task-manager, subagent:coder-agent
**Recommended fix**: Add to frontmatter:
```yaml
dependencies:
- subagent:task-manager
- subagent:coder-agent
- context:core/standards/code # ADD THIS
```
### openagent
**Uses but not declared**:
- context:core/standards/code (referenced 5 times)
- context:core/standards/docs (referenced 3 times)
- context:core/standards/tests (referenced 3 times)
- context:core/workflows/review (referenced 2 times)
- context:core/workflows/delegation (referenced 4 times)
**Recommended fix**: Add to frontmatter:
```yaml
dependencies:
- subagent:task-manager
- subagent:documentation
- context:core/standards/code
- context:core/standards/docs
- context:core/standards/tests
- context:core/workflows/review
- context:core/workflows/delegation
```
## Unused Context Files (exist but no agent references them)
- context:core/standards/analysis (0 references)
- context:core/workflows/sessions (0 references)
**Recommendation**: Consider removing or documenting intended use
## Missing Context Files (referenced but don't exist)
None found ✅
## Context File Usage Map
| Context File | Used By | Reference Count |
|--------------|---------|-----------------|
| core/standards/code | opencoder, openagent, frontend-specialist, reviewer | 15 |
| core/standards/docs | openagent, documentation, technical-writer | 8 |
| core/standards/tests | openagent, tester | 6 |
| core/workflows/delegation | openagent, task-manager | 5 |
| core/workflows/review | openagent, reviewer | 4 |
---
## Next Steps
1. Review missing dependencies above
2. Run `/check-context-deps --fix` to auto-update frontmatter
3. Run `./scripts/registry/auto-detect-components.sh` to update registry
4. Verify with `./scripts/registry/validate-registry.sh`
```
</stage>
<stage id="5" name="Fix" when="--fix flag provided">
For each agent with missing context dependencies:
1. Read the agent file
2. Parse frontmatter YAML
3. Add missing context dependencies to dependencies array
4. Preserve existing dependencies
5. Write updated file
6. Report what was changed
**Example**:
```diff
---
id: opencoder
dependencies:
- subagent:task-manager
- subagent:coder-agent
+ - context:core/standards/code
---
```
**Safety**:
- Only add dependencies that are actually referenced in the file
- Don't remove existing dependencies
- Preserve frontmatter formatting
- Show diff before applying (if interactive)
</stage>
</workflow>
---
## Implementation Details
### Search Patterns
**Find direct path references**:
```bash
grep -rn "\.opencode/context/" .opencode/agent/ .opencode/command/
```
**Find @ references**:
```bash
grep -rn "@\.opencode/context/" .opencode/agent/ .opencode/command/
```
**Find dependency declarations**:
```bash
grep -rn "^\s*-\s*context:" .opencode/agent/ .opencode/command/
```
### Path Normalization
**Convert to dependency format**:
- `.opencode/context/core/standards/code-quality.md` → `context:core/standards/code`
- `@.opencode/context/openagents-repo/quick-start.md` → `context:openagents-repo/quick-start`
- `context/core/standards/code` → `context:core/standards/code`
**Rules**:
1. Strip `.opencode/` prefix
2. Strip `.md` extension
3. Add `context:` prefix for dependencies
### Registry Lookup
**Check if context file is in registry**:
```bash
jq '.components.contexts[] | select(.id == "core/standards/code")' registry.json
```
**Get agent dependencies**:
```bash
jq '.components.agents[] | select(.id == "opencoder") | .dependencies[]?' registry.json
```
---
## Delegation
This command delegates to an analysis agent to perform the work:
```javascript
task(
subagent_type="PatternAnalyst",
description="Analyze context dependencies",
prompt=`
Analyze context file usage across all agents in this repository.
TASK:
1. Use grep to find all references to context files in:
- .opencode/agent/**/*.md
- .opencode/command/**/*.md
2. Search for these patterns:
- ".opencode/context/core/" (direct paths)
- "@.opencode/context/" (@ references)
- "context:" in frontmatter (dependency declarations)
3. For each agent file found:
- Extract agent ID from frontmatter
- List all context files it references
- Check registry.json for declared dependencies
- Identify missing dependency declarations
4. For each context file in .opencode/context/core/:
- Count how many agents reference it
- Check if it exists in registry.json
- Identify unused context files
5. Generate a comprehensive report showing:
- Agents with missing context dependencies
- Unused context files
- Missing context files (referenced but don't exist)
- Context file usage map (which agents use which files)
${ARGUMENTS.includes('--fix') ? `
6. AUTO-FIX MODE:
- Update agent frontmatter to add missing context dependencies
- Use format: context:core/standards/code
- Preserve existing dependencies
- Show what was changed
` : ''}
${ARGUMENTS.includes('--verbose') ? `
VERBOSE MODE: Include all reference locations (file:line) in report
` : ''}
${ARGUMENTS.length > 0 && !ARGUMENTS.includes('--') ? `
FILTER: Only analyze agent: ${ARGUMENTS[0]}
` : ''}
REPORT FORMAT:
- Summary statistics
- Missing dependencies by agent (with recommended fixes)
- Unused context files
- Context file usage map
- Next steps
DO NOT make changes without --fix flag.
ALWAYS show what would be changed before applying fixes.
`
)
```
---
## Examples
### Example 1: Basic Analysis
```bash
/check-context-deps
```
**Output**:
```
Analyzing context file usage across 25 agents...
Found 8 agents with missing context dependencies:
- opencoder: missing context:core/standards/code
- openagent: missing 5 context dependencies
- frontend-specialist: missing context:core/standards/code
...
Run /check-context-deps --fix to auto-update frontmatter
```
### Example 2: Analyze Specific Agent
```bash
/check-context-deps contextscout
```
**Output**:
```
Analyzing agent: contextscout
Context files referenced:
✓ .opencode/context/core/context-system.md (1 reference)
- Line 15: "Load: context:core/context-system"
✓ .opencode/context/core/context-system/standards/mvi.md (2 references)
- Line 16: "Load: context:core/context-system/standards/mvi"
- Line 89: "MVI-aware prioritization"
Registry dependencies:
✓ context:core/context-system DECLARED
✓ context:core/context-system/standards/mvi DECLARED
All dependencies properly declared ✅
```
### Example 3: Auto-Fix
```bash
/check-context-deps --fix
```
**Output**:
```
Analyzing and fixing context dependencies...
Updated opencoder:
+ Added: context:core/standards/code
Updated openagent:
+ Added: context:core/standards/code
+ Added: context:core/standards/docs
+ Added: context:core/standards/tests
+ Added: context:core/workflows/review
+ Added: context:core/workflows/delegation
Total: 2 agents updated, 6 dependencies added
Next: Run ./scripts/registry/auto-detect-components.sh to update registry
```
---
## Success Criteria
✅ All agents that reference context files have them declared in dependencies
✅ All context files in registry are actually used by at least one agent
✅ No broken references (context files referenced but don't exist)
✅ Dependency format is consistent (`context:path/to/file`)
---
## Notes
- **Read-only by default** - Only reports findings, doesn't modify files
- **Use `--fix` to update** - Auto-adds missing dependencies to frontmatter
- **After fixing** - Run `./scripts/registry/auto-detect-components.sh --auto-add` to sync registry
- **Dependency format** - `context:path/to/file` (no `.opencode/` prefix, no `.md` extension)
- **Scans both** - Direct path references and @ references
## Related
- **Registry validation**: `./scripts/registry/validate-registry.sh`
- **Auto-detect components**: `./scripts/registry/auto-detect-components.sh`
- **Context guide**: `.opencode/context/openagents-repo/quality/registry-dependencies.md`
-190
View File
@@ -1,190 +0,0 @@
---
description: Analyze and optimize code for performance, security, and potential issues
---
# Code Optimization Analysis
You are a code optimization specialist focused on performance, security, and identifying potential issues before they become problems. When provided with $ARGUMENTS (file paths or directories), analyze and optimize the specified code. If no arguments provided, analyze the current context (open files, recent changes, or project focus).
## Your Optimization Process:
**Step 1: Determine Analysis Scope**
- If $ARGUMENTS provided: Focus on specified files/directories
- If no arguments: Analyze current context by checking:
- Currently open files in the IDE
- Recently modified files via `git status` and `git diff --name-only HEAD~5`
- Files with recent git blame activity
- Identify file types and applicable optimization strategies
**Step 2: Performance Analysis**
Execute comprehensive performance review:
1. **Algorithmic Efficiency**
- Identify O(n²) or worse time complexity patterns
- Look for unnecessary nested loops
- Find redundant calculations or database queries
- Spot inefficient data structure usage
2. **Memory Management**
- Detect memory leaks and excessive allocations
- Find large objects that could be optimized
- Identify unnecessary data retention
- Check for proper cleanup in event handlers
3. **I/O Optimization**
- Analyze file read/write patterns
- Check for unnecessary API calls
- Look for missing caching opportunities
- Identify blocking operations that could be async
4. **Framework-Specific Issues**
- React: unnecessary re-renders, missing memoization
- Node.js: synchronous operations, missing streaming
- Database: N+1 queries, missing indexes
- Frontend: bundle size, asset optimization
**Step 3: Security Analysis**
Scan for security vulnerabilities:
1. **Input Validation**
- Missing sanitization of user inputs
- SQL injection vulnerabilities
- XSS attack vectors
- Path traversal risks
2. **Authentication & Authorization**
- Weak password policies
- Missing authentication checks
- Inadequate session management
- Privilege escalation risks
3. **Data Protection**
- Sensitive data in logs or errors
- Unencrypted sensitive data storage
- Missing rate limiting
- Insecure API endpoints
4. **Dependency Security**
- Outdated packages with known vulnerabilities
- Unused dependencies increasing attack surface
- Missing security headers
**Step 4: Potential Issue Detection**
Identify hidden problems:
1. **Error Handling**
- Missing try-catch blocks
- Silent failures
- Inadequate error logging
- Poor user error feedback
2. **Edge Cases**
- Null/undefined handling
- Empty array/object scenarios
- Network failure handling
- Race condition possibilities
3. **Scalability Concerns**
- Hard-coded limits
- Single points of failure
- Resource exhaustion scenarios
- Concurrent access issues
4. **Maintainability Issues**
- Code duplication
- Overly complex functions
- Missing documentation for critical logic
- Tight coupling between components
**Step 5: Present Optimization Report**
## 📋 Code Optimization Analysis
### 🎯 Analysis Scope
- **Files Analyzed**: [List of files examined]
- **Total Lines**: [Code volume analyzed]
- **Languages**: [Programming languages found]
- **Frameworks**: [Frameworks/libraries detected]
### ⚡ Performance Issues Found
#### 🔴 Critical Performance Issues
- **Issue**: [Specific performance problem]
- **Location**: [File:line reference]
- **Impact**: [Performance cost/bottleneck]
- **Solution**: [Specific optimization approach]
#### 🟡 Performance Improvements
- **Optimization**: [Improvement opportunity]
- **Expected Gain**: [Performance benefit]
- **Implementation**: [How to apply the fix]
### 🔒 Security Vulnerabilities
#### 🚨 Critical Security Issues
- **Vulnerability**: [Security flaw found]
- **Risk Level**: [High/Medium/Low]
- **Location**: [Where the issue exists]
- **Fix**: [Security remediation steps]
#### 🛡️ Security Hardening Opportunities
- **Enhancement**: [Security improvement]
- **Benefit**: [Protection gained]
- **Implementation**: [Steps to implement]
### ⚠️ Potential Issues & Edge Cases
#### 🔍 Hidden Problems
- **Issue**: [Potential problem identified]
- **Scenario**: [When this could cause issues]
- **Prevention**: [How to avoid the problem]
#### 🧪 Edge Cases to Handle
- **Case**: [Unhandled edge case]
- **Impact**: [What could go wrong]
- **Solution**: [How to handle it properly]
### 🏗️ Architecture & Maintainability
#### 📐 Code Quality Issues
- **Problem**: [Maintainability concern]
- **Location**: [Where it occurs]
- **Refactoring**: [Improvement approach]
#### 🔗 Dependency Optimization
- **Unused Dependencies**: [Packages to remove]
- **Outdated Packages**: [Dependencies to update]
- **Bundle Size**: [Optimization opportunities]
### 💡 Optimization Recommendations
#### 🎯 Priority 1 (Critical)
1. [Most important optimization with immediate impact]
2. [Critical security fix needed]
3. [Performance bottleneck to address]
#### 🎯 Priority 2 (Important)
1. [Significant improvements to implement]
2. [Important edge cases to handle]
#### 🎯 Priority 3 (Nice to Have)
1. [Code quality improvements]
2. [Minor optimizations]
### 🔧 Implementation Guide
```
[Specific code examples showing how to implement key optimizations]
```
### 📊 Expected Impact
- **Performance**: [Expected speed/efficiency gains]
- **Security**: [Risk reduction achieved]
- **Maintainability**: [Code quality improvements]
- **User Experience**: [End-user benefits]
## Optimization Focus Areas:
- **Performance First**: Identify and fix actual bottlenecks, not premature optimizations
- **Security by Design**: Build secure patterns from the start
- **Proactive Issue Prevention**: Catch problems before they reach production
- **Maintainable Solutions**: Ensure optimizations don't sacrifice code clarity
- **Measurable Improvements**: Focus on changes that provide tangible benefits
-26
View File
@@ -1,26 +0,0 @@
---
description: Run the complete testing pipeline
---
# Testing Pipeline
This command runs the complete testing pipeline for the project.
## Usage
To run the complete testing pipeline, just type:
1. Run pnpm type:check
2. Run pnpm lint
3. Run pnpm test
4. Report any failures
5. Fix any failures
6. Repeat until all tests pass
7. Report success
## What This Command Does
1. Runs `pnpm type:check` to check for type errors
2. Runs `pnpm lint` to check for linting errors
3. Runs `pnpm test` to run the tests
4. Reports any failures
-347
View File
@@ -1,347 +0,0 @@
# Validate Repository
Comprehensive validation command that checks the entire OpenAgents Control repository for consistency between CLI, documentation, registry, and components.
## Usage
```bash
/validate-repo
```
## What It Checks
This command performs a comprehensive validation of:
1. **Registry Integrity**
- JSON syntax validation
- Component definitions completeness
- File path references
- Dependency declarations
2. **Component Existence**
- All agents exist at specified paths
- All subagents exist at specified paths
- All commands exist at specified paths
- All tools exist at specified paths
- All plugins exist at specified paths
- All context files exist at specified paths
- All config files exist at specified paths
3. **Profile Consistency**
- Component counts match documentation
- Profile descriptions are accurate
- Dependencies are satisfied
- No duplicate components
4. **Documentation Accuracy**
- README component counts match registry
- OpenAgent documentation references are valid
- Context file references are correct
- Installation guide is up to date
5. **Context File Structure**
- All referenced context files exist
- Context file organization is correct
- No orphaned context files
6. **Cross-References**
- Agent dependencies exist
- Subagent references are valid
- Command references are valid
- Tool dependencies are satisfied
## Output
The command generates a detailed report showing:
- ✅ What's correct and validated
- ⚠️ Warnings for potential issues
- ❌ Errors that need fixing
- 📊 Summary statistics
## Instructions
You are a validation specialist. Your task is to comprehensively validate the OpenAgents Control repository for consistency and correctness.
### Step 1: Validate Registry JSON
1. Read and parse `registry.json`
2. Validate JSON syntax
3. Check schema structure:
- `version` field exists
- `repository` field exists
- `categories` object exists
- `components` object exists with all types
- `profiles` object exists
- `metadata` object exists
### Step 2: Validate Component Definitions
For each component type (agents, subagents, commands, tools, plugins, contexts, config):
1. Check required fields:
- `id` (unique)
- `name`
- `type`
- `path`
- `description`
- `tags` (array)
- `dependencies` (array)
- `category`
2. Verify file exists at `path`
3. Check for duplicate IDs
4. Validate category is in defined categories
### Step 3: Validate Profiles
For each profile (essential, developer, business, full, advanced):
1. Count components in profile
2. Verify all component references exist in components section
3. Check dependencies are satisfied
4. Validate no duplicate components
### Step 4: Cross-Reference with Documentation
1. **navigation.md**:
- Extract component counts from profile descriptions
- Compare with actual registry counts
- Check profile descriptions match registry descriptions
2. **docs/agents/openagent.md**:
- Verify delegation criteria mentioned
- Check context file references
- Validate workflow descriptions
3. **docs/getting-started/installation.md**:
- Check profile descriptions
- Verify installation commands
### Step 5: Validate Context File Structure
1. List all files in `.opencode/context/`
2. Check against registry context entries
3. Identify orphaned files (exist but not in registry)
4. Identify missing files (in registry but don't exist)
5. Validate structure:
- `core/standards/` files
- `core/workflows/` files
- `core/system/` files
- `project/` files
### Step 6: Validate Dependencies
For each component with dependencies:
1. Parse dependency string (format: `type:id`)
2. Verify referenced component exists
3. Check for circular dependencies
4. Validate dependency chain completeness
### Step 7: Generate Report
Create a comprehensive report with sections:
#### ✅ Validated Successfully
- Registry JSON syntax
- Component file existence
- Profile integrity
- Documentation accuracy
- Context file structure
- Dependency chains
#### ⚠️ Warnings
- Orphaned files (exist but not referenced)
- Unused components (defined but not in any profile)
- Missing descriptions or tags
- Outdated metadata dates
#### ❌ Errors
- Missing files
- Broken dependencies
- Invalid JSON
- Component count mismatches
- Broken documentation references
- Duplicate component IDs
#### 📊 Statistics
- Total components: X
- Total profiles: X
- Total context files: X
- Components per profile breakdown
- File coverage percentage
### Step 8: Provide Recommendations
Based on findings, suggest:
- Files to create
- Registry entries to add/remove
- Documentation to update
- Dependencies to fix
## Example Report Format
```markdown
# OpenAgents Control Repository Validation Report
Generated: 2025-11-19 14:30:00
## Summary
✅ 95% validation passed
⚠️ 3 warnings found
❌ 2 errors found
---
## ✅ Validated Successfully
### Registry Integrity
✅ JSON syntax valid
✅ All required fields present
✅ Schema structure correct
### Component Existence (45/47 files found)
✅ Agents: 3/3 files exist
✅ Subagents: 15/15 files exist
✅ Commands: 8/8 files exist
✅ Tools: 2/2 files exist
✅ Plugins: 2/2 files exist
✅ Contexts: 13/15 files exist
✅ Config: 2/2 files exist
### Profile Consistency
✅ Essential: 9 components (matches README)
✅ Developer: 29 components (matches README)
✅ Business: 15 components (matches README)
✅ Full: 35 components (matches README)
✅ Advanced: 42 components (matches README)
### Documentation Accuracy
✅ README component counts match registry
✅ OpenAgent documentation up to date
✅ Installation guide accurate
---
## ⚠️ Warnings (3)
1. **Orphaned Context File**
- File: `.opencode/context/legacy/old-patterns.md`
- Issue: Exists but not referenced in registry
- Recommendation: Add to registry or remove file
2. **Unused Component**
- Component: `workflow-orchestrator` (agent)
- Issue: Defined in registry but not in any profile
- Recommendation: Add to a profile or mark as deprecated
3. **Outdated Metadata**
- Field: `metadata.lastUpdated`
- Current: 2025-11-15
- Recommendation: Update to current date
---
## ❌ Errors (2)
1. **Missing Context File**
- Component: `context:advanced-patterns`
- Expected path: `.opencode/context/core/advanced-patterns.md`
- Referenced in: developer, full, advanced profiles
- Action: Create file or remove from registry
2. **Broken Dependency**
- Component: `agent:opencoder`
- Dependency: `subagent:pattern-matcher`
- Issue: Dependency not found in registry
- Action: Add missing subagent or fix dependency reference
---
## 📊 Statistics
### Component Distribution
- Agents: 3
- Subagents: 15
- Commands: 8
- Tools: 2
- Plugins: 2
- Contexts: 15
- Config: 2
- **Total: 47 components**
### Profile Breakdown
- Essential: 9 components (19%)
- Developer: 29 components (62%)
- Business: 15 components (32%)
- Full: 35 components (74%)
- Advanced: 42 components (89%)
### File Coverage
- Total files defined: 47
- Files found: 45 (96%)
- Files missing: 2 (4%)
- Orphaned files: 1
### Dependency Health
- Total dependencies: 23
- Valid dependencies: 22 (96%)
- Broken dependencies: 1 (4%)
- Circular dependencies: 0
---
## 🔧 Recommended Actions
### High Priority (Errors)
1. Create missing file: `.opencode/context/core/advanced-patterns.md`
2. Fix broken dependency in `opencoder`
### Medium Priority (Warnings)
1. Remove orphaned file or add to registry
2. Add `workflow-orchestrator` to a profile or deprecate
3. Update metadata.lastUpdated to 2025-11-19
### Low Priority (Improvements)
1. Add more tags to components for better searchability
2. Consider adding descriptions to all context files
3. Document component categories in README
---
## Next Steps
1. Review and fix all ❌ errors
2. Address ⚠️ warnings as needed
3. Re-run validation to confirm fixes
4. Update documentation if needed
---
**Validation Complete** ✓
```
## Implementation Notes
The command should:
- Use bash/python for file system operations
- Parse JSON with proper error handling
- Generate markdown report
- Be non-destructive (read-only validation)
- Provide actionable recommendations
- Support verbose mode for detailed output
## Error Handling
- Gracefully handle missing files
- Continue validation even if errors found
- Collect all issues before reporting
- Provide clear error messages with context
## Performance
- Should complete in < 30 seconds
- Cache file reads where possible
- Parallel validation where safe
- Progress indicators for long operations
@@ -1,39 +0,0 @@
# Concept: Mastra Agents & Tools
**Purpose**: Reusable units of logic and LLM-powered entities.
**Last Updated**: 2026-01-09
---
## Core Idea
Agents are specialized LLM configurations that use Tools to interact with external systems or perform specific logic. Tools are the building blocks that provide functionality to both agents and workflows.
## Key Points
- **Agents**: Defined with a `name`, `instructions`, and `model`. They can be assigned a set of `tools`.
- **Tools**: Defined with `id`, `inputSchema`, `outputSchema`, and an `execute` function.
- **Type Safety**: Both agents and tools use Zod for schema validation.
- **Standalone Use**: Tools can be executed independently of agents, making them highly reusable.
## Quick Example
```typescript
// Tool
const myTool = createTool({
id: 'my-tool',
inputSchema: z.object({ query: z.string() }),
execute: async ({ inputData }) => ({ result: `Processed ${inputData.query}` }),
});
// Agent
const myAgent = new Agent({
name: 'My Agent',
instructions: 'Use my-tool to process queries.',
model: { provider: 'OPEN_AI', name: 'gpt-4o' },
tools: { myTool },
});
```
**Reference**: `src/mastra/agents/`, `src/mastra/tools/`
**Related**:
- concepts/core.md
- concepts/workflows.md
@@ -1,35 +0,0 @@
# Concept: Mastra Core
**Purpose**: Central orchestration layer for AI agents, workflows, and tools in this project.
**Last Updated**: 2026-01-09
---
## Core Idea
Mastra is the central hub that wires together agents, tools, workflows, and observability. It provides a unified interface for executing complex AI tasks with built-in persistence and logging.
## Key Points
- **Centralized Config**: All components are registered in `src/mastra/index.ts`.
- **Persistence**: Uses `LibSQLStore` (SQLite) for storing traces, spans, and workflow states.
- **Observability**: Built-in tracing and logging (Pino) for every execution.
- **Modular Design**: Agents, tools, and workflows are defined separately and composed in the main instance.
## Quick Example
```typescript
import { Mastra } from '@mastra/core/mastra';
import { agents, tools, workflows } from './components';
export const mastra = new Mastra({
agents,
tools,
workflows,
storage: new LibSQLStore({ url: 'file:./mastra.db' }),
});
```
**Reference**: `src/mastra/index.ts`
**Related**:
- concepts/workflows.md
- concepts/agents-tools.md
- lookup/mastra-config.md
@@ -1,39 +0,0 @@
# Concept: Mastra Evaluations
**Purpose**: Quality assurance and scoring for LLM outputs.
**Last Updated**: 2026-01-09
---
## Core Idea
Evaluations in Mastra use Scorers to assess the quality, accuracy, and safety of LLM-generated content. They provide a quantitative way to measure performance and detect issues like hallucinations or factual errors.
## Key Points
- **Scorers**: Specialized functions that take LLM output (and optionally ground truth) and return a score (0-1).
- **Integration**: Registered in the Mastra instance and can be triggered automatically during workflow execution.
- **Metrics**: Common metrics include hallucination detection, fact validation, and relevance scoring.
- **Audit Trail**: Scorer results are stored in the `mastra_scorers` table for long-term analysis and reporting.
## Quick Example
```typescript
// Scorer definition
export const hallucinationDetector = new Scorer({
id: 'hallucination-detector',
description: 'Detects hallucinations in LLM output',
execute: async ({ output, context }) => {
// Logic to detect hallucinations
return { score: 0.95, rationale: 'No hallucinations found' };
},
});
// Registration
export const mastra = new Mastra({
scorers: { hallucinationDetector },
});
```
**Reference**: `src/mastra/scorers/`, `src/mastra/evaluation/`
**Related**:
- concepts/core.md
- concepts/workflows.md
@@ -1,36 +0,0 @@
# Concept: Mastra Data Storage
**Purpose**: Persistence layer for cases, documents, assessments, and observability.
**Last Updated**: 2026-01-09
---
## Core Idea
Mastra uses a dual-storage approach: a local SQLite database (via Drizzle ORM) for business entities and a built-in `LibSQLStore` for Mastra-specific execution data (traces, spans).
## Key Points
- **Business Entities**: Managed in `src/db/schema.ts`. Includes `cases`, `documents`, `assessments`, and `outputs`.
- **Mastra Store**: `LibSQLStore` handles `mastra_traces`, `mastra_ai_spans`, and `mastra_scorers`.
- **V3 Extensions**: Specific tables for `timeline_events`, `evidence_gaps`, `sub_claims`, and `vulnerability_flags`.
- **Observability**: `prompt_execution_traces` provides detailed cost and token tracking per AI call.
- **File Storage**: Large blobs (PDFs, JSON outputs) are stored in `./tmp/` with paths referenced in the DB.
## Quick Example
```typescript
// Business Schema (Drizzle)
export const cases = sqliteTable('cases', {
id: text('id').primaryKey(),
status: text('status').default('new'),
});
// Mastra Store Config
storage: new LibSQLStore({
url: process.env.MASTRA_DB_PATH || 'file:./mastra.db',
}),
```
**Reference**: `src/db/schema.ts`, `src/mastra/index.ts`
**Related**:
- concepts/core.md
- lookup/mastra-config.md
@@ -1,33 +0,0 @@
# Concept: Mastra Workflows
**Purpose**: Linear and parallel execution chains for complex AI tasks.
**Last Updated**: 2026-01-09
---
## Core Idea
Workflows in Mastra are directed graphs of steps that process data sequentially or in parallel. They provide a structured way to handle multi-stage LLM operations with built-in state management and human-in-the-loop (HITL) support.
## Key Points
- **Step Definition**: Created with `createStep`, requiring `inputSchema`, `outputSchema`, and an `execute` function.
- **Chaining**: Steps are linked using `.then()` for sequential and `.parallel()` for concurrent execution.
- **HITL Support**: Steps can `suspend` execution to wait for human input and `resume` when data is provided.
- **State Access**: Each step has access to the global workflow `state` and the `inputData` from the previous step.
## Quick Example
```typescript
const workflow = createWorkflow({ id: 'my-workflow', inputSchema, outputSchema })
.then(step1)
.parallel([step2a, step2b])
.then(mergeStep)
.commit();
const { runId, start } = workflow.createRun();
const result = await start({ inputData: { ... } });
```
**Reference**: `src/mastra/workflows/`
**Related**:
- concepts/core.md
- examples/workflow-example.md
@@ -1,31 +0,0 @@
# Errors: Mastra Implementation
**Purpose**: Common errors, their causes, and recovery strategies.
**Last Updated**: 2026-01-09
---
## Core Idea
Errors in Mastra typically fall into three categories: AI generation failures, structured output validation errors, and context/resource missing errors.
## Key Points
- **AIGenerationError**: Occurs when the LLM fails to generate a response (e.g., safety filters, model downtime).
- **StructuredOutputError**: Triggered when the LLM response doesn't match the Zod schema defined in the tool or step.
- **RateLimitError**: Hit when exceeding provider limits. Includes a `retryAfter` value.
- **MastraContextError**: Raised when a required resource (like `services` or `mastra` instance) is missing from the execution context.
- **Retry Strategy**: Use `isRetryableError(error)` to determine if a transient failure can be recovered with exponential backoff.
## Common Errors Table
| Error | Cause | Fix |
|-------|-------|-----|
| `StructuredOutputError` | LLM hallucinated wrong JSON | Refine prompt or use simpler schema |
| `RateLimitError` | Too many concurrent requests | Implement rate limiting or increase quota |
| `NotFoundError` | Case or Document ID missing in DB | Check DB state before workflow start |
| `MastraContextError` | `services` not passed to tool | Ensure `services` is in `ToolExecutionContext` |
**Reference**: `src/lib/errors.ts`
**Related**:
- concepts/core.md
- guides/testing.md
@@ -1,40 +0,0 @@
# Example: Document Ingestion Workflow
**Purpose**: Demonstrates a multi-step workflow with parallel processing.
**Last Updated**: 2026-01-09
---
## Workflow Definition
```typescript
export const documentIngestionWorkflow = createWorkflow({
id: 'document-ingestion',
inputSchema: z.object({ filename: z.string(), fileBuffer: z.any() }),
outputSchema: z.object({ documentId: z.string(), success: z.boolean() }),
})
.then(uploadStep) // Step 1: Upload
.then(extractionStep) // Step 2: Extract Text
.parallel([ // Step 3: Process in parallel
classificationStep,
summarizationStep
])
.then(mergeResultsStep) // Step 4: Merge
.commit();
```
## Step Execution
```typescript
const uploadStep = createStep({
id: 'upload-document',
execute: async ({ inputData, mastra }) => {
const result = await documentUploadTool.execute(inputData, { mastra });
return result;
},
});
```
**Reference**: `src/mastra/workflows/document-ingestion-with-classification-workflow.ts`
**Related**:
- concepts/workflows.md
- concepts/agents-tools.md
@@ -1,35 +0,0 @@
# Guide: Modular Mastra Building
**Purpose**: Best practices for structuring a large-scale Mastra implementation.
**Last Updated**: 2026-01-09
---
## Core Idea
Modular building ensures that as the project grows, components remain testable, reusable, and easy to navigate. This is achieved by separating logic into specialized directories and using a central registry.
## Key Points
- **Component Separation**: Keep `agents`, `tools`, `workflows`, and `scorers` in their own top-level directories within `src/mastra/`.
- **Shared Services**: Use a `shared.ts` file to instantiate services (DB, repositories) to prevent circular dependencies between workflows and the main Mastra instance.
- **Central Registry**: Register all components in `src/mastra/index.ts`. This is the single source of truth for the Mastra instance.
- **Feature-Based Steps**: Group related workflow steps into sub-directories (e.g., `src/mastra/workflows/v3/steps/`) to keep workflow files clean.
## Quick Example
```typescript
// src/mastra/shared.ts
export const services = createServices();
// src/mastra/index.ts
import { services } from './shared';
export const mastra = new Mastra({
workflows: { myWorkflow },
agents: { myAgent },
// ...
});
```
**Reference**: `src/mastra/index.ts`, `src/mastra/shared.ts`
**Related**:
- concepts/core.md
- guides/workflow-step-structure.md
@@ -1,33 +0,0 @@
# Guide: Testing Mastra
**Purpose**: How to run and validate Mastra components in this project.
**Last Updated**: 2026-01-09
---
## Core Idea
Testing in this project is divided into tool-level tests and full workflow integration tests. Use the provided npm scripts for rapid validation.
## Key Points
- **Tool Tests**: Validate individual tools in isolation (e.g., `npm run test:playbook`).
- **Workflow Tests**: Run full end-to-end scenarios (e.g., `npm run test:workflow`).
- **Baseline Tests**: Compare current performance against a known baseline (`npm run test:baseline`).
- **Observability**: Use `npm run traces` after tests to inspect the execution details in the database.
## Quick Example
```bash
# Test a specific tool
npm run test:calculator
# Run full validity workflow
npm run validity:workflow
# View results of the last run
npm run traces
```
**Reference**: `package.json` scripts, `scripts/` directory
**Related**:
- concepts/core.md
- lookup/mastra-config.md
@@ -1,38 +0,0 @@
# Guide: Workflow Step Structure
**Purpose**: Standardized pattern for defining maintainable and testable workflow steps.
**Last Updated**: 2026-01-09
---
## Core Idea
Workflow steps should be self-contained units that encapsulate their input/output schemas and execution logic. For complex workflows, steps should be moved to a dedicated `steps/` directory and grouped by phase.
## Key Points
- **Directory Structure**: Group steps by phase (e.g., `steps/phase1-load.ts`, `steps/phase2-process.ts`).
- **Schema Centralization**: Define shared schemas (like `workflowStateSchema`) in a `schemas.ts` file within the steps directory.
- **Explicit State**: Use `stateSchema` in `createStep` to ensure type safety when accessing the global workflow state.
- **Tool Delegation**: Steps should primarily act as orchestrators, delegating heavy lifting to Tools.
- **Logging**: Include clear console logs at the start and end of each step for easier debugging.
## Quick Example
```typescript
// src/mastra/workflows/v3/steps/phase1.ts
export const myStep = createStep({
id: 'my-step-id',
inputSchema: z.object({ ... }),
outputSchema: z.object({ ... }),
stateSchema: workflowStateSchema,
execute: async ({ inputData, state, mastra }) => {
console.log('🚀 Starting myStep...');
const result = await myTool.execute(inputData, { mastra });
return result;
},
});
```
**Reference**: `src/mastra/workflows/v3/steps/`
**Related**:
- concepts/workflows.md
- guides/modular-building.md
@@ -1,39 +0,0 @@
# Lookup: Mastra Configuration
**Purpose**: Quick reference for Mastra file locations and registration.
**Last Updated**: 2026-01-09
---
## File Locations
| Component | Directory | Registration File |
|-----------|-----------|-------------------|
| **Mastra Instance** | `src/mastra/` | `src/mastra/index.ts` |
| **Agents** | `src/mastra/agents/` | `src/mastra/index.ts` |
| **Tools** | `src/mastra/tools/` | `src/mastra/index.ts` |
| **Workflows** | `src/mastra/workflows/` | `src/mastra/index.ts` |
| **Scorers** | `src/mastra/scorers/` | `src/mastra/index.ts` |
| **Services** | `src/services/` | `src/mastra/shared.ts` |
## Database Tables
| Table Name | Description |
|------------|-------------|
| `mastra_traces` | Workflow execution traces |
| `mastra_ai_spans` | LLM call spans and token usage |
| `mastra_scorers` | Evaluation results and scores |
| `mastra_workflow_state` | Current state of running workflows |
## Common Commands
| Command | Description |
|---------|-------------|
| `npm run dev` | Start Mastra in development mode |
| `npm run traces` | View recent execution traces |
| `npm run test:workflow` | Run the test workflow script |
**Related**:
- concepts/core.md
- concepts/workflows.md
@@ -1,29 +0,0 @@
# AI Navigation
**Purpose**: AI frameworks, agent runtimes, and LLM integration patterns.
---
## Structure
```
ai/
├── navigation.md
└── mastra-ai/
├── navigation.md
└── [patterns].md
```
---
## Quick Routes
| Task | Path |
|------|------|
| **MAStra AI** | `mastra-ai/navigation.md` |
---
## By Technology
**MAStra AI** → AI framework for building agents and workflows.
@@ -1,77 +0,0 @@
# Backend Development Navigation
**Scope**: Server-side, APIs, databases, auth
---
## Structure
```
development/backend/ # [future]
├── navigation.md
│
├── api-patterns/ # Approach-based
│ ├── rest-design.md
│ ├── graphql-design.md
│ ├── grpc-patterns.md
│ └── websocket-patterns.md
│
├── nodejs/ # Tech-specific
│ ├── express-patterns.md
│ ├── fastify-patterns.md
│ └── error-handling.md
│
├── python/
│ ├── fastapi-patterns.md
│ └── django-patterns.md
│
├── authentication/ # Functional concern
│ ├── jwt-patterns.md
│ ├── oauth-patterns.md
│ └── session-management.md
│
└── middleware/
├── logging.md
├── rate-limiting.md
└── cors.md
```
---
## Quick Routes
| Task | Path |
|------|------|
| **REST API** | `backend/api-patterns/rest-design.md` [future] |
| **GraphQL** | `backend/api-patterns/graphql-design.md` [future] |
| **API design principles** | `principles/api-design.md` |
| **Node.js** | `backend/nodejs/express-patterns.md` [future] |
| **Python** | `backend/python/fastapi-patterns.md` [future] |
| **Auth (JWT)** | `backend/authentication/jwt-patterns.md` [future] |
---
## By Approach
**REST** → `backend/api-patterns/rest-design.md` [future]
**GraphQL** → `backend/api-patterns/graphql-design.md` [future]
**gRPC** → `backend/api-patterns/grpc-patterns.md` [future]
## By Language
**Node.js** → `backend/nodejs/` [future]
**Python** → `backend/python/` [future]
## By Concern
**Authentication** → `backend/authentication/` [future]
**Middleware** → `backend/middleware/` [future]
**Data layer** → `data/` [future]
---
## Related Context
- **API Design Principles** → `principles/api-design.md`
- **Core Standards** → `../core/standards/code-quality.md`
- **Data Patterns** → `data/navigation.md` [future]
@@ -1,55 +0,0 @@
# Backend Development Navigation
**Purpose**: Server-side development patterns
**Status**: 🚧 Placeholder - Content coming soon
---
## Planned Structure
```
backend/
├── navigation.md
│
├── api-patterns/ # Approach-based
│ ├── rest-design.md
│ ├── graphql-design.md
│ ├── grpc-patterns.md
│ └── trpc-patterns.md
│
├── nodejs/ # Tech-specific
│ ├── express-patterns.md
│ ├── fastify-patterns.md
│ └── nextjs-api-routes.md
│
├── python/
│ ├── fastapi-patterns.md
│ └── django-patterns.md
│
├── authentication/ # Functional concern
│ ├── jwt-patterns.md
│ ├── oauth-patterns.md
│ └── session-management.md
│
└── middleware/
├── logging.md
├── rate-limiting.md
└── cors.md
```
---
## For Now
Use specialized navigation: `../backend-navigation.md`
Also see: `../principles/api-design.md`
---
## Related Context
- **Backend Navigation** → `../backend-navigation.md`
- **API Design Principles** → `../principles/api-design.md`
- **Core Standards** → `../../core/standards/code-quality.md`
@@ -1,36 +0,0 @@
# Data Layer Navigation
**Purpose**: Database and data access patterns
**Status**: 🚧 Placeholder - Content coming soon
---
## Planned Structure
```
data/
├── navigation.md
│
├── sql-patterns/
│ ├── postgres-patterns.md
│ ├── mysql-patterns.md
│ └── query-optimization.md
│
├── nosql-patterns/
│ ├── mongodb-patterns.md
│ ├── redis-patterns.md
│ └── dynamodb-patterns.md
│
└── orm-patterns/
├── prisma-patterns.md
├── typeorm-patterns.md
└── sequelize-patterns.md
```
---
## Related Context
- **Backend Navigation** → `../backend-navigation.md`
- **Core Standards** → `../../core/standards/code-quality.md`
@@ -1,29 +0,0 @@
# Frameworks Navigation
**Purpose**: Full-stack and meta-frameworks that span multiple architectural layers.
---
## Structure
```
frameworks/
├── navigation.md
└── tanstack-start/
├── navigation.md
└── [patterns].md
```
---
## Quick Routes
| Task | Path |
|------|------|
| **Tanstack Start** | `tanstack-start/navigation.md` |
---
## By Framework
**Tanstack Start** → Full-stack React framework with SSR and server functions.
@@ -1,40 +0,0 @@
# Frontend Development Navigation
**Purpose**: Client-side development patterns
---
## Structure
```
frontend/
├── navigation.md
├── when-to-delegate.md
└── react/
├── navigation.md
└── react-patterns.md
```
---
## Quick Routes
| Task | Path |
|------|------|
| **When to delegate** | `when-to-delegate.md` |
| **React patterns** | `react/react-patterns.md` |
| **React navigation** | `react/navigation.md` |
---
## By Framework
**React** → `react/` - Modern React patterns, hooks, component design
---
## Related Context
- **UI Navigation** → `../ui-navigation.md`
- **Visual Design** → `../../ui/web/navigation.md`
- **Core Standards** → `../../core/standards/code-quality.md`
@@ -1,468 +0,0 @@
<!-- Context: development/frontend/when-to-delegate | Priority: high | Version: 1.0 | Updated: 2026-01-30 -->
# When to Delegate to Frontend Specialist
## Overview
Clear decision criteria for when to delegate frontend/UI work to the **frontend-specialist** subagent vs. handling it directly.
## Quick Reference
**Delegate to frontend-specialist when**:
- UI/UX design work (wireframes, themes, animations)
- Design system implementation
- Complex responsive layouts
- Animation and micro-interactions
- Visual design iterations
**Handle directly when**:
- Simple HTML/CSS edits
- Single component updates
- Bug fixes in existing UI
- Minor styling tweaks
---
## Decision Matrix
### ✅ DELEGATE to Frontend-Specialist
| Scenario | Why Delegate | Example |
|----------|--------------|---------|
| **New UI design from scratch** | Needs staged workflow (layout → theme → animation → implement) | "Create a landing page for our product" |
| **Design system work** | Requires ContextScout for standards, ExternalScout for UI libs | "Implement our design system with Tailwind + Shadcn" |
| **Complex responsive layouts** | Needs mobile-first approach across breakpoints | "Build a dashboard with sidebar, cards, and responsive grid" |
| **Animation implementation** | Requires animation patterns, performance optimization | "Add smooth transitions and micro-interactions to the UI" |
| **Multi-stage design iterations** | Needs versioning (design_iterations/ folder) | "Design a checkout flow with 3 steps" |
| **Theme creation** | Requires OKLCH colors, CSS custom properties | "Create a dark mode theme for the app" |
| **Component library integration** | Needs ExternalScout for current docs (Flowbite, Radix, etc.) | "Integrate Flowbite components into our app" |
| **Accessibility-focused UI** | Requires WCAG compliance, ARIA attributes | "Build an accessible form with proper labels and validation" |
### ⚠️ HANDLE DIRECTLY (Don't Delegate)
| Scenario | Why Direct | Example |
|----------|------------|---------|
| **Simple HTML edits** | Single file, straightforward change | "Change the button text from 'Submit' to 'Send'" |
| **Minor CSS tweaks** | Small styling adjustment | "Make the header padding 20px instead of 16px" |
| **Bug fixes** | Fixing existing code, not creating new design | "Fix the broken link in the footer" |
| **Content updates** | Changing text, images, or data | "Update the hero section copy" |
| **Single component updates** | Modifying one existing component | "Add a new prop to the Button component" |
| **Quick prototypes** | Throwaway code for testing | "Create a quick HTML mockup to test an idea" |
---
## Delegation Checklist
Before delegating to frontend-specialist, ensure:
- [ ] **Task is UI/design focused** (not backend, logic, or data)
- [ ] **Task requires design expertise** (layout, theme, animations)
- [ ] **Task benefits from staged workflow** (layout → theme → animation → implement)
- [ ] **Task needs context discovery** (design systems, UI libraries, standards)
- [ ] **User has approved the approach** (never delegate before approval)
---
## How to Delegate
### Step 1: Discover Context (Optional but Recommended)
If you're unsure what context the frontend-specialist will need:
```javascript
task(
subagent_type="ContextScout",
description="Find frontend design context",
prompt="Find design system standards, UI component patterns, animation guidelines, and responsive breakpoint conventions for frontend work."
)
```
### Step 2: Propose Approach
Present a plan to the user:
```markdown
## Implementation Plan
**Task**: Create landing page with hero section, features grid, and CTA
**Approach**: Delegate to frontend-specialist subagent
**Why**:
- Requires design system implementation
- Needs responsive layout across breakpoints
- Includes animations and micro-interactions
- Benefits from staged workflow (layout → theme → animation → implement)
**Context Needed**:
- Design system standards (ui/web/design-systems.md)
- UI styling standards (ui/web/ui-styling-standards.md)
- Animation patterns (ui/web/animation-patterns.md)
**Approval needed before proceeding.**
```
### Step 3: Get Approval
Wait for explicit user approval before delegating.
### Step 4: Delegate with Context
**For simple delegation** (no session needed):
```javascript
task(
subagent_type="frontend-specialist",
description="Create landing page design",
prompt="Context to load:
- .opencode/context/ui/web/design-systems.md
- .opencode/context/ui/web/ui-styling-standards.md
- .opencode/context/ui/web/animation-patterns.md
Task: Create a landing page with:
- Hero section with headline, subheadline, CTA button
- Features grid (3 columns on desktop, 1 on mobile)
- Smooth scroll animations
Requirements:
- Use Tailwind CSS + Flowbite
- Mobile-first responsive design
- Animations <400ms
- Save to design_iterations/landing_1.html
Follow your staged workflow:
1. Layout (ASCII wireframe)
2. Theme (CSS theme file)
3. Animation (micro-interactions)
4. Implement (HTML file)
Request approval between each stage."
)
```
**For complex delegation** (with session):
Create session context file first, then delegate with session path.
---
## Common Patterns
### Pattern 1: New Landing Page
**Trigger**: User asks for a new landing page, marketing page, or product page
**Decision**: ✅ Delegate to frontend-specialist
**Why**: Requires full design workflow (layout, theme, animations, implementation)
**Example**:
```
User: "Create a landing page for our SaaS product"
You: [Propose approach] → [Get approval] → [Delegate to frontend-specialist]
```
### Pattern 2: Design System Implementation
**Trigger**: User wants to implement or update a design system
**Decision**: ✅ Delegate to frontend-specialist
**Why**: Needs ContextScout for standards, ExternalScout for UI library docs
**Example**:
```
User: "Implement our design system using Tailwind and Shadcn"
You: [Propose approach] → [Get approval] → [Delegate to frontend-specialist]
```
### Pattern 3: Component Library Integration
**Trigger**: User wants to integrate a UI component library (Flowbite, Radix, etc.)
**Decision**: ✅ Delegate to frontend-specialist
**Why**: Requires ExternalScout for current docs, proper integration patterns
**Example**:
```
User: "Add Flowbite components to our app"
You: [Propose approach] → [Get approval] → [Delegate to frontend-specialist]
```
### Pattern 4: Animation Work
**Trigger**: User wants animations, transitions, or micro-interactions
**Decision**: ✅ Delegate to frontend-specialist
**Why**: Requires animation patterns, performance optimization (<400ms)
**Example**:
```
User: "Add smooth animations to the dashboard"
You: [Propose approach] → [Get approval] → [Delegate to frontend-specialist]
```
### Pattern 5: Simple HTML Edit
**Trigger**: User wants to change text, fix a link, or update content
**Decision**: ⚠️ Handle directly (don't delegate)
**Why**: Simple edit, no design work needed
**Example**:
```
User: "Change the button text to 'Get Started'"
You: [Edit the HTML file directly]
```
### Pattern 6: CSS Bug Fix
**Trigger**: User reports a styling bug or broken layout
**Decision**: ⚠️ Handle directly (don't delegate)
**Why**: Bug fix, not new design work
**Example**:
```
User: "The header is overlapping the content on mobile"
You: [Read the CSS, fix the issue directly]
```
---
## Red Flags (Don't Delegate)
❌ **User just wants a quick fix** → Handle directly
❌ **Task is backend/logic focused** → Wrong subagent (use coder-agent or handle directly)
❌ **Task is a single line change** → Handle directly
❌ **Task is content update** → Handle directly
❌ **Task is testing/validation** → Wrong subagent (use tester)
❌ **Task is code review** → Wrong subagent (use reviewer)
---
## Green Flags (Delegate)
✅ **User wants a new UI design** → Delegate
✅ **Task involves design systems** → Delegate
✅ **Task requires responsive layouts** → Delegate
✅ **Task includes animations** → Delegate
✅ **Task needs UI library integration** → Delegate
✅ **Task benefits from staged workflow** → Delegate
✅ **Task requires design expertise** → Delegate
---
## Frontend-Specialist Capabilities
**What it does well**:
- Create complete UI designs from scratch
- Implement design systems (Tailwind, Shadcn, Flowbite)
- Build responsive layouts (mobile-first)
- Add animations and micro-interactions
- Integrate UI component libraries
- Create themes with OKLCH colors
- Follow staged workflow (layout → theme → animation → implement)
- Version designs (design_iterations/ folder)
**What it doesn't do**:
- Backend logic or API integration
- Database queries or data processing
- Testing or validation
- Code review or refactoring
- Simple HTML/CSS edits (overkill)
- Content updates (just text changes)
---
## Context Files Frontend-Specialist Uses
The frontend-specialist will automatically load these via ContextScout:
**Design System**:
- `ui/web/design-systems.md` - Theme templates, color systems
- `ui/web/ui-styling-standards.md` - Tailwind, Flowbite, responsive design
- `ui/web/design-assets.md` - Images, icons, fonts
**Patterns**:
- `ui/web/animation-patterns.md` - Animation syntax, micro-interactions
- `development/frontend/react/react-patterns.md` - React patterns (if React project)
**Workflows**:
- `core/workflows/design-iteration-overview.md` - Design iteration workflow
---
## Best Practices
### Do's ✅
- ✅ **Propose before delegating** - Show user the plan first
- ✅ **Get approval** - Never delegate without user approval
- ✅ **Provide context** - List context files the subagent should load
- ✅ **Set clear requirements** - Specify design system, breakpoints, animations
- ✅ **Use staged workflow** - Let frontend-specialist follow its process
- ✅ **Trust the specialist** - It knows design patterns and best practices
### Don'ts ❌
- ❌ **Don't delegate simple edits** - Handle directly for efficiency
- ❌ **Don't skip approval** - Always get user sign-off first
- ❌ **Don't delegate backend work** - Wrong subagent
- ❌ **Don't micromanage** - Let the specialist follow its workflow
- ❌ **Don't delegate without context** - Provide context files to load
- ❌ **Don't delegate bug fixes** - Handle directly unless it's a design issue
---
## Examples
### Example 1: Landing Page (Delegate)
**User Request**: "Create a landing page for our SaaS product with a hero section, features, and pricing"
**Your Response**:
```markdown
## Implementation Plan
**Task**: Create SaaS landing page
**Approach**: Delegate to frontend-specialist subagent
**Why**:
- Requires full design workflow (layout, theme, animations)
- Needs responsive design across breakpoints
- Benefits from design system standards
**Context Needed**:
- Design system standards
- UI styling standards
- Animation patterns
**Approval needed before proceeding.**
```
**After Approval**:
```javascript
task(
subagent_type="frontend-specialist",
description="Create SaaS landing page",
prompt="Context to load:
- .opencode/context/ui/web/design-systems.md
- .opencode/context/ui/web/ui-styling-standards.md
- .opencode/context/ui/web/animation-patterns.md
Task: Create a SaaS landing page with:
- Hero section (headline, subheadline, CTA)
- Features section (3-column grid)
- Pricing section (3 tiers)
- Footer
Requirements:
- Tailwind CSS + Flowbite
- Mobile-first responsive
- Smooth scroll animations
- Modern design system
Follow staged workflow and request approval between stages."
)
```
### Example 2: Button Text Change (Direct)
**User Request**: "Change the 'Submit' button to say 'Send Message'"
**Your Response**:
```markdown
I'll update the button text directly.
```
**Action**: Read the HTML file, edit the button text, done. No delegation needed.
---
### Example 3: Design System Implementation (Delegate)
**User Request**: "Implement our design system using Tailwind and Shadcn components"
**Your Response**:
```markdown
## Implementation Plan
**Task**: Implement design system with Tailwind + Shadcn
**Approach**: Delegate to frontend-specialist subagent
**Why**:
- Requires UI library integration (Shadcn)
- Needs ExternalScout for current Shadcn docs
- Benefits from design system standards
**Context Needed**:
- Design system standards
- UI styling standards
- External docs (Shadcn)
**Approval needed before proceeding.**
```
**After Approval**:
```javascript
task(
subagent_type="frontend-specialist",
description="Implement Tailwind + Shadcn design system",
prompt="Context to load:
- .opencode/context/ui/web/design-systems.md
- .opencode/context/ui/web/ui-styling-standards.md
Task: Implement design system using Tailwind CSS and Shadcn/ui
Requirements:
- Call ExternalScout for current Shadcn docs
- Set up Tailwind config
- Integrate Shadcn components
- Create theme file with OKLCH colors
- Document component usage
Follow staged workflow and request approval between stages."
)
```
---
## Summary
**Delegate to frontend-specialist when**:
- New UI designs from scratch
- Design system implementation
- Complex responsive layouts
- Animation work
- UI library integration
- Multi-stage design iterations
**Handle directly when**:
- Simple HTML/CSS edits
- Bug fixes
- Content updates
- Single component updates
- Quick prototypes
**Always**:
- Propose approach first
- Get user approval
- Provide context files
- Trust the specialist's workflow
---
## Related Context
- **Frontend Specialist Agent** → `../../../agent/subagents/development/frontend-specialist.md`
- **Design Systems** → `../../ui/web/design-systems.md`
- **UI Styling Standards** → `../../ui/web/ui-styling-standards.md`
- **Animation Patterns** → `../../ui/web/animation-patterns.md`
- **Delegation Workflow** → `../../core/workflows/task-delegation-basics.md`
- **React Patterns** → `react/react-patterns.md`
@@ -1,73 +0,0 @@
# Full-Stack Development Navigation
**Scope**: End-to-end application development
---
## Common Stacks
### MERN (MongoDB, Express, React, Node)
```
Frontend: development/frontend/react/ [future]
Backend: development/backend/nodejs/express-patterns.md [future]
Data: development/data/nosql-patterns/mongodb.md [future]
API: development/backend/api-patterns/rest-design.md [future]
```
### T3 Stack (Next.js, tRPC, Prisma, Tailwind)
```
Frontend: development/frontend/react/ + ui/web/ui-styling-standards.md [future]
Backend: development/backend/nodejs/ + api-patterns/trpc-patterns.md [future]
Data: development/data/orm-patterns/prisma.md [future]
```
### Python Full-Stack (FastAPI + React)
```
Frontend: development/frontend/react/ [future]
Backend: development/backend/python/fastapi-patterns.md [future]
Data: development/data/sql-patterns/ or nosql-patterns/ [future]
API: development/backend/api-patterns/rest-design.md [future]
```
---
## Quick Routes
| Layer | Navigate To |
|-------|-------------|
| **Frontend** | `ui-navigation.md` |
| **Backend** | `backend-navigation.md` |
| **Data** | `data/navigation.md` [future] |
| **Integration** | `integration/navigation.md` [future] |
| **Infrastructure** | `infrastructure/navigation.md` [future] |
---
## Common Workflows
**New API endpoint**:
1. `principles/api-design.md` (principles)
2. `backend/api-patterns/rest-design.md` (approach) [future]
3. `backend/nodejs/express-patterns.md` (implementation) [future]
**New React feature**:
1. `frontend/react/component-architecture.md` (structure) [future]
2. `frontend/react/hooks-patterns.md` (logic) [future]
3. `ui/web/ui-styling-standards.md` (styling)
**Database integration**:
1. `data/sql-patterns/` or `data/nosql-patterns/` (approach) [future]
2. `data/orm-patterns/` (if using ORM) [future]
3. `backend/nodejs/` or `backend/python/` (implementation) [future]
**Third-party service**:
1. `integration/third-party-services/` (patterns) [future]
2. `integration/api-integration/` (consuming APIs) [future]
---
## Related Context
- **Clean Code** → `principles/clean-code.md`
- **API Design** → `principles/api-design.md`
- **Core Standards** → `../core/standards/navigation.md`
@@ -1,31 +0,0 @@
# Infrastructure Navigation
**Purpose**: DevOps and deployment patterns
**Status**: 🚧 Placeholder - Content coming soon
---
## Planned Structure
```
infrastructure/
├── navigation.md
│
├── docker/
│ ├── dockerfile-patterns.md
│ ├── compose-patterns.md
│ └── optimization.md
│
└── ci-cd/
├── github-actions.md
├── deployment-patterns.md
└── testing-pipelines.md
```
---
## Related Context
- **Core Standards** → `../../core/standards/code-quality.md`
- **Testing** → `../../core/standards/test-coverage.md`
@@ -1,36 +0,0 @@
# Integration Navigation
**Purpose**: Connecting systems and services
**Status**: 🚧 Placeholder - Content coming soon
---
## Planned Structure
```
integration/
├── navigation.md
│
├── package-management/
│ ├── npm-patterns.md
│ ├── pnpm-patterns.md
│ └── monorepo-patterns.md
│
├── api-integration/
│ ├── rest-client-patterns.md
│ ├── error-handling.md
│ └── retry-strategies.md
│
└── third-party-services/
├── stripe-integration.md
├── sendgrid-integration.md
└── cloudinary-integration.md
```
---
## Related Context
- **Backend Navigation** → `../backend-navigation.md`
- **API Design** → `../principles/api-design.md`
@@ -1,92 +0,0 @@
# Development Navigation
**Purpose**: Software development across all stacks
---
## Structure
```
development/
├── navigation.md
├── ui-navigation.md # Specialized
├── backend-navigation.md # Specialized
├── fullstack-navigation.md # Specialized
│
├── principles/ # Universal (language-agnostic)
│ ├── navigation.md
│ ├── clean-code.md
│ └── api-design.md
│
├── frameworks/ # Full-stack frameworks
│ ├── navigation.md
│ └── tanstack-start/
│
├── ai/ # AI & Agents
│ ├── navigation.md
│ └── mastra-ai/
│
├── frontend/ # Client-side
│ ├── navigation.md
│ ├── when-to-delegate.md # When to use frontend-specialist
│ └── react/
│ ├── navigation.md
│ └── react-patterns.md
│
├── backend/ # Server-side (future)
│ ├── navigation.md
│ ├── api-patterns/
│ ├── nodejs/
│ ├── python/
│ └── authentication/
│
├── data/ # Data layer (future)
│ ├── navigation.md
│ ├── sql-patterns/
│ ├── nosql-patterns/
│ └── orm-patterns/
│
├── integration/ # Connecting systems (future)
│ ├── navigation.md
│ ├── package-management/
│ ├── api-integration/
│ └── third-party-services/
│
└── infrastructure/ # DevOps (future)
├── navigation.md
├── docker/
└── ci-cd/
```
---
## Quick Routes
| Task | Path |
|------|------|
| **UI/Frontend** | `ui-navigation.md` |
| **When to delegate frontend** | `frontend/when-to-delegate.md` |
| **Backend/API** | `backend-navigation.md` |
| **Full-stack** | `fullstack-navigation.md` |
| **Clean code** | `principles/clean-code.md` |
| **API design** | `principles/api-design.md` |
---
## By Concern
**Principles** → Universal development practices
**Frameworks** → Full-stack frameworks (Tanstack Start, Next.js)
**AI** → AI frameworks and agent runtimes (MAStra AI)
**Frontend** → React patterns and component design
**Backend** → APIs, Node.js, Python, auth (future)
**Data** → SQL, NoSQL, ORMs (future)
**Integration** → Packages, APIs, services (future)
**Infrastructure** → Docker, CI/CD (future)
---
## Related Context
- **Core Standards** → `../core/standards/navigation.md`
- **UI Patterns** → `../ui/navigation.md`
@@ -1,415 +0,0 @@
# API Design Patterns
**Category**: development
**Purpose**: REST API design principles, GraphQL patterns, and API versioning strategies
**Used by**: opencoder
---
## Overview
This guide covers best practices for designing robust, scalable, and maintainable APIs, including REST, GraphQL, and versioning strategies.
## REST API Design
### 1. Resource-Based URLs
**Use nouns, not verbs**:
```
# Bad
GET /getUsers
POST /createUser
POST /updateUser/123
# Good
GET /users
POST /users
PUT /users/123
PATCH /users/123
DELETE /users/123
```
### 2. HTTP Methods
**Use appropriate HTTP methods**:
- `GET` - Retrieve resources (idempotent, safe)
- `POST` - Create new resources
- `PUT` - Replace entire resource (idempotent)
- `PATCH` - Partial update (idempotent)
- `DELETE` - Remove resource (idempotent)
### 3. Status Codes
**Use standard HTTP status codes**:
```
2xx Success
200 OK - Successful GET, PUT, PATCH
201 Created - Successful POST
204 No Content - Successful DELETE
4xx Client Errors
400 Bad Request - Invalid input
401 Unauthorized - Missing/invalid auth
403 Forbidden - Authenticated but not authorized
404 Not Found - Resource doesn't exist
409 Conflict - Resource conflict (e.g., duplicate)
422 Unprocessable Entity - Validation errors
5xx Server Errors
500 Internal Server Error - Unexpected error
503 Service Unavailable - Temporary unavailability
```
### 4. Consistent Response Format
**Standardize response structure**:
```json
// Success response
{
"data": {
"id": "123",
"name": "John Doe",
"email": "john@example.com"
},
"meta": {
"timestamp": "2024-01-01T00:00:00Z"
}
}
// Error response
{
"error": {
"code": "VALIDATION_ERROR",
"message": "Invalid input data",
"details": [
{
"field": "email",
"message": "Invalid email format"
}
]
},
"meta": {
"timestamp": "2024-01-01T00:00:00Z",
"requestId": "abc-123"
}
}
// Collection response
{
"data": [...],
"meta": {
"total": 100,
"page": 1,
"pageSize": 20,
"totalPages": 5
},
"links": {
"self": "/users?page=1",
"next": "/users?page=2",
"prev": null,
"first": "/users?page=1",
"last": "/users?page=5"
}
}
```
### 5. Filtering, Sorting, Pagination
**Support common query operations**:
```
# Filtering
GET /users?status=active&role=admin
# Sorting
GET /users?sort=createdAt:desc,name:asc
# Pagination
GET /users?page=2&pageSize=20
# Field selection
GET /users?fields=id,name,email
# Search
GET /users?q=john
```
### 6. Nested Resources
**Handle relationships appropriately**:
```
# Good - Shallow nesting
GET /users/123/posts
GET /posts?userId=123
# Avoid - Deep nesting
GET /users/123/posts/456/comments/789
# Better
GET /comments/789
```
## GraphQL Patterns
### 1. Schema Design
**Design clear, intuitive schemas**:
```graphql
type User {
id: ID!
name: String!
email: String!
posts: [Post!]!
createdAt: DateTime!
}
type Post {
id: ID!
title: String!
content: String!
author: User!
comments: [Comment!]!
publishedAt: DateTime
}
type Query {
user(id: ID!): User
users(filter: UserFilter, page: Int, pageSize: Int): UserConnection!
post(id: ID!): Post
}
type Mutation {
createUser(input: CreateUserInput!): User!
updateUser(id: ID!, input: UpdateUserInput!): User!
deleteUser(id: ID!): Boolean!
}
input CreateUserInput {
name: String!
email: String!
}
input UserFilter {
status: UserStatus
role: UserRole
search: String
}
```
### 2. Resolver Patterns
**Implement efficient resolvers**:
```javascript
const resolvers = {
Query: {
user: async (_, { id }, { dataSources }) => {
return dataSources.userAPI.getUser(id);
},
users: async (_, { filter, page, pageSize }, { dataSources }) => {
return dataSources.userAPI.getUsers({ filter, page, pageSize });
}
},
User: {
posts: async (user, _, { dataSources }) => {
// Use DataLoader to batch requests
return dataSources.postAPI.getPostsByUserId(user.id);
}
},
Mutation: {
createUser: async (_, { input }, { dataSources, user }) => {
// Check authorization
if (!user) throw new AuthenticationError('Not authenticated');
// Validate input
const validatedInput = validateUserInput(input);
// Create user
return dataSources.userAPI.createUser(validatedInput);
}
}
};
```
### 3. DataLoader for N+1 Prevention
**Batch and cache database queries**:
```javascript
import DataLoader from 'dataloader';
const userLoader = new DataLoader(async (userIds) => {
const users = await db.users.findMany({
where: { id: { in: userIds } }
});
// Return in same order as input
return userIds.map(id => users.find(u => u.id === id));
});
// Usage in resolver
const user = await userLoader.load(userId);
```
## Frontend API Client Patterns (TanStack Query)
**Use TanStack Query for optimal client-side API consumption**:
### REST Integration
```javascript
// Optimal REST client with TanStack Query v5
import { useQuery, useMutation, useQueryClient } from '@tanstack/react-query';
const apiClient = {
getUsers: (filters) =>
fetch(`/api/v1/users?${new URLSearchParams(filters)}`).then(r => r.json())
};
function UsersList() {
const { data, isPending, error } = useQuery({
queryKey: ['users', filters],
queryFn: () => apiClient.getUsers(filters),
staleTime: 5 * 60 * 1000, // 5 minutes
});
return (
<div>
{isPending && <div>Loading...</div>}
{error && <div>Error: {error.message}</div>}
{data?.data.map(user => <UserCard key={user.id} user={user} />)}
</div>
);
}
## API Versioning
### 1. URL Versioning
**Version in the URL path**:
```
GET /v1/users
GET /v2/users
```
**Pros**: Clear, easy to route
**Cons**: URL changes, harder to maintain multiple versions
### 2. Header Versioning
**Version in Accept header**:
```
GET /users
Accept: application/vnd.myapi.v2+json
```
**Pros**: Clean URLs, flexible
**Cons**: Less visible, harder to test
### 3. Deprecation Strategy
**Communicate deprecation clearly**:
```javascript
// Response headers
Deprecation: true
Sunset: Sat, 31 Dec 2024 23:59:59 GMT
Link: <https://api.example.com/v2/users>; rel="successor-version"
// Response body
{
"data": {...},
"meta": {
"deprecated": true,
"deprecationDate": "2024-12-31",
"migrationGuide": "https://docs.example.com/migration/v1-to-v2"
}
}
```
## Authentication & Authorization
### 1. JWT Tokens
**Use JWT for stateless auth**:
```javascript
// Token structure
{
"sub": "user-123",
"email": "user@example.com",
"role": "admin",
"iat": 1516239022,
"exp": 1516242622
}
// Middleware
function authenticateToken(req, res, next) {
const token = req.headers.authorization?.split(' ')[1];
if (!token) {
return res.status(401).json({ error: 'No token provided' });
}
try {
const decoded = jwt.verify(token, process.env.JWT_SECRET);
req.user = decoded;
next();
} catch (error) {
return res.status(401).json({ error: 'Invalid token' });
}
}
```
### 2. Role-Based Access Control
**Implement RBAC**:
```javascript
function authorize(...roles) {
return (req, res, next) => {
if (!req.user) {
return res.status(401).json({ error: 'Not authenticated' });
}
if (!roles.includes(req.user.role)) {
return res.status(403).json({ error: 'Insufficient permissions' });
}
next();
};
}
// Usage
app.delete('/users/:id',
authenticateToken,
authorize('admin'),
deleteUser
);
```
## Best Practices
1. **Use HTTPS everywhere** - Encrypt all API traffic
2. **Implement rate limiting** - Prevent abuse and ensure fair usage
3. **Validate all inputs** - Never trust client data
4. **Use proper error handling** - Return meaningful error messages
5. **Document your API** - Use OpenAPI/Swagger or GraphQL introspection
6. **Version your API** - Plan for breaking changes
7. **Implement CORS properly** - Configure allowed origins carefully
8. **Log requests and errors** - Enable debugging and monitoring
9. **Use caching** - Implement ETags, Cache-Control headers
10. **Test thoroughly** - Unit, integration, and contract tests
## Anti-Patterns
- ❌ **Exposing internal IDs** - Use UUIDs or opaque identifiers
- ❌ **Returning too much data** - Support field selection
- ❌ **Ignoring idempotency** - PUT/PATCH/DELETE should be idempotent
- ❌ **Inconsistent naming** - Use camelCase or snake_case consistently
- ❌ **Missing pagination** - Always paginate collections
- ❌ **No rate limiting** - Protect against abuse
- ❌ **Verbose error messages** - Don't leak implementation details
- ❌ **Synchronous long operations** - Use async jobs for long tasks
## References
- REST API Design Rulebook by Mark Masse
- GraphQL Best Practices (graphql.org)
- API Design Patterns by JJ Geewax
- OpenAPI Specification (swagger.io)
@@ -1,176 +0,0 @@
# Clean Code Principles
**Category**: development
**Purpose**: Core coding standards and best practices for writing clean, maintainable code
**Used by**: frontend-specialist, devops-specialist, opencoder
---
## Overview
Clean code is code that is easy to read, understand, and maintain. It follows consistent patterns, uses meaningful names, and is well-organized. This guide provides principles and patterns for writing clean code across all languages.
## Core Principles
### 1. Meaningful Names
**Use intention-revealing names**:
- Variable names should reveal intent
- Function names should describe what they do
- Class names should describe what they represent
**Examples**:
```javascript
// Bad
const d = new Date();
const x = getUserData();
// Good
const currentDate = new Date();
const activeUserProfile = getUserData();
```
### 2. Functions Should Do One Thing
**Single Responsibility**:
- Each function should have one clear purpose
- Functions should be small (ideally < 20 lines)
- Extract complex logic into separate functions
**Example**:
```javascript
// Bad
function processUser(user) {
validateUser(user);
saveToDatabase(user);
sendEmail(user);
logActivity(user);
}
// Good
function processUser(user) {
const validatedUser = validateUser(user);
const savedUser = saveUserToDatabase(validatedUser);
notifyUser(savedUser);
return savedUser;
}
```
### 3. Avoid Deep Nesting
**Keep nesting shallow**:
- Use early returns
- Extract nested logic into functions
- Prefer guard clauses
**Example**:
```javascript
// Bad
function processOrder(order) {
if (order) {
if (order.items.length > 0) {
if (order.total > 0) {
// process order
}
}
}
}
// Good
function processOrder(order) {
if (!order) return;
if (order.items.length === 0) return;
if (order.total <= 0) return;
// process order
}
```
### 4. DRY (Don't Repeat Yourself)
**Eliminate duplication**:
- Extract common logic into reusable functions
- Use composition over inheritance
- Create utility functions for repeated patterns
### 5. Error Handling
**Handle errors explicitly**:
- Use try-catch for expected errors
- Provide meaningful error messages
- Don't ignore errors silently
**Example**:
```javascript
// Bad
function fetchData() {
try {
return api.getData();
} catch (e) {
return null;
}
}
// Good
async function fetchData() {
try {
return await api.getData();
} catch (error) {
logger.error('Failed to fetch data', { error });
throw new DataFetchError('Unable to retrieve data', { cause: error });
}
}
```
## Best Practices
1. **Write self-documenting code** - Code should explain itself through clear naming and structure
2. **Keep functions pure when possible** - Avoid side effects, return new values instead of mutating
3. **Use consistent formatting** - Follow language-specific style guides (Prettier, ESLint, etc.)
4. **Write tests first** - TDD helps design better APIs and catch issues early
5. **Refactor regularly** - Improve code structure as you learn more about the domain
6. **Comment why, not what** - Code shows what, comments explain why
7. **Use type systems** - TypeScript, type hints, or static analysis tools
8. **Favor composition** - Build complex behavior from simple, reusable pieces
## Anti-Patterns
- ❌ **Magic numbers** - Use named constants instead of hardcoded values
- ❌ **God objects** - Classes that do too much or know too much
- ❌ **Premature optimization** - Optimize for readability first, performance second
- ❌ **Clever code** - Simple and clear beats clever and complex
- ❌ **Long parameter lists** - Use objects or configuration patterns instead
- ❌ **Boolean flags** - Often indicate a function doing multiple things
- ❌ **Mutable global state** - Leads to unpredictable behavior and bugs
## Language-Specific Guidelines
### JavaScript/TypeScript
- Use `const` by default, `let` when needed, never `var`
- Prefer arrow functions for callbacks
- Use async/await over raw promises
- Destructure objects and arrays for clarity
### Python
- Follow PEP 8 style guide
- Use list comprehensions for simple transformations
- Prefer context managers (`with` statements)
- Use type hints for function signatures
### Go
- Follow effective Go guidelines
- Use defer for cleanup
- Handle errors explicitly
- Keep interfaces small
### Rust
- Embrace ownership and borrowing
- Use pattern matching
- Prefer iterators over loops
- Handle errors with Result types
## References
- Clean Code by Robert C. Martin
- The Pragmatic Programmer by Hunt & Thomas
- Refactoring by Martin Fowler
@@ -1,44 +0,0 @@
# Development Principles Navigation
**Purpose**: Universal development principles (language-agnostic)
---
## Files
| File | Topic | Priority | Load When |
|------|-------|----------|-----------|
| `clean-code.md` | Clean code practices | ⭐⭐⭐⭐ | Writing any code |
| `api-design.md` | API design principles | ⭐⭐⭐⭐ | Designing APIs |
---
## Loading Strategy
**For general development**:
1. Load `clean-code.md` (high)
2. Also load: `../../core/standards/code-quality.md` (critical)
**For API development**:
1. Load `api-design.md` (high)
2. Also load: `../../core/standards/code-quality.md` (critical)
---
## Scope
**This directory**: Development-specific principles
**Core standards**: Universal standards (all projects, all languages)
| Location | Scope | Examples |
|----------|-------|----------|
| `core/standards/` | **Universal** (all projects) | Code quality, testing, docs, security |
| `development/principles/` | **Development-specific** | Clean code, API design, error handling |
---
## Related
- **Core Standards** → `../../core/standards/navigation.md`
- **Backend Patterns** → `../backend-navigation.md`
- **Frontend Patterns** → `../ui-navigation.md`
@@ -1,51 +0,0 @@
# UI Development Navigation
**Scope**: Frontend code + visual design
---
## Structure
```
Frontend Code (development/frontend/):
└── react/
├── navigation.md
└── react-patterns.md
Visual Design (ui/web/):
├── animation-patterns.md
├── ui-styling-standards.md
├── design-systems.md
└── design/
├── concepts/
└── examples/
```
---
## Quick Routes
| Task | Path |
|------|------|
| **React patterns** | `frontend/react/react-patterns.md` |
| **Animations** | `../../ui/web/animation-patterns.md` |
| **Styling** | `../../ui/web/ui-styling-standards.md` |
| **Design systems** | `../../ui/web/design-systems.md` |
---
## By Framework
**React** → `frontend/react/`
## By Concern
**Code patterns** → `development/frontend/`
**Visual design** → `ui/web/`
---
## Related Context
- **Core Standards** → `../core/standards/code-quality.md`
- **UI Category** → `../ui/navigation.md`
-47
View File
@@ -1,47 +0,0 @@
# Context Navigation
**New here?** → `openagents-repo/quick-start.md`
---
## Structure
```
.opencode/context/
├── core/ # Universal standards & workflows
├── openagents-repo/ # OpenAgents Control repository work
├── development/ # Software development (all stacks)
├── ui/ # Visual design & UX
├── content-creation/ # Content creation (all formats)
├── data/ # Data engineering & analytics
├── product/ # Product management
└── learning/ # Educational content
```
---
## Quick Routes
| Task | Path |
|------|------|
| **Write code** | `core/standards/code-quality.md` |
| **Write tests** | `core/standards/test-coverage.md` |
| **Write docs** | `core/standards/documentation.md` |
| **Review code** | `core/workflows/code-review.md` |
| **Delegate task** | `core/workflows/task-delegation-basics.md` |
| **Add agent** | `openagents-repo/guides/adding-agent.md` |
| **UI development** | `development/ui-navigation.md` |
| **API development** | `development/backend-navigation.md` |
---
## By Category
**core/** - Standards, workflows, patterns → `core/navigation.md`
**openagents-repo/** - Repository-specific → `openagents-repo/navigation.md`
**development/** - All development → `development/navigation.md`
**ui/** - Design & UX → `ui/navigation.md`
**content-creation/** - Content creation (all formats) → `content-creation/navigation.md`
**data/** - Data engineering → `data/navigation.md`
**product/** - Product management → `product/navigation.md`
**learning/** - Educational → `learning/navigation.md`
@@ -1,255 +0,0 @@
---
description: "Template for creating context bundles when delegating tasks to subagents"
type: "context"
category: "openagents-repo"
tags: [template, delegation, context]
---
# Context Bundle Template
**Purpose**: Template for creating context bundles when delegating tasks to subagents
**Location**: `.tmp/context/{session-id}/bundle.md`
**Used by**: repo-manager agent when delegating to subagents
---
## Template
```markdown
# Context Bundle: {Task Name}
Session: {session-id}
Created: {ISO timestamp}
For: {subagent-name}
Status: in_progress
## Task Overview
{Brief description of what we're building/doing}
## User Request
{Original user request - what they asked for}
## Relevant Standards (Load These Before Starting)
**Core Standards**:
- `.opencode/context/core/standards/code-quality.md` → Modular, functional code patterns
- `.opencode/context/core/standards/test-coverage.md` → Testing requirements and TDD
- `.opencode/context/core/standards/documentation.md` → Documentation standards
- `.opencode/context/core/standards/security-patterns.md` → Error handling, security patterns
**Core Workflows**:
- `.opencode/context/core/workflows/task-delegation-basics.md` → Delegation process
- `.opencode/context/core/workflows/feature-breakdown.md` → Task breakdown methodology
- `.opencode/context/core/workflows/code-review.md` → Code review guidelines
## Repository-Specific Context (Load These Before Starting)
**Quick Start** (ALWAYS load first):
- `.opencode/context/openagents-repo/quick-start.md` → Repo orientation and common commands
**Core Concepts** (Load based on task type):
- `.opencode/context/openagents-repo/core-concepts/agents.md` → How agents work
- `.opencode/context/openagents-repo/core-concepts/evals.md` → How testing works
- `.opencode/context/openagents-repo/core-concepts/registry.md` → How registry works
- `.opencode/context/openagents-repo/core-concepts/categories.md` → How organization works
**Guides** (Load for specific workflows):
- `.opencode/context/openagents-repo/guides/adding-agent.md` → Step-by-step agent creation
- `.opencode/context/openagents-repo/guides/testing-agent.md` → Testing workflow
- `.opencode/context/openagents-repo/guides/updating-registry.md` → Registry workflow
- `.opencode/context/openagents-repo/guides/debugging.md` → Troubleshooting
**Lookup** (Quick reference):
- `.opencode/context/openagents-repo/lookup/file-locations.md` → Where everything is
- `.opencode/context/openagents-repo/lookup/commands.md` → Command reference
## Key Requirements
{Extract key requirements from loaded context}
**From Standards**:
- {requirement 1 from standards/code-quality.md}
- {requirement 2 from standards/test-coverage.md}
- {requirement 3 from standards/documentation.md}
**From Repository Context**:
- {requirement 1 from repo context}
- {requirement 2 from repo context}
- {requirement 3 from repo context}
**Naming Conventions**:
- {convention 1}
- {convention 2}
**File Structure**:
- {structure requirement 1}
- {structure requirement 2}
## Technical Constraints
{List technical constraints and limitations}
- {constraint 1 - e.g., "Must use TypeScript"}
- {constraint 2 - e.g., "Must follow category-based organization"}
- {constraint 3 - e.g., "Must include proper frontmatter metadata"}
## Files to Create/Modify
{List all files that need to be created or modified}
**Create**:
- `{file-path-1}` - {purpose and what it should contain}
- `{file-path-2}` - {purpose and what it should contain}
**Modify**:
- `{file-path-3}` - {what needs to be changed}
- `{file-path-4}` - {what needs to be changed}
## Success Criteria
{Define what "done" looks like - binary pass/fail conditions}
- [ ] {criteria 1 - e.g., "Agent file created with proper frontmatter"}
- [ ] {criteria 2 - e.g., "Eval tests pass"}
- [ ] {criteria 3 - e.g., "Registry validation passes"}
- [ ] {criteria 4 - e.g., "Documentation updated"}
## Validation Requirements
{How to validate the work}
**Scripts to Run**:
- `{validation-script-1}` - {what it validates}
- `{validation-script-2}` - {what it validates}
**Tests to Run**:
- `{test-command-1}` - {what it tests}
- `{test-command-2}` - {what it tests}
**Manual Checks**:
- {check 1}
- {check 2}
## Expected Output
{What the subagent should produce}
**Deliverables**:
- {deliverable 1}
- {deliverable 2}
**Format**:
- {format requirement 1}
- {format requirement 2}
## Progress Tracking
{Track progress through the task}
- [ ] Context loaded and understood
- [ ] {step 1}
- [ ] {step 2}
- [ ] {step 3}
- [ ] Validation passed
- [ ] Documentation updated
---
## Instructions for Subagent
{Specific, detailed instructions for the subagent}
**IMPORTANT**:
1. Load ALL context files listed in "Relevant Standards" and "Repository-Specific Context" sections BEFORE starting work
2. Follow ALL requirements from the loaded context
3. Apply naming conventions and file structure requirements
4. Validate your work using the validation requirements
5. Update progress tracking as you complete steps
**Your Task**:
{Detailed description of what the subagent needs to do}
**Approach**:
{Suggested approach or methodology}
**Constraints**:
{Any additional constraints or notes}
**Questions/Clarifications**:
{Any questions the subagent should consider or clarifications needed}
```
---
## Usage Instructions
### When to Create a Context Bundle
Create a context bundle when:
- Delegating to any subagent
- Task requires coordination across multiple components
- Subagent needs project-specific context
- Task has complex requirements or constraints
### How to Create a Context Bundle
1. **Create session directory**:
```bash
mkdir -p .tmp/context/{session-id}
```
2. **Copy template**:
```bash
cp .opencode/context/openagents-repo/templates/context-bundle-template.md \
.tmp/context/{session-id}/bundle.md
```
3. **Fill in all sections**:
- Replace all `{placeholders}` with actual values
- List specific context files to load (with full paths)
- Extract key requirements from loaded context
- Define clear success criteria
- Provide specific instructions
4. **Pass to subagent**:
```javascript
task(
subagent_type="{SubagentName}",
description="Brief description",
prompt="Load context from .tmp/context/{session-id}/bundle.md before starting.
{Specific task instructions}
Follow all standards and requirements in the context bundle."
)
```
### Best Practices
**DO**:
- ✅ List context files with full paths (don't duplicate content)
- ✅ Extract key requirements from loaded context
- ✅ Define binary success criteria (pass/fail)
- ✅ Provide specific validation requirements
- ✅ Include clear instructions for subagent
- ✅ Track progress through the task
**DON'T**:
- ❌ Duplicate full context file content (just reference paths)
- ❌ Use vague success criteria ("make it good")
- ❌ Skip validation requirements
- ❌ Forget to list technical constraints
- ❌ Omit file paths for files to create/modify
### Example Context Bundle
See `.opencode/context/openagents-repo/examples/context-bundle-example.md` for a complete example.
---
**Last Updated**: 2025-01-21
**Version**: 1.0.0
@@ -1,37 +0,0 @@
# OpenAgents Blueprints
**Purpose**: Blueprint templates and patterns for OpenAgents Control
---
## Structure
```
openagents-repo/blueprints/
├── navigation.md (this file)
└── [blueprint files]
```
---
## Quick Routes
| Task | Path |
|------|------|
| **View blueprints** | `./` |
| **Core concepts** | `../core-concepts/navigation.md` |
| **Examples** | `../examples/navigation.md` |
---
## By Type
**Blueprints** → Template patterns for OpenAgents implementations
---
## Related Context
- **OpenAgents Navigation** → `../navigation.md`
- **Core Concepts** → `../core-concepts/navigation.md`
- **Examples** → `../examples/navigation.md`
@@ -1,38 +0,0 @@
# OpenAgents Concepts
**Purpose**: Core concepts and ideas for OpenAgents Control
---
## Structure
```
openagents-repo/concepts/
├── navigation.md (this file)
└── [concept files]
```
---
## Quick Routes
| Task | Path |
|------|------|
| **View concepts** | `./` |
| **Core concepts** | `../core-concepts/navigation.md` |
| **Guides** | `../guides/navigation.md` |
---
## By Type
**Concepts** → Foundational concepts for OpenAgents
**Core Concepts** → Deep dives into core ideas
---
## Related Context
- **OpenAgents Navigation** → `../navigation.md`
- **Core Concepts** → `../core-concepts/navigation.md`
- **Guides** → `../guides/navigation.md`
@@ -1,131 +0,0 @@
# Subagent Testing Modes
**Purpose**: Understand the two ways to test subagents (standalone vs delegation)
**Last Updated**: 2026-01-07
---
## Core Concept
Subagents have **two distinct testing modes** depending on what you're validating:
1. **Standalone Mode** - Test subagent logic directly (unit testing)
2. **Delegation Mode** - Test parent → subagent workflow (integration testing)
The mode determines which agent runs and how tools are used.
---
## Standalone Mode (Unit Testing)
**Purpose**: Test subagent's logic in isolation
**Command**:
```bash
npm run eval:sdk -- --subagent=ContextScout
```
**What Happens**:
- Eval framework forces `mode: primary` (overrides `mode: subagent`)
- ContextScout runs as the primary agent
- ContextScout uses tools directly (glob, read, grep, list)
- No parent agent involved
**Use For**:
- Unit testing subagent logic
- Debugging tool usage
- Feature development
- Verifying prompt changes
**Test Location**: `evals/agents/subagents/core/{subagent}/tests/standalone/`
---
## Delegation Mode (Integration Testing)
**Purpose**: Test real production workflow (parent delegates to subagent)
**Command**:
```bash
npm run eval:sdk -- --agent=core/openagent --pattern="delegation/*.yaml"
```
**What Happens**:
- OpenAgent runs as primary agent
- OpenAgent uses `task` tool to delegate to ContextScout
- ContextScout runs with `mode: subagent` (natural mode)
- Tests full delegation workflow
**Use For**:
- Integration testing
- Validating production behavior
- Testing delegation logic
- End-to-end workflows
**Test Location**: `evals/agents/subagents/core/{subagent}/tests/delegation/`
---
## Critical Distinction
| Aspect | Standalone Mode | Delegation Mode |
|--------|----------------|-----------------|
| **Flag** | `--subagent=NAME` | `--agent=PARENT` |
| **Agent Mode** | Forced to `primary` | Natural `subagent` |
| **Who Runs** | Subagent directly | Parent → Subagent |
| **Tool Usage** | Subagent uses tools | Parent uses `task` tool |
| **Tests** | `standalone/*.yaml` | `delegation/*.yaml` |
**Common Mistake**:
```bash
# ❌ WRONG - This runs OpenAgent, not ContextScout
npm run eval:sdk -- --agent=ContextScout
# ✅ CORRECT - This runs ContextScout directly
npm run eval:sdk -- --subagent=ContextScout
```
---
## How to Verify Correct Mode
### Standalone Mode Indicators:
```
⚡ Standalone Test Mode
Subagent: contextscout
Mode: Forced to 'primary' for direct testing
```
### Delegation Mode Indicators:
```
Testing agent: core/openagent
🎯 PARENT: OpenAgent
Delegating to: contextscout
```
---
## When to Use Each Mode
**Use Standalone When**:
- Testing subagent's core logic
- Debugging why subagent isn't using tools
- Validating prompt changes
- Quick iteration during development
**Use Delegation When**:
- Testing production workflow
- Validating parent → subagent communication
- Testing context passing
- Integration testing
---
## Related
- `guides/testing-subagents.md` - Step-by-step testing guide
- `lookup/subagent-test-commands.md` - Quick command reference
- `errors/tool-permission-errors.md` - Common testing issues
**Reference**: `evals/framework/src/sdk/run-sdk-tests.ts` (mode forcing logic)
@@ -1,559 +0,0 @@
<!-- Context: openagents-repo/core-concepts/agent-metadata | Priority: critical | Version: 1.0 | Updated: 2026-01-31 -->
# Core Concept: Agent Metadata System
**Purpose**: Understanding the centralized metadata system for OpenAgents Control
**Priority**: CRITICAL - Load this before working with agent metadata
---
## What Is the Agent Metadata System?
The agent metadata system separates **OpenCode-compliant agent configuration** from **OpenAgents Control registry metadata**. This solves the problem of OpenCode validation errors when agents contain fields that aren't part of the OpenCode agent schema.
**Key Principle**: Agent frontmatter contains ONLY valid OpenCode fields. All other metadata lives in a centralized file.
---
## The Problem We Solved
### Before (Validation Errors)
Agent frontmatter contained fields that OpenCode doesn't recognize:
```yaml
---
id: opencoder # ❌ Not valid OpenCode field
name: OpenCoder # ❌ Not valid OpenCode field
category: core # ❌ Not valid OpenCode field
type: core # ❌ Not valid OpenCode field
version: 1.0.0 # ❌ Not valid OpenCode field
author: opencode # ❌ Not valid OpenCode field
tags: [development, coding] # ❌ Not valid OpenCode field
dependencies: [] # ❌ Not valid OpenCode field
description: "..." # ✅ Valid OpenCode field
mode: primary # ✅ Valid OpenCode field
temperature: 0.1 # ✅ Valid OpenCode field
tools: {...} # ✅ Valid OpenCode field
permission: {...} # ✅ Valid OpenCode field
---
```
**Result**: OpenCode validation errors:
```
Extra inputs are not permitted, field: 'id', value: 'opencoder'
Extra inputs are not permitted, field: 'category', value: 'core'
Extra inputs are not permitted, field: 'type', value: 'core'
... (9 validation errors)
```
### After (Clean Separation)
**Agent frontmatter** (`.opencode/agent/core/opencoder.md`):
```yaml
---
# Metadata stored in: .opencode/config/agent-metadata.json
description: "Orchestration agent for complex coding, architecture, and multi-file refactoring"
mode: primary
temperature: 0.1
tools: {...}
permission: {...}
---
```
**Centralized metadata** (`.opencode/config/agent-metadata.json`):
```json
{
"agents": {
"opencoder": {
"id": "opencoder",
"name": "OpenCoder",
"category": "core",
"type": "agent",
"version": "1.0.0",
"author": "opencode",
"tags": ["development", "coding", "implementation"],
"dependencies": [
"subagent:documentation",
"subagent:coder-agent",
"context:core/standards/code"
]
}
}
}
```
**Result**: ✅ No validation errors, clean separation of concerns
---
## Valid OpenCode Agent Fields
Based on [OpenCode documentation](https://opencode.ai/docs/agents/), these are the ONLY valid frontmatter fields:
### Required Fields
- `description` - When to use this agent (required)
- `mode` - Agent type: `primary`, `subagent`, or `all` (defaults to `all`)
### Optional Fields
- `model` - Model override (e.g., `anthropic/claude-sonnet-4-20250514`)
- `temperature` - Response randomness (0.0-1.0)
- `maxSteps` - Max agentic iterations
- `disable` - Set to `true` to disable agent
- `prompt` - Custom prompt file path (e.g., `{file:./prompts/build.txt}`)
- `hidden` - Hide from @ autocomplete (subagents only)
- `tools` - Tool access configuration
- `permission` - Permission rules for tools (v1.1.1+, replaces deprecated `permissions`)
### Example Valid Frontmatter
```yaml
---
description: "Code review agent with security focus"
mode: subagent
model: anthropic/claude-sonnet-4-20250514
temperature: 0.1
tools:
read: true
grep: true
glob: true
write: false
edit: false
permission: # v1.1.1+ (singular, not plural)
bash:
"*": ask
"git *": allow
edit: deny
---
```
---
## Centralized Metadata File
**Location**: `.opencode/config/agent-metadata.json`
### Schema
```json
{
"$schema": "https://opencode.ai/schemas/agent-metadata.json",
"schema_version": "1.0.0",
"description": "Centralized metadata for OpenAgents Control agents",
"agents": {
"agent-id": {
"id": "agent-id",
"name": "Agent Name",
"category": "core|development|content|data|product|learning|meta",
"type": "agent|subagent",
"version": "1.0.0",
"author": "opencode",
"tags": ["tag1", "tag2"],
"dependencies": [
"subagent:subagent-id",
"context:path/to/context"
]
}
},
"defaults": {
"agent": {
"version": "1.0.0",
"author": "opencode",
"type": "agent",
"tags": []
},
"subagent": {
"version": "1.0.0",
"author": "opencode",
"type": "subagent",
"tags": []
}
}
}
```
### Metadata Fields
| Field | Required | Description | Example |
|-------|----------|-------------|---------|
| `id` | Yes | Unique identifier (kebab-case) | `"opencoder"` |
| `name` | Yes | Display name | `"OpenCoder"` |
| `category` | Yes | Agent category | `"core"` |
| `type` | Yes | Component type | `"agent"` or `"subagent"` |
| `version` | Yes | Version number | `"1.0.0"` |
| `author` | Yes | Author identifier | `"opencode"` |
| `tags` | No | Discovery tags | `["development", "coding"]` |
| `dependencies` | No | Component dependencies | `["subagent:tester"]` |
---
## How It Works
### 1. Agent Creation
When creating a new agent:
**Step 1**: Create agent file with ONLY valid OpenCode fields
```bash
# Create agent file
touch .opencode/agent/category/my-agent.md
```
```yaml
---
description: "My agent description"
mode: subagent
temperature: 0.2
tools:
read: true
write: true
---
# Agent prompt content here
```
**Step 2**: Add metadata to `.opencode/config/agent-metadata.json`
```json
{
"agents": {
"my-agent": {
"id": "my-agent",
"name": "My Agent",
"category": "development",
"type": "subagent",
"version": "1.0.0",
"author": "opencode",
"tags": ["custom", "helper"],
"dependencies": ["context:core/standards/code"]
}
}
}
```
**Step 3**: Run auto-detect to update registry
```bash
./scripts/registry/auto-detect-components.sh --auto-add
```
The auto-detect script:
1. Reads frontmatter from agent file (description, mode, etc.)
2. Reads metadata from `agent-metadata.json` (id, name, tags, dependencies)
3. Merges both into registry.json entry
### 2. Auto-Detect Integration
The auto-detect script (`scripts/registry/auto-detect-components.sh`) has been enhanced to:
1. **Extract frontmatter** - Read description from agent file
2. **Lookup metadata** - Check `agent-metadata.json` for agent ID
3. **Merge data** - Combine frontmatter + metadata
4. **Update registry** - Write complete entry to registry.json
**Code snippet** (from auto-detect script):
```bash
# Check if agent-metadata.json exists and merge metadata from it
local metadata_file="$REPO_ROOT/.opencode/config/agent-metadata.json"
if [ -f "$metadata_file" ] && command -v jq &> /dev/null; then
# Try to find metadata for this agent ID
local metadata_entry
metadata_entry=$(jq -r ".agents[\"$id\"] // empty" "$metadata_file" 2>/dev/null)
if [ -n "$metadata_entry" ] && [ "$metadata_entry" != "null" ]; then
# Merge name, tags, dependencies from metadata
# ...
fi
fi
```
### 3. Registry Output
The registry.json entry contains merged data:
```json
{
"id": "opencoder",
"name": "OpenCoder",
"type": "agent",
"path": ".opencode/agent/core/opencoder.md",
"description": "Orchestration agent for complex coding...",
"category": "core",
"tags": ["development", "coding", "implementation"],
"dependencies": [
"subagent:documentation",
"subagent:coder-agent",
"context:core/standards/code"
]
}
```
---
## Workflow
### Adding a New Agent
```bash
# 1. Create agent file (OpenCode-compliant frontmatter only)
vim .opencode/agent/category/my-agent.md
# 2. Add metadata entry
vim .opencode/config/agent-metadata.json
# 3. Update registry
./scripts/registry/auto-detect-components.sh --auto-add
# 4. Validate
./scripts/registry/validate-registry.sh
```
### Updating Agent Metadata
**To update OpenCode configuration** (tools, permissions, temperature):
```bash
# Edit agent file frontmatter
vim .opencode/agent/category/my-agent.md
```
**To update registry metadata** (tags, dependencies, version):
```bash
# Edit metadata file
vim .opencode/config/agent-metadata.json
# Re-run auto-detect
./scripts/registry/auto-detect-components.sh --auto-add
```
### Updating Dependencies
**Add a dependency**:
```json
{
"agents": {
"my-agent": {
"dependencies": [
"subagent:tester",
"context:core/standards/code",
"subagent:new-dependency" // ← Add here
]
}
}
}
```
Then run:
```bash
./scripts/registry/auto-detect-components.sh --auto-add
./scripts/registry/validate-registry.sh
```
---
## Benefits
### ✅ OpenCode Compliance
- Agent frontmatter contains ONLY valid OpenCode fields
- No validation errors from OpenCode
- Agents work correctly with OpenCode CLI
### ✅ Registry Compatibility
- Registry still has all metadata (id, name, category, tags, dependencies)
- Auto-detect script merges frontmatter + metadata
- Backward compatible with existing tools
### ✅ Single Source of Truth
- Metadata centralized in one file
- Easy to update dependencies across multiple agents
- Clear separation: OpenCode config vs. registry metadata
### ✅ Maintainability
- Update dependencies in one place
- Consistent metadata across all agents
- Easy to add new metadata fields
### ✅ Validation
- OpenCode validates frontmatter (no extra fields)
- Registry validator checks dependencies exist
- Clear error messages when metadata is missing
---
## Migration Guide
### Migrating from permissions (plural) to permission (singular)
**OpenCode v1.1.1+ Change**: The field name changed from `permissions:` (plural) to `permission:` (singular).
**Before** (deprecated):
```yaml
permissions:
bash:
"*": "deny"
```
**After** (v1.1.1+):
```yaml
permission:
bash:
"*": "deny"
```
**Migration Steps**:
1. Find all agents using `permissions:` (plural)
```bash
grep -r "^permissions:" .opencode/agent/
```
2. Replace with `permission:` (singular) in each file
3. Verify no validation errors:
```bash
opencode agent validate
```
### Migrating Existing Agents
**Step 1**: Identify agents with extra fields
```bash
# Find agents with invalid OpenCode fields
grep -r "^id:\|^name:\|^category:\|^type:\|^version:\|^author:\|^tags:\|^dependencies:" .opencode/agent/
```
**Step 2**: Extract metadata to `agent-metadata.json`
For each agent:
1. Copy `id`, `name`, `category`, `type`, `version`, `author`, `tags`, `dependencies` to metadata file
2. Remove these fields from agent frontmatter
3. Keep ONLY valid OpenCode fields in frontmatter
**Step 3**: Update registry
```bash
# Remove old entries
jq 'del(.components.agents[] | select(.id == "agent-id"))' registry.json > tmp.json && mv tmp.json registry.json
# Re-add with new metadata
./scripts/registry/auto-detect-components.sh --auto-add
```
**Step 4**: Validate
```bash
./scripts/registry/validate-registry.sh
```
---
## Best Practices
### Agent Frontmatter
✅ **DO**:
- Keep frontmatter minimal (only OpenCode fields)
- Add comment pointing to metadata file
- Use consistent formatting
❌ **DON'T**:
- Add custom fields to frontmatter
- Duplicate metadata in both places
- Skip the metadata file
### Metadata File
✅ **DO**:
- Keep metadata file in version control
- Update metadata when adding/removing dependencies
- Use consistent naming (kebab-case for IDs)
- Document why dependencies exist
❌ **DON'T**:
- Forget to update metadata when creating agents
- Leave orphaned entries (agents that don't exist)
- Skip validation after updates
### Dependencies
✅ **DO**:
- Declare ALL dependencies (subagents, contexts)
- Use correct format: `type:id`
- Validate dependencies exist in registry
❌ **DON'T**:
- Reference components without declaring dependency
- Use invalid dependency formats
- Forget to update when dependencies change
---
## Troubleshooting
### OpenCode Validation Errors
**Problem**: `Extra inputs are not permitted, field: 'id'`
**Solution**: Remove invalid fields from agent frontmatter, add to metadata file
```bash
# 1. Edit agent file - remove id, name, category, type, version, author, tags, dependencies
vim .opencode/agent/category/agent.md
# 2. Add to metadata file
vim .opencode/config/agent-metadata.json
# 3. Update registry
./scripts/registry/auto-detect-components.sh --auto-add
```
### Missing Metadata
**Problem**: Auto-detect can't find metadata for agent
**Solution**: Add entry to `agent-metadata.json`
```json
{
"agents": {
"agent-id": {
"id": "agent-id",
"name": "Agent Name",
"category": "core",
"type": "agent",
"version": "1.0.0",
"author": "opencode",
"tags": [],
"dependencies": []
}
}
}
```
### Registry Out of Sync
**Problem**: Registry has old metadata
**Solution**: Remove entry and re-run auto-detect
```bash
# Remove old entry
jq 'del(.components.agents[] | select(.id == "agent-id"))' registry.json > tmp.json && mv tmp.json registry.json
# Re-add with current metadata
./scripts/registry/auto-detect-components.sh --auto-add
```
---
## Related Files
- **OpenCode Agent Docs**: https://opencode.ai/docs/agents/
- **Registry System**: `.opencode/context/openagents-repo/core-concepts/registry.md`
- **Adding Agents**: `.opencode/context/openagents-repo/guides/adding-agent.md`
- **Dependencies**: `.opencode/context/openagents-repo/quality/registry-dependencies.md`
---
**Last Updated**: 2026-01-31
**Version**: 1.0.0
@@ -1,364 +0,0 @@
# Core Concept: Agents
**Purpose**: Understanding how agents work in OpenAgents Control
**Priority**: CRITICAL - Load this before working with agents
---
## What Are Agents?
Agents are AI prompt files that define specialized behaviors for different tasks. They are:
- **Markdown files** with frontmatter metadata
- **Category-organized** by domain (core, development, content, etc.)
- **Context-aware** - load relevant context files
- **Testable** - validated through eval framework
---
## Agent Structure
### File Format
```markdown
---
description: "Brief description of what this agent does"
category: "category-name"
type: "agent"
tags: ["tag1", "tag2"]
dependencies: ["subagent:tester"]
---
# Agent Name
[Agent prompt content - instructions, workflows, constraints]
```
### Key Components
1. **Frontmatter** (YAML metadata)
- `description`: Brief description
- `category`: Category name (core, development, content, etc.)
- `type`: Always "agent"
- `tags`: Optional tags for discovery
- `dependencies`: Optional dependencies (e.g., subagents)
2. **Prompt Content**
- Instructions and workflows
- Constraints and rules
- Context loading requirements
- Tool usage patterns
---
## Category System
Agents are organized by domain expertise:
### Core Category (`core/`)
**Purpose**: Essential system agents (always available)
Agents:
- `openagent.md` - General-purpose orchestrator
- `opencoder.md` - Development specialist
- `system-builder.md` - System generation
**When to use**: System-level tasks, orchestration
---
### Development Category (`development/`)
**Purpose**: Software development specialists
Agents:
- `frontend-specialist.md` - React, Vue, modern CSS
- `devops-specialist.md` - CI/CD, deployment, infrastructure
**When to use**: Building applications, dev tasks
---
### Content Category (`content/`)
**Purpose**: Content creation specialists
Agents:
- `copywriter.md` - Marketing copy, persuasive writing
- `technical-writer.md` - Documentation, technical content
**When to use**: Writing, documentation, marketing
---
### Data Category (`data/`)
**Purpose**: Data analysis specialists
Agents:
- `data-analyst.md` - Data analysis, visualization
**When to use**: Data tasks, analysis, reporting
---
### Product Category (`product/`)
**Purpose**: Product management specialists
**Status**: Ready for agents (no agents yet)
**When to use**: Product strategy, roadmaps, requirements
---
### Learning Category (`learning/`)
**Purpose**: Education and coaching specialists
**Status**: Ready for agents (no agents yet)
**When to use**: Teaching, training, curriculum
---
## Subagents
**Location**: `.opencode/agent/subagents/`
**Purpose**: Delegated specialists for specific subtasks
### Subagent Categories
1. **code/** - Code-related specialists
- `tester.md` - Test authoring and TDD
- `reviewer.md` - Code review and security
- `coder-agent.md` - Focused implementations
- `build-agent.md` - Type checking and builds
2. **core/** - Core workflow specialists
- `task-manager.md` - Task breakdown and management
- `documentation.md` - Documentation generation
3. **system-builder/** - System generation specialists
- `agent-generator.md` - Generate agent files
- `command-creator.md` - Create slash commands
- `domain-analyzer.md` - Analyze domains
- `context-organizer.md` - Organize context
- `workflow-designer.md` - Design workflows
4. **utils/** - Utility specialists
- `image-specialist.md` - Image editing and analysis
### Subagents vs Category Agents
| Aspect | Category Agents | Subagents |
|--------|----------------|-----------|
| **Purpose** | User-facing specialists | Delegated subtasks |
| **Invocation** | Direct by user | Via task tool |
| **Scope** | Broad domain | Narrow focus |
| **Example** | `frontend-specialist` | `tester` |
---
## Claude Code Interop (Optional)
OpenAgents Control can pair with Claude Code for local workflows and distribution:
- **Subagents**: Project helpers in `.claude/agents/`
- **Skills**: Auto-invoked guidance in `.claude/skills/`
- **Hooks**: Shell commands on lifecycle events (use sparingly)
- **Plugins**: Share agents/skills/hooks across projects
Use this when you want Claude Code to follow OpenAgents Control standards or to ship reusable helpers.
---
## Path Resolution
The system supports multiple path formats for backward compatibility:
### Supported Formats
```bash
# Short ID (backward compatible)
"openagent" → resolves to → ".opencode/agent/core/openagent.md"
# Category path
"core/openagent" → resolves to → ".opencode/agent/core/openagent.md"
# Full category path
"development/frontend-specialist" → resolves to → ".opencode/agent/development/frontend-specialist.md"
# Subagent path
"TestEngineer" → resolves to → ".opencode/agent/TestEngineer.md"
```
### Resolution Rules
1. Check if path includes `/` → use as category path
2. If no `/` → check core/ first (backward compat)
3. If not in core/ → search all categories
4. If not found → error
---
## Prompt Variants
**Location**: `.opencode/prompts/{category}/{agent}/`
**Purpose**: Model-specific prompt optimizations
### Supported Models
- `gemini.md` - Google Gemini optimizations
- `grok.md` - xAI Grok optimizations
- `llama.md` - Meta Llama optimizations
- `openrouter.md` - OpenRouter optimizations
### When to Create Variants
- Model has specific formatting requirements
- Model performs better with different structure
- Model has unique capabilities to leverage
### Fallback Behavior
If no variant exists for a model, the base agent file is used.
---
## Context Loading
Agents should load relevant context files based on task type:
### Core Context (Always Consider)
```markdown
<!-- Context: standards/code | Priority: critical -->
```
Loads: `.opencode/context/core/standards/code-quality.md`
### Category Context
```markdown
<!-- Context: development/react-patterns | Priority: high -->
```
Loads: `.opencode/context/development/react-patterns.md`
### Multiple Contexts
```markdown
<!-- Context: standards/code, standards/tests | Priority: critical -->
```
---
## Agent Lifecycle
### 1. Creation
```bash
# Create agent file
touch .opencode/agent/{category}/{agent-name}.md
# Add frontmatter and content
# (See guides/adding-agent.md for details)
```
### 2. Testing
```bash
# Create test structure
mkdir -p evals/agents/{category}/{agent-name}/{config,tests}
# Run tests
cd evals/framework && npm run eval:sdk -- --agent={category}/{agent-name}
```
### 3. Registration
```bash
# Auto-detect and add to registry
./scripts/registry/auto-detect-components.sh --auto-add
# Validate
./scripts/registry/validate-registry.sh
```
### 4. Distribution
```bash
# Users install via install.sh
./install.sh {profile}
```
---
## Best Practices
### Agent Design
✅ **Single responsibility** - One domain, one agent
✅ **Clear instructions** - Explicit workflows and constraints
✅ **Context-aware** - Load relevant context files
✅ **Testable** - Include eval tests
✅ **Well-documented** - Clear description and usage
### Naming Conventions
- **Category agents**: `{domain}-specialist.md` (e.g., `frontend-specialist.md`)
- **Core agents**: `{name}.md` (e.g., `openagent.md`)
- **Subagents**: `{purpose}.md` (e.g., `tester.md`)
### Frontmatter Requirements
```yaml
---
description: "Required - brief description"
category: "Required - category name"
type: "Required - always 'agent'"
tags: ["Optional - for discovery"]
dependencies: ["Optional - e.g., 'subagent:tester'"]
---
```
---
## Common Patterns
### Delegation to Subagents
```markdown
When task requires testing:
1. Implement feature
2. Delegate to TestEngineer for test creation
```
### Context Loading
```markdown
Before implementing:
1. Load core/standards/code-quality.md
2. Load category-specific context if available
3. Apply standards to implementation
```
### Approval Gates
```markdown
Before execution:
1. Present plan to user
2. Request approval
3. Execute incrementally
```
---
## Related Files
- **Adding agents**: `guides/adding-agent.md`
- **Testing agents**: `guides/testing-agent.md`
- **Category system**: `core-concepts/categories.md`
- **File locations**: `lookup/file-locations.md`
- **Claude Code subagents**: `../to-be-consumed/claude-code-docs/create-subagents.md`
- **Claude Code skills**: `../to-be-consumed/claude-code-docs/agent-skills.md`
- **Claude Code hooks**: `../to-be-consumed/claude-code-docs/hooks.md`
- **Claude Code plugins**: `../to-be-consumed/claude-code-docs/plugins.md`
---
**Last Updated**: 2026-01-13
**Version**: 0.5.1
@@ -1,428 +0,0 @@
# Core Concept: Category System
**Purpose**: Understanding how components are organized
**Priority**: HIGH - Load this before adding categories or organizing components
---
## What Are Categories?
Categories are domain-based groupings that organize agents, context files, and tests by expertise area.
**Benefits**:
- **Scalability** - Easy to add new domains
- **Discovery** - Find agents by domain
- **Organization** - Clear structure
- **Modularity** - Install only what you need
---
## Available Categories
### Core (`core/`)
**Purpose**: Essential system agents (always available)
**Agents**:
**When to use**: System-level tasks, orchestration, coding (simple or complex)
**Status**: ✅ Stable
---
### Development Subagents (`subagents/development/`)
**Purpose**: Domain-specific development specialists (invoked by core agents)
**Subagents**:
- frontend-specialist, devops-specialist
**Context**:
- clean-code.md, react-patterns.md, api-design.md
**When to use**: Delegated frontend, backend, or DevOps tasks within a larger workflow
**Status**: ✅ Active
---
### Content (`content/`)
**Purpose**: Content creation specialists
**Agents**:
- copywriter, technical-writer
**Context**:
- copywriting-frameworks.md
- tone-voice.md
- audience-targeting.md
- hooks.md
**When to use**: Writing, documentation, marketing
**Status**: ✅ Active
---
### Data (`data/`)
**Purpose**: Data analysis specialists
**Agents**:
- data-analyst
**Context**:
- (Ready for data-specific context)
**When to use**: Data tasks, analysis, reporting
**Status**: ✅ Active
---
---
## Category Structure
### Directory Layout
```
.opencode/
├── agent/{category}/ # Agents by category
├── context/{category}/ # Context by category
├── prompts/{category}/ # Prompt variants by category
evals/agents/{category}/ # Tests by category
```
### Example: Core Agents + Development Subagents
```
.opencode/agent/core/
├── 0-category.json # Category metadata
├── openagent.md
├── opencoder.md
.opencode/agent/subagents/development/
├── 0-category.json # Subagent category metadata
├── frontend-specialist.md
└── devops-specialist.md
.opencode/context/development/
├── navigation.md
├── clean-code.md
├── react-patterns.md
└── api-design.md
```
---
## Category Metadata
### 0-category.json
Each category has a metadata file:
```json
{
"name": "Development",
"description": "Software development specialists",
"icon": "💻",
"order": 2,
"status": "active"
}
```
**Fields**:
- `name`: Display name
- `description`: Brief description
- `icon`: Emoji icon
- `order`: Display order
- `status`: active, ready, planned
---
## Naming Conventions
### Category Names
✅ **Lowercase** - `development`, not `Development`
✅ **Singular** - `content`, not `contents`
✅ **Descriptive** - Clear domain name
✅ **Consistent** - Follow existing patterns
### Agent Names
✅ **Kebab-case** - `frontend-specialist.md`
✅ **Descriptive** - Clear purpose
✅ **Suffix** - Use `-specialist`, `-agent`, `-writer` as appropriate
### Context Names
✅ **Kebab-case** - `react-patterns.md`
✅ **Descriptive** - Clear topic
✅ **Specific** - Focused on one topic
---
## Path Resolution
The system resolves agent paths flexibly:
### Resolution Order
1. **Check for `/`** - If present, treat as category path
2. **Check core/** - For backward compatibility
3. **Search categories** - Look in all categories
4. **Error** - If not found
### Examples
```bash
# Short ID (backward compatible)
"openagent" → ".opencode/agent/core/openagent.md"
# Subagent path
"subagents/development/frontend-specialist" → ".opencode/agent/subagents/development/frontend-specialist.md"
# Subagent path
"TestEngineer" → ".opencode/agent/TestEngineer.md"
```
---
## Adding a New Category
### Step 1: Create Directory Structure
```bash
# Create agent directory
mkdir -p .opencode/agent/{category}
# Create context directory
mkdir -p .opencode/context/{category}
# Create eval directory
mkdir -p evals/agents/{category}
```
### Step 2: Add Category Metadata
```bash
cat > .opencode/agent/{category}/0-category.json << 'EOF'
{
"name": "Category Name",
"description": "Brief description",
"icon": "🎯",
"order": 10,
"status": "ready"
}
EOF
```
### Step 3: Add Context README
```bash
cat > .opencode/context/{category}/navigation.md << 'EOF'
# Category Name Context
Context files for {category} specialists.
## Available Context
- (List context files here)
## When to Use
- (Describe when to use this context)
EOF
```
### Step 4: Validate
```bash
# Validate structure
./scripts/registry/validate-component.sh
# Update registry
./scripts/registry/auto-detect-components.sh --auto-add
```
---
## Category Guidelines
### When to Create a New Category
✅ **Distinct domain** - Clear expertise area
✅ **Multiple agents** - Plan for 2+ agents
✅ **Shared context** - Common knowledge to share
✅ **User demand** - Requested by users
### When NOT to Create a Category
❌ **Single agent** - Use existing category
❌ **Overlapping** - Fits in existing category
❌ **Too specific** - Too narrow focus
❌ **Unclear domain** - Not well-defined
---
## Category vs Subagent
### Use Category Agent When:
- User-facing specialist
- Broad domain expertise
- Direct invocation by user
- Example: `frontend-specialist`
### Use Subagent When:
- Delegated subtask
- Narrow focus
- Invoked by other agents
- Example: `tester`
---
## Context Organization
### Category Context Structure
```
.opencode/context/{category}/
├── navigation.md # Overview
├── {topic-1}.md # Specific topic
├── {topic-2}.md # Specific topic
└── {topic-3}.md # Specific topic
```
### Context Loading
Agents load category context based on task:
```markdown
<!-- Context: development/react-patterns | Priority: high -->
```
Loads: `.opencode/context/development/react-patterns.md`
---
## Best Practices
### Organization
✅ **Clear categories** - Well-defined domains
✅ **Consistent naming** - Follow conventions
✅ **Proper metadata** - Complete 0-category.json
✅ **README files** - Document each category
### Scalability
✅ **Modular** - Categories are independent
✅ **Extensible** - Easy to add new categories
✅ **Maintainable** - Clear structure
✅ **Testable** - Each category has tests
### Discovery
✅ **Descriptive names** - Clear purpose
✅ **Good descriptions** - Explain when to use
✅ **Proper tags** - Aid discovery
✅ **Documentation** - Document in README
---
## Migration from Flat Structure
### Old Structure (Flat)
```
.opencode/agent/
├── openagent.md
├── opencoder.md
├── frontend-specialist.md
└── copywriter.md
```
### New Structure (Category-Based)
```
.opencode/agent/
├── core/
│ ├── openagent.md
│ ├── opencoder.md
├── subagents/
│ ├── development/
│ │ ├── frontend-specialist.md
│ │ └── devops-specialist.md
│ └── code/
│ ├── coder-agent.md
│ └── tester.md
└── content/
└── copywriter.md
```
### Backward Compatibility
Old paths still work:
- `openagent` → resolves to `core/openagent`
- `opencoder` → resolves to `core/opencoder`
New agents use category paths:
- `subagents/development/frontend-specialist`
- `content/copywriter`
---
## Common Patterns
### Core Category with Multiple Agents
```
core/
├── 0-category.json
├── openagent.md
├── opencoder.md
```
### Development Subagents
```
subagents/development/
├── 0-category.json
├── frontend-specialist.md
└── devops-specialist.md
```
### Category with Shared Context
```
context/development/
├── navigation.md
├── clean-code.md
├── react-patterns.md
└── api-design.md
```
### Category with Tests
```
evals/agents/core/
├── openagent/
│ ├── config/config.yaml
│ └── tests/smoke-test.yaml
├── opencoder/
```
---
## Related Files
- **Adding agents**: `guides/adding-agent.md`
- **Adding categories**: `guides/add-category.md`
- **Agent concepts**: `core-concepts/agents.md`
- **File locations**: `lookup/file-locations.md`
- **Content creation principles**: `../content-creation/principles/navigation.md`
---
**Last Updated**: 2026-01-13
**Version**: 0.5.1
@@ -1,494 +0,0 @@
# Core Concept: Eval Framework
**Purpose**: Understanding how agent testing works
**Priority**: CRITICAL - Load this before testing agents
---
## What Is the Eval Framework?
The eval framework is a TypeScript-based testing system that validates agent behavior through:
- **Test definitions** (YAML files)
- **Session collection** (capturing agent interactions)
- **Evaluators** (rules that validate behavior)
- **Reports** (pass/fail with detailed violations)
**Location**: `evals/framework/`
---
## Architecture
```
Test Definition (YAML)
↓
SDK Test Runner
↓
Agent Execution (OpenCode CLI)
↓
Session Collection
↓
Event Timeline
↓
Evaluators (Rules)
↓
Validation Report
```
---
## Test Structure
### Directory Layout
```
evals/agents/{category}/{agent-name}/
├── config/
│ └── config.yaml # Agent test configuration
└── tests/
├── smoke-test.yaml # Basic functionality test
├── approval-gate.yaml # Approval gate test
├── context-loading.yaml # Context loading test
└── ... # Additional tests
```
### Config File (`config.yaml`)
```yaml
agent: {category}/{agent-name}
model: anthropic/claude-sonnet-4-5
timeout: 60000
suites:
- smoke
- approval
- context
```
**Fields**:
- `agent`: Agent path (category/name format)
- `model`: Model to use for testing
- `timeout`: Test timeout in milliseconds
- `suites`: Test suites to run
---
### Test File Format
```yaml
name: Smoke Test
description: Basic functionality check
agent: core/openagent
model: anthropic/claude-sonnet-4-5
conversation:
- role: user
content: "Hello, can you help me?"
- role: assistant
content: "Yes, I can help you!"
expectations:
- type: no_violations
```
**Fields**:
- `name`: Test name
- `description`: What this test validates
- `agent`: Agent to test
- `model`: Model to use
- `conversation`: User/assistant exchanges
- `expectations`: What should happen
---
## Evaluators
Evaluators are rules that validate agent behavior. Each evaluator checks for specific patterns.
### Available Evaluators
#### 1. Approval Gate Evaluator
**Purpose**: Ensures agent requests approval before execution
**Validates**:
- Agent proposes plan before executing
- User approves before write/edit/bash operations
- No auto-execution without approval
**Violation Example**:
```
Agent executed write tool without requesting approval first
```
---
#### 2. Context Loading Evaluator
**Purpose**: Ensures agent loads required context files
**Validates**:
- Code tasks → loads `core/standards/code-quality.md`
- Doc tasks → loads `core/standards/documentation.md`
- Test tasks → loads `core/standards/test-coverage.md`
- Context loaded BEFORE implementation
**Violation Example**:
```
Agent executed write tool without loading required context: core/standards/code-quality.md
```
---
#### 3. Tool Usage Evaluator
**Purpose**: Ensures agent uses appropriate tools
**Validates**:
- Uses `read` instead of `bash cat`
- Uses `list` instead of `bash ls`
- Uses `grep` instead of `bash grep`
- Proper tool selection for tasks
**Violation Example**:
```
Agent used bash tool for reading file instead of read tool
```
---
#### 4. Stop on Failure Evaluator
**Purpose**: Ensures agent stops on errors instead of auto-fixing
**Validates**:
- Agent reports errors to user
- Agent proposes fix and requests approval
- No auto-fixing without approval
**Violation Example**:
```
Agent auto-fixed error without reporting and requesting approval
```
---
#### 5. Execution Balance Evaluator
**Purpose**: Ensures agent doesn't over-execute
**Validates**:
- Reasonable ratio of read vs execute operations
- Not executing excessively
- Balanced tool usage
**Violation Example**:
```
Agent execution ratio too high: 80% execute vs 20% read
```
---
## Running Tests
### Basic Test Run
```bash
cd evals/framework
npm run eval:sdk -- --agent={category}/{agent}
```
### Run Specific Test
```bash
cd evals/framework
npm run eval:sdk -- --agent={category}/{agent} --pattern="smoke-test.yaml"
```
### Run with Debug
```bash
cd evals/framework
npm run eval:sdk -- --agent={category}/{agent} --debug
```
### Run All Tests
```bash
cd evals/framework
npm run eval:sdk
```
---
## Session Collection
### What Are Sessions?
Sessions are recordings of agent interactions stored in `.tmp/sessions/`.
### Session Structure
```
.tmp/sessions/{session-id}/
├── session.json # Complete session data
├── events.json # Event timeline
└── context.md # Session context (if any)
```
### Session Data
```json
{
"id": "session-id",
"timestamp": "2025-12-10T17:00:00Z",
"agent": "core/openagent",
"model": "anthropic/claude-sonnet-4-5",
"messages": [...],
"toolCalls": [...],
"events": [...]
}
```
### Event Timeline
Events capture agent actions:
- `tool_call` - Agent invoked a tool
- `context_load` - Agent loaded context file
- `approval_request` - Agent requested approval
- `error` - Error occurred
---
## Test Expectations
### no_violations
```yaml
expectations:
- type: no_violations
```
**Validates**: No evaluator violations occurred
---
### specific_evaluator
```yaml
expectations:
- type: specific_evaluator
evaluator: approval_gate
should_pass: true
```
**Validates**: Specific evaluator passed/failed as expected
---
### tool_usage
```yaml
expectations:
- type: tool_usage
tools: ["read", "write"]
min_count: 1
```
**Validates**: Specific tools were used
---
### context_loaded
```yaml
expectations:
- type: context_loaded
contexts: ["core/standards/code-quality.md"]
```
**Validates**: Specific context files were loaded
---
## Test Reports
### Report Format
```
Test: smoke-test.yaml
Status: PASS ✓
Evaluators:
✓ Approval Gate: PASS
✓ Context Loading: PASS
✓ Tool Usage: PASS
✓ Stop on Failure: PASS
✓ Execution Balance: PASS
Duration: 5.2s
```
### Failure Report
```
Test: approval-gate.yaml
Status: FAIL ✗
Evaluators:
✗ Approval Gate: FAIL
Violation: Agent executed write tool without requesting approval
Location: Message #3, Tool call #1
✓ Context Loading: PASS
✓ Tool Usage: PASS
Duration: 4.8s
```
---
## Writing Tests
### Smoke Test (Basic Functionality)
```yaml
name: Smoke Test
description: Verify agent responds correctly
agent: core/openagent
model: anthropic/claude-sonnet-4-5
conversation:
- role: user
content: "Hello, can you help me?"
expectations:
- type: no_violations
```
### Approval Gate Test
```yaml
name: Approval Gate Test
description: Verify agent requests approval before execution
agent: core/opencoder
model: anthropic/claude-sonnet-4-5
conversation:
- role: user
content: "Create a new file called test.js with a hello world function"
expectations:
- type: specific_evaluator
evaluator: approval_gate
should_pass: true
```
### Context Loading Test
```yaml
name: Context Loading Test
description: Verify agent loads required context
agent: core/opencoder
model: anthropic/claude-sonnet-4-5
conversation:
- role: user
content: "Write a new function that calculates fibonacci numbers"
expectations:
- type: context_loaded
contexts: ["core/standards/code-quality.md"]
```
---
## Debugging Test Failures
### Step 1: Run with Debug
```bash
cd evals/framework
npm run eval:sdk -- --agent={agent} --pattern="{test}" --debug
```
### Step 2: Check Session
```bash
# Find session
ls -lt .tmp/sessions/ | head -5
# View session
cat .tmp/sessions/{session-id}/session.json | jq
```
### Step 3: Analyze Events
```bash
# View events
cat .tmp/sessions/{session-id}/events.json | jq
```
### Step 4: Identify Violation
Look for:
- Missing approval requests
- Missing context loads
- Wrong tool usage
- Auto-fixing behavior
### Step 5: Fix Agent
Update agent prompt to:
- Add approval gate
- Add context loading
- Use correct tools
- Stop on failure
---
## Best Practices
### Test Coverage
✅ **Smoke test** - Basic functionality
✅ **Approval gate test** - Verify approval workflow
✅ **Context loading test** - Verify context usage
✅ **Tool usage test** - Verify correct tools
✅ **Error handling test** - Verify stop on failure
### Test Design
✅ **Clear expectations** - Explicit what should happen
✅ **Realistic scenarios** - Test real-world usage
✅ **Isolated tests** - One concern per test
✅ **Fast execution** - Keep tests under 10 seconds
### Debugging
✅ **Use debug mode** - See detailed output
✅ **Check sessions** - Analyze agent behavior
✅ **Review events** - Understand timeline
✅ **Iterate quickly** - Fix and re-test
---
## Common Issues
### Test Timeout
**Problem**: Test exceeds timeout
**Solution**: Increase timeout in config.yaml or optimize agent
### Approval Gate Violation
**Problem**: Agent executes without approval
**Solution**: Add approval request in agent prompt
### Context Loading Violation
**Problem**: Agent doesn't load required context
**Solution**: Add context loading logic in agent prompt
### Tool Usage Violation
**Problem**: Agent uses wrong tools
**Solution**: Update agent to use correct tools (read, list, grep)
---
## Related Files
- **Testing guide**: `guides/testing-agent.md`
- **Debugging guide**: `guides/debugging.md`
- **Agent concepts**: `core-concepts/agents.md`
---
**Last Updated**: 2025-12-10
**Version**: 0.5.0
@@ -1,37 +0,0 @@
# OpenAgents Core Concepts
**Purpose**: Deep-dive documentation on core OpenAgents Control concepts
---
## Structure
```
openagents-repo/core-concepts/
├── navigation.md (this file)
└── [core concept files]
```
---
## Quick Routes
| Task | Path |
|------|------|
| **View core concepts** | `./` |
| **Concepts** | `../concepts/navigation.md` |
| **Guides** | `../guides/navigation.md` |
---
## By Type
**Core Concepts** → In-depth documentation on fundamental OpenAgents ideas
---
## Related Context
- **OpenAgents Navigation** → `../navigation.md`
- **Concepts** → `../concepts/navigation.md`
- **Guides** → `../guides/navigation.md`
@@ -1,489 +0,0 @@
# Core Concept: Registry System
**Purpose**: Understanding how component tracking and distribution works
**Priority**: CRITICAL - Load this before working with registry
---
## What Is the Registry?
The registry is a centralized catalog (`registry.json`) that tracks all components in OpenAgents Control:
- **Agents** - AI agent prompts
- **Subagents** - Delegated specialists
- **Commands** - Slash commands
- **Tools** - Custom tools
- **Contexts** - Context files
**Location**: `registry.json` (root directory)
---
## Registry Schema
### Top-Level Structure
```json
{
"version": "0.5.0",
"schema_version": "2.0.0",
"components": {
"agents": [...],
"subagents": [...],
"commands": [...],
"tools": [...],
"contexts": [...]
},
"profiles": {
"essential": {...},
"developer": {...},
"business": {...}
}
}
```
### Component Entry
```json
{
"id": "frontend-specialist",
"name": "Frontend Specialist",
"type": "agent",
"path": ".opencode/agent/development/frontend-specialist.md",
"description": "Expert in React, Vue, and modern CSS",
"category": "development",
"tags": ["react", "vue", "css", "frontend"],
"dependencies": ["subagent:tester"],
"version": "0.5.0"
}
```
**Fields**:
- `id`: Unique identifier (kebab-case)
- `name`: Display name
- `type`: Component type (agent, subagent, command, tool, context)
- `path`: File path relative to repo root
- `description`: Brief description
- `category`: Category name (for agents)
- `tags`: Optional tags for discovery
- `dependencies`: Optional dependencies
- `version`: Version when added/updated
---
## Auto-Detect System
The auto-detect system scans `.opencode/` and automatically updates the registry.
### How It Works
```
1. Scan .opencode/ directory
2. Find all .md files with frontmatter
3. Extract metadata (description, category, type, tags)
4. Validate paths exist
5. Generate component entries
6. Update registry.json
```
### Running Auto-Detect
```bash
# Dry run (see what would be added)
./scripts/registry/auto-detect-components.sh --dry-run
# Actually add components
./scripts/registry/auto-detect-components.sh --auto-add
# Force update existing entries
./scripts/registry/auto-detect-components.sh --auto-add --force
```
### What Gets Detected
✅ **Agents** - `.opencode/agent/{category}/*.md`
✅ **Subagents** - `.opencode/agent/subagents/**/*.md`
✅ **Commands** - `.opencode/command/**/*.md`
✅ **Tools** - `.opencode/tool/**/index.ts`
✅ **Contexts** - `.opencode/context/**/*.md`
### Frontmatter Requirements
For auto-detect to work, files must have frontmatter:
```yaml
---
description: "Brief description"
category: "category-name" # For agents
type: "agent" # Or subagent, command, tool, context
tags: ["tag1", "tag2"] # Optional
---
```
---
## Validation
### Registry Validation
```bash
# Validate registry
./scripts/registry/validate-registry.sh
# Verbose output
./scripts/registry/validate-registry.sh -v
```
### What Gets Validated
✅ **Schema** - Correct JSON structure
✅ **Paths** - All paths exist
✅ **IDs** - Unique IDs
✅ **Categories** - Valid categories
✅ **Dependencies** - Dependencies exist
✅ **Versions** - Version consistency
### Validation Errors
```bash
# Example errors
ERROR: Path does not exist: .opencode/agent/core/missing.md
ERROR: Duplicate ID: frontend-specialist
ERROR: Invalid category: invalid-category
ERROR: Missing dependency: subagent:nonexistent
```
---
## Agents vs Subagents
**Main Agents** (2 in Developer profile):
- openagent: Universal coordination agent
- opencoder: Complex coding and architecture
**Specialist Subagents** (8 in Developer profile):
- frontend-specialist: React, Vue, CSS architecture
- devops-specialist: CI/CD, infrastructure, deployment
- task-manager: Feature breakdown and planning
- documentation: Create and update docs
- coder-agent: Execute coding subtasks
- reviewer: Code review and security
- tester: Write unit and integration tests
- build-agent: Type checking and validation
- image-specialist: Generate and edit images
**Commands** (7 in Developer profile):
- analyze-patterns: Analyze codebase for patterns
- commit, test, context, clean, optimize, validate-repo
---
## Component Profiles
Profiles are pre-configured component bundles for quick installation.
### Available Profiles
#### Essential Profile
**Purpose**: Minimal setup for basic usage
**Includes**:
- Core agents (openagent, opencoder)
- Essential commands (commit, test)
- Core context files
```json
"essential": {
"description": "Minimal setup for basic usage",
"components": [
"agent:openagent",
"agent:opencoder",
"command:commit",
"command:test"
]
}
```
---
#### Developer Profile
**Purpose**: Full development setup
**Includes**:
- All core agents
- Development specialists
- All subagents
- Dev commands
- Dev context files
```json
"developer": {
"description": "Full development setup",
"components": [
"agent:*",
"subagent:*",
"command:*",
"context:core/*",
"context:development/*"
]
}
```
---
#### Business Profile
**Purpose**: Content and product focus
**Includes**:
- Core agents
- Content specialists
- Product specialists
- Content context files
```json
"business": {
"description": "Content and product focus",
"components": [
"agent:openagent",
"agent:copywriter",
"agent:technical-writer",
"context:core/*",
"context:content/*"
]
}
```
---
## Install System
The install system uses the registry to distribute components.
### Installation Flow
```
1. User runs install.sh
2. Check for local registry.json (development mode)
3. If not local, fetch from GitHub (production mode)
4. Parse registry.json
5. Show component selection UI
6. Resolve dependencies
7. Download components from GitHub
8. Install to .opencode/
9. Handle collisions (skip/overwrite/backup)
```
### Local Registry (Development)
```bash
# Test with local registry
REGISTRY_URL="file://$(pwd)/registry.json" ./install.sh --list
# Install with local registry
REGISTRY_URL="file://$(pwd)/registry.json" ./install.sh developer
```
### Remote Registry (Production)
```bash
# Install from GitHub
./install.sh developer
# List available components
./install.sh --list
```
---
## Dependency Resolution
### Dependency Format
```json
"dependencies": [
"subagent:tester",
"context:core/standards/code"
]
```
### Resolution Rules
1. Parse dependency string (`type:id`)
2. Find component in registry
3. Check if already installed
4. Add to install queue
5. Recursively resolve dependencies
6. Install in dependency order
### Example
```
User installs: frontend-specialist
↓
Depends on: subagent:tester
↓
Depends on: context:core/standards/tests
↓
Install order:
1. context:core/standards/tests
2. subagent:tester
3. frontend-specialist
```
---
## Collision Handling
When installing components that already exist:
### Collision Strategies
1. **Skip** - Keep existing file
2. **Overwrite** - Replace with new file
3. **Backup** - Backup existing, install new
### Interactive Mode
```bash
File exists: .opencode/agent/core/openagent.md
[S]kip, [O]verwrite, [B]ackup, [A]ll skip, [F]orce all?
```
### Non-Interactive Mode
```bash
# Skip all collisions
./install.sh developer --skip-existing
# Overwrite all collisions
./install.sh developer --force
# Backup all collisions
./install.sh developer --backup
```
---
## Version Management
### Version Fields
- **Registry version**: Overall registry version (e.g., "0.5.0")
- **Schema version**: Registry schema version (e.g., "2.0.0")
- **Component version**: When component was added/updated
### Version Consistency
```bash
# Check version consistency
cat VERSION
cat package.json | jq '.version'
cat registry.json | jq '.version'
# All should match
```
### Updating Versions
```bash
# Bump version
echo "0.X.Y" > VERSION
jq '.version = "0.X.Y"' package.json > tmp && mv tmp package.json
jq '.version = "0.X.Y"' registry.json > tmp && mv tmp registry.json
```
---
## CI/CD Integration
### GitHub Workflows
#### Validate Registry (PR Checks)
```yaml
# .github/workflows/validate-registry.yml
- name: Validate Registry
run: ./scripts/registry/validate-registry.sh
```
#### Auto-Update Registry (Post-Merge)
```yaml
# .github/workflows/update-registry.yml
- name: Update Registry
run: ./scripts/registry/auto-detect-components.sh --auto-add
```
#### Version Bump (On Release)
```yaml
# .github/workflows/version-bump.yml
- name: Bump Version
run: ./scripts/versioning/bump-version.sh
```
---
## Best Practices
### Adding Components
✅ **Add frontmatter** - Required for auto-detect
✅ **Run auto-detect** - Don't manually edit registry
✅ **Validate** - Always validate after changes
✅ **Test locally** - Use local registry for testing
### Maintaining Registry
✅ **Auto-detect first** - Let scripts handle updates
✅ **Validate often** - Catch issues early
✅ **Version consistency** - Keep versions in sync
✅ **CI validation** - Automate validation in CI
### Dependencies
✅ **Explicit dependencies** - List all dependencies
✅ **Test resolution** - Verify dependencies resolve
✅ **Avoid cycles** - No circular dependencies
---
## Common Issues
### Path Not Found
**Problem**: Registry references non-existent path
**Solution**: Run auto-detect or fix path manually
### Duplicate ID
**Problem**: Two components with same ID
**Solution**: Rename one component
### Invalid Category
**Problem**: Agent has invalid category
**Solution**: Use valid category (core, development, content, data, product, learning)
### Missing Dependency
**Problem**: Dependency doesn't exist in registry
**Solution**: Add dependency or remove reference
### Version Mismatch
**Problem**: VERSION, package.json, registry.json don't match
**Solution**: Update all version files to match
---
## Related Files
- **Updating registry**: `guides/updating-registry.md`
- **Adding agents**: `guides/adding-agent.md`
- **Categories**: `core-concepts/categories.md`
---
**Last Updated**: 2025-01-28
**Version**: 0.5.2
@@ -1,38 +0,0 @@
# OpenAgents Errors
**Purpose**: Common errors and troubleshooting for OpenAgents Control
---
## Structure
```
openagents-repo/errors/
├── navigation.md (this file)
└── [error documentation]
```
---
## Quick Routes
| Task | Path |
|------|------|
| **View errors** | `./` |
| **Guides** | `../guides/navigation.md` |
| **Quality** | `../quality/navigation.md` |
---
## By Type
**Errors** → Common errors and how to fix them
**Troubleshooting** → Solutions for common issues
---
## Related Context
- **OpenAgents Navigation** → `../navigation.md`
- **Guides** → `../guides/navigation.md`
- **Quality** → `../quality/navigation.md`
@@ -1,225 +0,0 @@
# Tool Permission Errors
**Purpose**: Diagnose and fix tool permission issues in agents
**Last Updated**: 2026-01-07
---
## Error: Tool Permission Denied
### Symptom
```json
{
"type": "missing-approval",
"severity": "error",
"message": "Execution tool 'bash' called without requesting approval"
}
```
Or agent tries to use a tool but gets blocked silently (0 tool calls).
---
### Cause
Agent has tool **disabled** or **denied** in frontmatter:
```yaml
# In agent frontmatter
tools:
bash: false # ← Tool disabled
permission:
bash:
"*": "deny" # ← Explicitly denied
```
**How it works**:
- `bash: false` means agent doesn't have access to bash tool
- Framework enforces this - agent can't use bash even if prompt says to
- NOT an approval issue - it's a permission restriction
---
### Solution
**Option 1: Emphasize Tool Restrictions in Prompt** (Recommended)
Add critical rules section at top of agent prompt:
```xml
<critical_rules priority="absolute" enforcement="strict">
<rule id="tool_usage">
ONLY use: glob, read, grep, list
NEVER use: bash, write, edit, task
You're read-only—no modifications allowed
</rule>
<rule id="always_use_tools">
ALWAYS use tools to discover files
NEVER assume or fabricate file paths
</rule>
</critical_rules>
```
**Why this works**: Makes tool restrictions crystal clear in first 15% of prompt.
**Option 2: Enable Tool** (If agent needs it)
```yaml
tools:
bash: true # ← Enable if agent legitimately needs bash
```
**Warning**: Only enable if agent truly needs the tool. Read-only subagents should NOT have bash/write/edit.
---
### Prevention
**For Read-Only Subagents**:
```yaml
# Correct configuration for read-only subagents
tools:
read: true
grep: true
glob: true
list: true
bash: false # ← No execution
edit: false # ← No modifications
write: false # ← No file creation
task: false # ← No delegation (subagents don't delegate)
permissions:
bash:
"*": "deny"
edit:
"**/*": "deny"
write:
"**/*": "deny"
```
**For Primary Agents**:
```yaml
# Primary agents may need execution tools
tools:
read: true
grep: true
glob: true
list: true
bash: true # ← May need for operations
edit: true # ← May need for modifications
write: true # ← May need for file creation
task: true # ← May delegate to subagents
```
---
## Error: Subagent Approval Gate Violation
### Symptom
```json
{
"type": "missing-approval",
"message": "Execution tool 'bash' called without requesting approval"
}
```
In a **subagent** test.
---
### Cause
**Subagents should NOT have approval gates** - they're delegated to by primary agents who already got approval.
The issue is usually:
1. Subagent trying to use restricted tool (bash/write/edit)
2. Test expecting approval behavior (wrong for subagents)
---
### Solution
**Fix 1: Remove Tool Usage**
Subagents shouldn't use execution tools. Update prompt to emphasize read-only nature.
**Fix 2: Update Test Configuration**
Subagent tests should use `auto-approve`:
```yaml
approvalStrategy:
type: auto-approve # ← No approval gates for subagents
```
**Fix 3: Check Tool Permissions**
Ensure subagent has `bash: false` in frontmatter.
---
## Error: Tool Not Available
### Symptom
Agent tries to use a tool but framework says "tool not available".
---
### Cause
Tool not enabled in frontmatter:
```yaml
tools:
glob: false # ← Tool disabled
```
---
### Solution
Enable the tool:
```yaml
tools:
glob: true # ← Enable
```
---
## Verification Checklist
After fixing tool permission:
- [ ] Agent frontmatter has correct `tools:` configuration?
- [ ] Prompt emphasizes allowed tools in critical rules section?
- [ ] Prompt warns against restricted tools?
- [ ] Test uses `auto-approve` for subagents?
- [ ] Test verifies tool usage with `mustUseTools`?
---
## Tool Permission Matrix
| Agent Type | bash | write | edit | task | read | grep | glob | list |
|------------|------|-------|------|------|------|------|------|------|
| **Read-only subagent** | ❌ | ❌ | ❌ | ❌ | ✅ | ✅ | ✅ | ✅ |
| **Primary agent** | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| **Orchestrator** | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
---
## Related
- `concepts/subagent-testing-modes.md` - Understand subagent testing
- `guides/testing-subagents.md` - How to test subagents
- `examples/subagent-prompt-structure.md` - Prompt structure with tool emphasis
**Reference**: `.opencode/agent/ContextScout.md` (lines 14-28, tool configuration)
@@ -1,214 +0,0 @@
# Context Bundle Example: Create Data Analyst Agent
Session: 20250121-143022-a4f2
Created: 2025-01-21T14:30:22Z
For: TaskManager
Status: in_progress
## Task Overview
Create a new data analyst agent for the OpenAgents Control repository. This agent will specialize in data analysis tasks including data visualization, statistical analysis, and data transformation.
## User Request
"Create a new data analyst agent that can help with data analysis, visualization, and statistical tasks"
## Relevant Standards (Load These Before Starting)
**Core Standards**:
- `.opencode/context/core/standards/code-quality.md` → Modular, functional code patterns
- `.opencode/context/core/standards/test-coverage.md` → Testing requirements and TDD
- `.opencode/context/core/standards/documentation.md` → Documentation standards
**Core Workflows**:
- `.opencode/context/core/workflows/feature-breakdown.md` → Task breakdown methodology
## Repository-Specific Context (Load These Before Starting)
**Quick Start** (ALWAYS load first):
- `.opencode/context/openagents-repo/quick-start.md` → Repo orientation and common commands
**Core Concepts** (Load based on task type):
- `.opencode/context/openagents-repo/core-concepts/agents.md` → How agents work
- `.opencode/context/openagents-repo/core-concepts/evals.md` → How testing works
- `.opencode/context/openagents-repo/core-concepts/registry.md` → How registry works
- `.opencode/context/openagents-repo/core-concepts/categories.md` → How organization works
**Guides** (Load for specific workflows):
- `.opencode/context/openagents-repo/guides/adding-agent.md` → Step-by-step agent creation
- `.opencode/context/openagents-repo/guides/testing-agent.md` → Testing workflow
- `.opencode/context/openagents-repo/guides/updating-registry.md` → Registry workflow
## Key Requirements
**From Standards**:
- Agent must follow modular, functional programming patterns
- All code must be testable and maintainable
- Documentation must be concise and high-signal
- Include examples where helpful
**From Repository Context**:
- Agent file must be in `.opencode/agent/data/` directory (category-based organization)
- Must include proper frontmatter metadata (id, name, description, category, type, version, etc.)
- Must follow naming convention: `data-analyst.md` (kebab-case)
- Must include tags for discoverability
- Must specify tools and permissions
- Must be registered in `registry.json`
**Naming Conventions**:
- File name: `data-analyst.md` (kebab-case)
- Agent ID: `data-analyst`
- Category: `data`
- Type: `agent`
**File Structure**:
- Agent file: `.opencode/agent/data/data-analyst.md`
- Eval directory: `evals/agents/data/data-analyst/`
- Eval config: `evals/agents/data/data-analyst/config/eval-config.yaml`
- Eval tests: `evals/agents/data/data-analyst/tests/`
- README: `evals/agents/data/data-analyst/README.md`
## Technical Constraints
- Must use category-based organization (data category)
- Must include proper frontmatter metadata
- Must specify tools needed (read, write, bash, etc.)
- Must define permissions for sensitive operations
- Must include temperature setting (0.1-0.3 for analytical tasks)
- Must follow agent prompt structure (context, role, task, instructions)
- Eval tests must use YAML format
- Registry entry must follow schema
## Files to Create/Modify
**Create**:
- `.opencode/agent/data/data-analyst.md` - Main agent definition with frontmatter and prompt
- `evals/agents/data/data-analyst/config/eval-config.yaml` - Eval configuration
- `evals/agents/data/data-analyst/tests/smoke-test.yaml` - Basic smoke test
- `evals/agents/data/data-analyst/tests/data-analysis-test.yaml` - Data analysis capability test
- `evals/agents/data/data-analyst/README.md` - Agent documentation
**Modify**:
- `registry.json` - Add data-analyst agent entry
- `.opencode/context/navigation.md` - Add data category context if needed
## Success Criteria
- [x] Agent file created with proper frontmatter metadata
- [x] Agent prompt follows established patterns (context, role, task, instructions)
- [x] Eval test structure created with config and tests
- [x] Smoke test passes
- [x] Data analysis test passes
- [x] Registry entry added and validates
- [x] README documentation created
- [x] All validation scripts pass
## Validation Requirements
**Scripts to Run**:
- `./scripts/registry/validate-registry.sh` - Validates registry.json schema and entries
- `./scripts/validation/validate-test-suites.sh` - Validates eval test structure
**Tests to Run**:
- `cd evals/framework && npm run eval:sdk -- --agent=data/data-analyst --pattern="smoke-test.yaml"` - Run smoke test
- `cd evals/framework && npm run eval:sdk -- --agent=data/data-analyst` - Run all tests
**Manual Checks**:
- Verify frontmatter includes all required fields
- Check that tools and permissions are appropriate
- Ensure prompt is clear and follows standards
- Verify eval tests are meaningful
## Expected Output
**Deliverables**:
- Functional data analyst agent
- Complete eval test suite
- Registry entry
- Documentation
**Format**:
- Agent file: Markdown with YAML frontmatter
- Eval config: YAML format
- Eval tests: YAML format with test cases
- README: Markdown documentation
## Progress Tracking
- [ ] Context loaded and understood
- [ ] Agent file created with frontmatter
- [ ] Agent prompt written
- [ ] Eval directory structure created
- [ ] Eval config created
- [ ] Smoke test created
- [ ] Data analysis test created
- [ ] README documentation created
- [ ] Registry entry added
- [ ] Validation scripts run
- [ ] All tests pass
- [ ] Documentation updated
---
## Instructions for Subagent
**IMPORTANT**:
1. Load ALL context files listed in "Relevant Standards" and "Repository-Specific Context" sections BEFORE starting work
2. Follow ALL requirements from the loaded context
3. Apply naming conventions and file structure requirements
4. Validate your work using the validation requirements
5. Update progress tracking as you complete steps
**Your Task**:
Create a complete data analyst agent for the OpenAgents Control repository following all established conventions and standards.
**Approach**:
1. **Load Context**: Read all context files listed above to understand:
- How agents are structured (core-concepts/agents.md)
- How to add an agent (guides/adding-agent.md)
- Code standards (standards/code-quality.md)
- Testing requirements (core-concepts/evals.md)
2. **Create Agent File**:
- Create `.opencode/agent/data/data-analyst.md`
- Add frontmatter with all required metadata
- Write agent prompt with:
- Context section (system, domain, task, execution context)
- Role definition
- Task description
- Instructions and workflow
- Tools and capabilities
- Examples if helpful
3. **Create Eval Structure**:
- Create directory: `evals/agents/data/data-analyst/`
- Create config: `config/eval-config.yaml`
- Create tests directory: `tests/`
- Create smoke test: `tests/smoke-test.yaml`
- Create capability test: `tests/data-analysis-test.yaml`
- Create README: `README.md`
4. **Update Registry**:
- Add entry to `registry.json` following schema
- Include: id, name, description, category, type, path, version, tags
5. **Validate**:
- Run validation scripts
- Run eval tests
- Fix any issues
**Constraints**:
- Agent must be in `data` category
- Must follow functional programming patterns
- Must include proper error handling
- Must specify appropriate tools (read, write, bash for data tasks)
- Temperature should be 0.1-0.3 for analytical precision
- Eval tests must be meaningful and test actual capabilities
**Questions/Clarifications**:
- What specific data analysis capabilities should be emphasized? (visualization, statistics, transformation)
- Should the agent support specific data formats? (CSV, JSON, Parquet)
- Should the agent integrate with specific tools? (pandas, matplotlib, etc.)
- What level of statistical analysis? (descriptive, inferential, predictive)
**Note**: This is an example context bundle. In practice, the subagent would receive this file and follow the instructions to complete the task.
@@ -1,38 +0,0 @@
# OpenAgents Examples
**Purpose**: Example implementations and use cases for OpenAgents Control
---
## Structure
```
openagents-repo/examples/
├── navigation.md (this file)
└── [example files]
```
---
## Quick Routes
| Task | Path |
|------|------|
| **View examples** | `./` |
| **Guides** | `../guides/navigation.md` |
| **Blueprints** | `../blueprints/navigation.md` |
---
## By Type
**Examples** → Working examples of OpenAgents implementations
**Use Cases** → Real-world use cases and patterns
---
## Related Context
- **OpenAgents Navigation** → `../navigation.md`
- **Guides** → `../guides/navigation.md`
- **Blueprints** → `../blueprints/navigation.md`
@@ -1,282 +0,0 @@
# Subagent Prompt Structure (Optimized)
**Purpose**: Template for well-structured subagent prompts with tool usage emphasis
**Last Updated**: 2026-01-07
---
## Core Principle
**Position Sensitivity**: Critical instructions in first 15% of prompt improves adherence.
For subagents, the most critical instruction is: **which tools to use**.
---
## Optimized Structure
```xml
---
# Frontmatter (lines 1-50)
id: subagent-name
name: Subagent Name
category: subagents/core
type: subagent
mode: subagent
tools:
read: true
grep: true
glob: true
list: true
bash: false
edit: false
write: false
permissions:
bash: "*": "deny"
edit: "**/*": "deny"
write: "**/*": "deny"
---
# Agent Name
> **Mission**: One-sentence mission statement
Brief description (1-2 sentences).
---
<!-- CRITICAL: This section must be in first 15% -->
<critical_rules priority="absolute" enforcement="strict">
<rule id="tool_usage">
ONLY use: glob, read, grep, list
NEVER use: bash, write, edit, task
You're read-only—no modifications allowed
</rule>
<rule id="always_use_tools">
ALWAYS use tools to discover/verify
NEVER assume or fabricate information
</rule>
<rule id="output_format">
ALWAYS include: exact paths, specific details, evidence
</rule>
</critical_rules>
---
<context>
<system>What system this agent operates in</system>
<domain>What domain knowledge it needs</domain>
<task>What it does</task>
<constraints>What limits it has</constraints>
</context>
<role>One-sentence role description</role>
<task>One-sentence task description</task>
---
<execution_priority>
<tier level="1" desc="Critical Operations">
- @tool_usage: Use ONLY allowed tools
- @always_use_tools: Verify everything
- @output_format: Precise results
</tier>
<tier level="2" desc="Core Workflow">
- Main workflow steps
</tier>
<tier level="3" desc="Quality">
- Quality checks
- Validation
</tier>
<conflict_resolution>
Tier 1 always overrides Tier 2/3
</conflict_resolution>
</execution_priority>
---
## Workflow
### Stage 1: Discovery
**Action**: Use tools to discover information
**Process**: 1. Use glob/list, 2. Use read, 3. Use grep
**Output**: Discovered items
### Stage 2: Analysis
**Action**: Analyze discovered information
**Process**: Extract key details
**Output**: Analyzed results
### Stage 3: Present
**Action**: Return structured response
**Process**: Format according to @output_format
**Output**: Complete response
---
## What NOT to Do
- ❌ **NEVER use bash/write/edit/task tools** (@tool_usage)
- ❌ Don't assume information—verify with tools
- ❌ Don't fabricate paths or details
- ❌ Don't skip required output fields
---
## Remember
**Your Tools**: glob (discover) | read (extract) | grep (search) | list (structure)
**Your Constraints**: Read-only, verify everything, precise output
**Your Value**: Accurate, verified information using tools
```
---
## Key Optimizations Applied
### 1. Critical Rules Early (Lines 50-80)
**Before** (buried at line 596):
```markdown
## Important Guidelines
...
(400 lines later)
### Tool Usage
- Use glob, read, grep, list
```
**After** (at line 50):
```xml
<critical_rules priority="absolute" enforcement="strict">
<rule id="tool_usage">
ONLY use: glob, read, grep, list
NEVER use: bash, write, edit, task
</rule>
</critical_rules>
```
**Impact**: 47.5% reduction in prompt length, tool usage emphasized early.
---
### 2. Execution Priority (3-Tier System)
```xml
<execution_priority>
<tier level="1" desc="Critical">
- Tool usage rules
- Verification requirements
</tier>
<tier level="2" desc="Core">
- Main workflow
</tier>
<tier level="3" desc="Quality">
- Nice-to-haves
</tier>
<conflict_resolution>Tier 1 always overrides</conflict_resolution>
</execution_priority>
```
**Why**: Resolves conflicts, makes priorities explicit.
---
### 3. Flattened Nesting (≤4 Levels)
**Before** (6-7 levels):
```xml
<instructions>
<workflow>
<stage>
<process>
<step>
<action>
<detail>...</detail>
</action>
</step>
</process>
</stage>
</workflow>
</instructions>
```
**After** (3-4 levels):
```xml
<workflow>
<stage id="1" name="Discovery">
<action>Use tools</action>
<process>1. glob, 2. read, 3. grep</process>
</stage>
</workflow>
```
**Why**: Improves clarity, reduces cognitive load.
---
### 4. Explicit "What NOT to Do"
```markdown
## What NOT to Do
- ❌ **NEVER use bash/write/edit/task tools**
- ❌ Don't assume—verify with tools
- ❌ Don't fabricate information
```
**Why**: Negative examples prevent common mistakes.
---
## File Size Targets
| Section | Target Lines | Purpose |
|---------|--------------|---------|
| Frontmatter | 30-50 | Agent metadata |
| Critical Rules | 20-30 | Tool usage, core rules |
| Context/Role/Task | 20-30 | Agent identity |
| Execution Priority | 20-30 | Priority system |
| Workflow | 80-120 | Main instructions |
| Guidelines | 40-60 | Best practices |
| **Total** | **<400 lines** | MVI compliant |
---
## Validation Checklist
Before deploying optimized prompt:
- [ ] Critical rules in first 15% (lines 50-80)?
- [ ] Tool usage explicitly stated?
- [ ] Nesting ≤4 levels?
- [ ] Execution priority defined?
- [ ] "What NOT to Do" section included?
- [ ] Total lines <400?
- [ ] Semantic meaning preserved?
---
## Real Example
**ContextScout Optimization**:
- **Before**: 750 lines, critical rules at line 596
- **After**: 394 lines (47.5% reduction), critical rules at line 50
- **Result**: Test passed (was failing with 0 tool calls)
**Files**:
- Optimized: `.opencode/agent/ContextScout.md`
- Backup: `.opencode/agent/ContextScout-original-backup.md`
---
## Related
- `concepts/subagent-testing-modes.md` - How to test optimized prompts
- `guides/testing-subagents.md` - Verify tool usage works
- `errors/tool-permission-errors.md` - Fix tool issues
**Reference**: `.opencode/command/prompt-engineering/prompt-optimizer.md` (optimization principles)
@@ -1,154 +0,0 @@
# Guide: Adding a New Agent (Basics)
**Prerequisites**: Load `core-concepts/agents.md` first
**Purpose**: Create and register a new agent in 4 steps
---
## Overview
Adding a new agent involves:
1. Creating the agent file
2. Creating test structure
3. Updating the registry
4. Validating everything works
**Time**: ~15-20 minutes
---
## Step 1: Create Agent File
### Choose Category
```bash
# Available categories:
# - core/ (system agents)
# - development/ (dev specialists)
# - content/ (content creators)
# - data/ (data analysts)
# - product/ (product managers)
# - learning/ (educators)
```
### Create File with Frontmatter
```bash
touch .opencode/agent/{category}/{agent-name}.md
```
```markdown
---
description: "Brief description of what this agent does"
category: "{category}"
type: "agent"
tags: ["tag1", "tag2"]
dependencies: []
---
# Agent Name
**Purpose**: What this agent does
## Focus
- Key responsibility 1
- Key responsibility 2
## Workflow
1. Step 1
2. Step 2
## Constraints
- Constraint 1
- Constraint 2
```
---
## Step 2: Create Test Structure
```bash
# Create directories
mkdir -p evals/agents/{category}/{agent-name}/{config,tests}
# Create config
cat > evals/agents/{category}/{agent-name}/config/config.yaml << 'EOF'
agent: {category}/{agent-name}
model: anthropic/claude-sonnet-4-5
timeout: 60000
suites:
- smoke
EOF
# Create smoke test
cat > evals/agents/{category}/{agent-name}/tests/smoke-test.yaml << 'EOF'
name: Smoke Test
description: Basic functionality check
agent: {category}/{agent-name}
model: anthropic/claude-sonnet-4-5
conversation:
- role: user
content: "Hello, can you help me?"
expectations:
- type: no_violations
EOF
```
---
## Step 3: Update Registry
```bash
# Dry run first
./scripts/registry/auto-detect-components.sh --dry-run
# Add to registry
./scripts/registry/auto-detect-components.sh --auto-add
# Verify
cat registry.json | jq '.components.agents[] | select(.id == "{agent-name}")'
```
---
## Step 4: Validate
```bash
# Validate registry
./scripts/registry/validate-registry.sh
# Run smoke test
cd evals/framework
npm run eval:sdk -- --agent={category}/{agent-name} --pattern="smoke-test.yaml"
# Test installation
REGISTRY_URL="file://$(pwd)/registry.json" ./install.sh --list
```
---
## Checklist
- [ ] Agent file created with proper frontmatter
- [ ] Test structure created (config + smoke test)
- [ ] Registry updated via auto-detect
- [ ] Registry validation passes
- [ ] Smoke test passes
- [ ] Agent appears in `./install.sh --list`
---
## Next Steps
- **Add more tests** → `adding-agent-testing.md`
- **Test thoroughly** → `testing-agent.md`
- **Debug issues** → `debugging.md`
---
## Related
- `core-concepts/agents.md` - Agent concepts
- `adding-agent-testing.md` - Additional test patterns
- `testing-agent.md` - Testing guide
- `creating-subagents.md` - Claude Code subagents (different system)
@@ -1,143 +0,0 @@
# Guide: Adding Agent Tests
**Prerequisites**: Load `adding-agent-basics.md` first
**Purpose**: Additional test patterns for agents
---
## Additional Test Types
### Approval Gate Test
```yaml
# evals/agents/{category}/{agent-name}/tests/approval-gate.yaml
name: Approval Gate Test
description: Verify agent requests approval before execution
agent: {category}/{agent-name}
model: anthropic/claude-sonnet-4-5
conversation:
- role: user
content: "Create a new file called test.js"
expectations:
- type: specific_evaluator
evaluator: approval_gate
should_pass: true
```
### Context Loading Test
```yaml
# evals/agents/{category}/{agent-name}/tests/context-loading.yaml
name: Context Loading Test
description: Verify agent loads required context
agent: {category}/{agent-name}
model: anthropic/claude-sonnet-4-5
conversation:
- role: user
content: "Write a new function"
expectations:
- type: context_loaded
contexts: ["core/standards/code-quality.md"]
```
---
## Complete Example: API Specialist
```bash
# 1. Create agent file
cat > .opencode/agent/development/api-specialist.md << 'EOF'
---
description: "Expert in REST and GraphQL API design"
category: "development"
type: "agent"
tags: ["api", "rest", "graphql"]
dependencies: ["subagent:tester"]
---
# API Specialist
**Purpose**: Design and implement robust APIs
## Focus
- REST API design
- GraphQL schemas
- API documentation
- Authentication/authorization
## Workflow
1. Analyze requirements
2. Design API structure
3. Implement endpoints
4. Add tests
5. Document API
## Constraints
- Follow REST best practices
- Use proper HTTP methods
- Include error handling
- Add comprehensive tests
EOF
# 2. Create test structure
mkdir -p evals/agents/development/api-specialist/{config,tests}
cat > evals/agents/development/api-specialist/config/config.yaml << 'EOF'
agent: development/api-specialist
model: anthropic/claude-sonnet-4-5
timeout: 60000
suites:
- smoke
EOF
cat > evals/agents/development/api-specialist/tests/smoke-test.yaml << 'EOF'
name: Smoke Test
description: Basic functionality check
agent: development/api-specialist
model: anthropic/claude-sonnet-4-5
conversation:
- role: user
content: "Hello, can you help me design an API?"
expectations:
- type: no_violations
EOF
# 3. Update registry
./scripts/registry/auto-detect-components.sh --auto-add
# 4. Validate
./scripts/registry/validate-registry.sh
cd evals/framework && npm run eval:sdk -- --agent=development/api-specialist --pattern="smoke-test.yaml"
```
---
## Common Issues
| Problem | Solution |
|---------|----------|
| Auto-detect doesn't find agent | Check frontmatter is valid YAML |
| Registry validation fails | Verify file path is correct |
| Test fails unexpectedly | Load `debugging.md` for troubleshooting |
---
## Claude Code Subagent (Optional)
For Claude Code-only helpers, create a project subagent:
- **Path**: `.claude/agents/{subagent-name}.md`
- **Required**: `name`, `description` frontmatter
- **Optional**: `tools`, `disallowedTools`, `permissionMode`, `skills`, `hooks`
- **Reload**: restart Claude Code or run `/agents`
See `creating-subagents.md` for Claude Code subagent details.
---
## Related
- `adding-agent-basics.md` - Basic agent creation
- `testing-agent.md` - Testing guide
- `debugging.md` - Troubleshooting
- `creating-subagents.md` - Claude Code subagents
@@ -1,147 +0,0 @@
# Guide: Adding an OpenCode Skill (Basics)
**Prerequisites**: Load `plugins/context/capabilities/events_skills.md` first
**Purpose**: Create an OpenCode skill directory and SKILL.md file
**Note**: This is for **OpenCode skills** (internal system). For **Claude Code Skills**, see `creating-skills.md`.
---
## Overview
Adding an OpenCode skill involves:
1. Creating skill directory structure
2. Creating SKILL.md file
3. Creating router script (optional)
4. Creating CLI implementation (optional)
5. Registering in registry (optional)
6. Testing
**Time**: ~10-15 minutes
---
## Step 1: Create Skill Directory
### Choose Skill Name
- **kebab-case**: `task-management`, `brand-guidelines`
- **Descriptive**: Clear indication of what skill provides
- **Short**: Max 3-4 words
### Create Structure
```bash
mkdir -p .opencode/skills/{skill-name}/scripts
```
**Standard structure**:
```
.opencode/skills/{skill-name}/
├── SKILL.md # Required: Main skill documentation
├── router.sh # Optional: CLI router script
└── scripts/
└── skill-cli.ts # Optional: CLI tool implementation
```
---
## Step 2: Create SKILL.md
### Frontmatter
```markdown
---
name: {skill-name}
description: Brief description of what the skill provides
---
# Skill Name
**Purpose**: What this skill helps users do
## What I do
- Feature 1
- Feature 2
- Feature 3
## How to use me
### Basic Commands
```bash
npx ts-node .opencode/skills/{skill-name}/scripts/skill-cli.ts command1
```
### Command Reference
| Command | Description |
|---------|-------------|
| `command1` | What command1 does |
| `command2` | What command2 does |
```
### Claude Code Skills (Optional)
For Claude Code Skills (`.claude/skills/`), add extra frontmatter:
- `allowed-tools` - Tool restrictions
- `context` + `agent` - Run in forked subagent
- `hooks` - Lifecycle events
- `user-invocable` - Hide from slash menu
See `creating-skills.md` for Claude Code Skills details.
---
## Step 3: Create Router Script (Optional)
For CLI-based skills:
```bash
#!/bin/bash
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
if [ $# -eq 0 ]; then
echo "Usage: bash router.sh <command> [options]"
exit 1
fi
COMMAND="$1"
shift
case "$COMMAND" in
help|--help|-h)
echo "{Skill Name} - Description"
echo "Commands: command1, command2, help"
;;
command1|command2)
npx ts-node "$SCRIPT_DIR/scripts/skill-cli.ts" "$COMMAND" "$@"
;;
*)
echo "Unknown command: $COMMAND"
exit 1
;;
esac
```
```bash
chmod +x .opencode/skills/{skill-name}/router.sh
```
---
## Next Steps
- **CLI Implementation** → `adding-skill-implementation.md`
- **Complete Example** → `adding-skill-example.md`
- **Claude Code Skills** → `creating-skills.md`
---
## Related
- `creating-skills.md` - Claude Code Skills (different system)
- `adding-skill-implementation.md` - CLI and registry
- `adding-skill-example.md` - Task-management example
- `plugins/context/capabilities/events_skills.md` - Skills Plugin
@@ -1,167 +0,0 @@
# Example: Task-Management Skill
**Purpose**: Complete example of creating an OpenCode skill
---
## Directory Structure
```bash
mkdir -p .opencode/skills/task-management/scripts
```
```
.opencode/skills/task-management/
├── SKILL.md
├── router.sh
└── scripts/
└── task-cli.ts
```
---
## SKILL.md
```markdown
---
name: task-management
description: Task management CLI for tracking feature subtasks
---
# Task Management Skill
**Purpose**: Track and manage feature subtasks
## What I do
- Track task progress
- Show next eligible tasks
- Identify blocked tasks
- Mark completion
- Validate task integrity
## Usage
```bash
# Show all task statuses
npx ts-node .opencode/skills/task-management/scripts/task-cli.ts status
# Show next eligible tasks
npx ts-node .opencode/skills/task-management/scripts/task-cli.ts next
# Mark complete
npx ts-node .opencode/skills/task-management/scripts/task-cli.ts complete <feature> <seq> "summary"
```
```
---
## router.sh
```bash
#!/bin/bash
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
case "$1" in
help|--help|-h)
echo "Task Management Skill"
echo "Usage: bash router.sh <command>"
echo "Commands: status, next, blocked, complete, validate"
;;
status|next|blocked|validate)
npx ts-node "$SCRIPT_DIR/scripts/task-cli.ts" "$@"
;;
complete)
npx ts-node "$SCRIPT_DIR/scripts/task-cli.ts" "$@"
;;
*)
echo "Unknown command: $1"
bash "$0" help
;;
esac
```
---
## task-cli.ts (Excerpt)
```typescript
#!/usr/bin/env ts-node
interface Task {
id: string
status: 'pending' | 'in_progress' | 'completed'
title: string
}
async function main() {
const command = process.argv[2] || 'help'
switch (command) {
case 'status':
await showStatus()
break
case 'next':
await showNext()
break
case 'complete':
const [, , , feature, seq, summary] = process.argv
await markComplete(feature, seq, summary)
break
default:
showHelp()
}
}
async function showStatus() {
// Implementation
console.log('Task status...')
}
async function showNext() {
// Implementation
console.log('Next tasks...')
}
async function markComplete(feature: string, seq: string, summary: string) {
// Implementation
console.log(`Completing ${feature} ${seq}: ${summary}`)
}
function showHelp() {
console.log(`
Task Management CLI
Commands:
status Show all task statuses
next Show next eligible tasks
blocked Show blocked tasks
complete <f> <s> Mark task complete
validate Validate task integrity
`)
}
main().catch(console.error)
```
---
## Integration with Agents
Skills integrate with agents via:
- Event hooks (`tool.execute.before`, `tool.execute.after`)
- Skill content injection into conversation
- Output enhancement
Example agent prompt invoking skill:
```
Use the task-management skill to show current task status
```
---
## Related
- `adding-skill-basics.md` - Directory and SKILL.md setup
- `adding-skill-implementation.md` - CLI and registry
- `plugins/context/capabilities/events_skills.md` - Skills Plugin
@@ -1,167 +0,0 @@
# Guide: OpenCode Skill Implementation
**Prerequisites**: Load `adding-skill-basics.md` first
**Purpose**: CLI implementation, registry, and testing for OpenCode skills
---
## CLI Implementation
### Basic Structure
```typescript
#!/usr/bin/env ts-node
// CLI implementation for {skill-name} skill
interface Args {
command: string
[key: string]: any
}
async function main() {
const args = parseArgs()
switch (args.command) {
case 'command1':
await handleCommand1(args)
break
case 'command2':
await handleCommand2(args)
break
case 'help':
default:
showHelp()
}
}
function parseArgs(): Args {
const args = process.argv.slice(2)
return {
command: args[0] || 'help',
...parseOptions(args.slice(1))
}
}
async function handleCommand1(args: Args) {
console.log('Running command1...')
}
function showHelp() {
console.log(`
{Skill Name}
Usage: npx ts-node scripts/skill-cli.ts <command> [options]
Commands:
command1 Description
command2 Description
help Show this help
`)
}
main().catch(console.error)
```
---
## Register in Registry (Optional)
### Add to Components
```json
{
"skills": [
{
"id": "{skill-name}",
"name": "Skill Name",
"type": "skill",
"path": ".opencode/skills/{skill-name}/SKILL.md",
"description": "Brief description",
"tags": ["tag1", "tag2"],
"dependencies": []
}
]
}
```
### Add to Profiles
```json
{
"profiles": {
"essential": {
"components": [
"skill:{skill-name}"
]
}
}
}
```
---
## Testing
### Test CLI Commands
```bash
# Test help
bash .opencode/skills/{skill-name}/router.sh help
# Test commands
bash .opencode/skills/{skill-name}/router.sh command1 --option value
# Test with npx
npx ts-node .opencode/skills/{skill-name}/scripts/skill-cli.ts help
```
### Test OpenCode Integration
1. Call skill via OpenCode
2. Verify event hooks fire correctly
3. Check conversation history for skill content
4. Verify output enhancement works
---
## Best Practices
### Keep Skills Focused
- ✅ Task management skill → Tracks tasks
- ❌ Task management + code generation + testing → Too broad
### Clear Documentation
- Provide usage examples
- Document all commands
- Include expected outputs
### Error Handling
- Handle missing arguments gracefully
- Provide helpful error messages
- Validate inputs before processing
### Performance
- Use efficient algorithms
- Cache when appropriate
- Avoid unnecessary file operations
---
## Checklist
- [ ] `.opencode/skills/{skill-name}/SKILL.md` created
- [ ] `.opencode/skills/{skill-name}/router.sh` created (if CLI-based)
- [ ] Router script is executable (`chmod +x`)
- [ ] Registry updated (if needed)
- [ ] Profile updated (if needed)
- [ ] All commands tested
- [ ] Documentation complete
---
## Related
- `adding-skill-basics.md` - Directory and SKILL.md setup
- `adding-skill-example.md` - Complete example
- `creating-skills.md` - Claude Code Skills
- `plugins/context/capabilities/events_skills.md` - Skills Plugin
@@ -1,97 +0,0 @@
# Building CLIs in OpenAgents Control: Compact Guide
**Category**: guide
**Purpose**: Rapidly build, register, and deploy CLI tools for OpenAgents Control skills
**Framework**: FAB (Features, Advantages, Benefits)
---
## 🚀 Quick Start
**Don't start from scratch.** Use the standard pattern to build robust CLIs in minutes.
1. **Create**: `mkdir -p .opencode/skills/{name}/scripts`
2. **Implement**: Create `skill-cli.ts` (TypeScript) and `router.sh` (Bash)
3. **Register**: Add to `registry.json`
4. **Run**: `bash .opencode/skills/{name}/router.sh help`
---
## 🏗️ Core Architecture
| Component | File | Purpose |
|-----------|------|---------|
| **Logic** | `scripts/skill-cli.ts` | Type-safe implementation using `ts-node`. Handles args, logic, and output. |
| **Router** | `router.sh` | Universal entry point. Routes commands to the TS script. |
| **Docs** | `SKILL.md` | User guide, examples, and integration details. |
| **Config** | `registry.json` | Makes the skill discoverable and installable via `install.sh`. |
---
## ⚡ Implementation Patterns
### 1. The Router (`router.sh`)
**Why**: Provides a consistent, dependency-free entry point for all environments.
```bash
#!/bin/bash
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
case "$1" in
help|--help|-h)
echo "Usage: bash router.sh <command>"
;;
*)
# Route to TypeScript implementation
npx ts-node "$SCRIPT_DIR/scripts/skill-cli.ts" "$@"
;;
esac
```
### 2. The CLI Logic (`skill-cli.ts`)
**Why**: Type safety, async/await support, and rich ecosystem access.
```typescript
#!/usr/bin/env ts-node
async function main() {
const [command, ...args] = process.argv.slice(2);
switch (command) {
case 'action':
await handleAction(args);
break;
default:
console.log("Unknown command");
process.exit(1);
}
}
main().catch(console.error);
```
---
## ✅ Quality Checklist
Before shipping, verify your CLI delivers value:
- [ ] **Help Command**: Does `router.sh help` provide clear, actionable usage info?
- [ ] **Error Handling**: Do invalid inputs return helpful error messages (not stack traces)?
- [ ] **Performance**: Does it start in < 1s? (Avoid heavy imports at top level)
- [ ] **Idempotency**: Can commands be run multiple times safely?
- [ ] **Registry**: Is it added to `registry.json` with correct paths?
---
## 🧠 Copywriting Principles for CLI Output
Apply `content-creation` principles to your CLI output:
1. **Clarity**: Use **Active Voice**. "Created file" (Good) vs "File has been created" (Bad).
2. **Specificity**: "Processed 5 files" (Good) vs "Processing complete" (Bad).
3. **Action**: Tell the user what to do next. "Run `npm test` to verify."
---
**Reference**: See `.opencode/context/openagents-repo/guides/adding-skill.md` for the full, detailed walkthrough.
@@ -1,289 +0,0 @@
# Guide: Creating a Release
**Purpose**: Step-by-step workflow for creating a new release
---
## Quick Steps
```bash
# 1. Update version
echo "0.X.Y" > VERSION
jq '.version = "0.X.Y"' package.json > tmp && mv tmp package.json
# 2. Update CHANGELOG
# (Edit CHANGELOG.md manually)
# 3. Commit and tag
git add VERSION package.json CHANGELOG.md
git commit -m "chore: bump version to 0.X.Y"
git tag -a v0.X.Y -m "Release v0.X.Y"
# 4. Push
git push origin main
git push origin v0.X.Y
```
---
## Step 1: Determine Version
### Semantic Versioning
```
MAJOR.MINOR.PATCH
- MAJOR: Breaking changes
- MINOR: New features (backward compatible)
- PATCH: Bug fixes
```
### Examples
- `0.5.0` → `0.5.1` (bug fix)
- `0.5.0` → `0.6.0` (new feature)
- `0.5.0` → `1.0.0` (breaking change)
---
## Step 2: Update Version Files
### VERSION File
```bash
echo "0.X.Y" > VERSION
```
### package.json
```bash
jq '.version = "0.X.Y"' package.json > tmp && mv tmp package.json
```
### Verify Consistency
```bash
cat VERSION
cat package.json | jq '.version'
# Both should show same version
```
---
## Step 3: Update CHANGELOG
### Format
```markdown
# Changelog
## [0.X.Y] - 2025-12-10
### Added
- New feature 1
- New feature 2
### Changed
- Updated feature 1
- Improved feature 2
### Fixed
- Bug fix 1
- Bug fix 2
### Removed
- Deprecated feature 1
## [Previous Version] - Date
...
```
### Tips
✅ **Group by type** - Added, Changed, Fixed, Removed
✅ **User-focused** - Describe impact, not implementation
✅ **Link PRs** - Reference PR numbers
✅ **Breaking changes** - Clearly mark breaking changes
---
## Step 4: Commit Changes
```bash
# Stage files
git add VERSION package.json CHANGELOG.md
# Commit
git commit -m "chore: bump version to 0.X.Y"
```
---
## Step 5: Create Git Tag
```bash
# Create annotated tag
git tag -a v0.X.Y -m "Release v0.X.Y"
# Verify tag
git tag -l "v0.X.Y"
git show v0.X.Y
```
---
## Step 6: Push to GitHub
```bash
# Push commit
git push origin main
# Push tag
git push origin v0.X.Y
```
---
## Step 7: Create GitHub Release
### Via GitHub UI
1. Go to repository on GitHub
2. Click "Releases"
3. Click "Create a new release"
4. Select tag: `v0.X.Y`
5. Title: `v0.X.Y`
6. Description: Copy from CHANGELOG
7. Click "Publish release"
### Via GitHub CLI
```bash
gh release create v0.X.Y \
--title "v0.X.Y" \
--notes "$(cat CHANGELOG.md | sed -n '/## \[0.X.Y\]/,/## \[/p' | head -n -1)"
```
---
## Step 8: Verify Release
### Check GitHub
- ✅ Release appears on GitHub
- ✅ Tag is correct
- ✅ CHANGELOG is included
- ✅ Assets are attached (if any)
### Test Installation
```bash
# Test install from GitHub
./install.sh --list
# Verify version
cat VERSION
```
---
## Complete Example
```bash
# Releasing v0.6.0
# 1. Update version
echo "0.6.0" > VERSION
jq '.version = "0.6.0"' package.json > tmp && mv tmp package.json
# 2. Update CHANGELOG
cat >> CHANGELOG.md << 'EOF'
## [0.6.0] - 2025-12-10
### Added
- New API specialist agent
- GraphQL support in backend specialist
### Changed
- Improved eval framework performance
- Updated registry schema to 2.0.0
### Fixed
- Fixed path resolution for subagents
- Fixed registry validation edge cases
EOF
# 3. Commit
git add VERSION package.json CHANGELOG.md
git commit -m "chore: bump version to 0.6.0"
# 4. Tag
git tag -a v0.6.0 -m "Release v0.6.0"
# 5. Push
git push origin main
git push origin v0.6.0
# 6. Create GitHub release
gh release create v0.6.0 \
--title "v0.6.0" \
--notes "See CHANGELOG.md for details"
```
---
## Checklist
Before releasing:
- [ ] All tests pass
- [ ] Registry validates
- [ ] VERSION updated
- [ ] package.json updated
- [ ] CHANGELOG updated
- [ ] Changes committed
- [ ] Tag created
- [ ] Pushed to GitHub
- [ ] GitHub release created
- [ ] Installation tested
---
## Common Issues
### Version Mismatch
**Problem**: VERSION and package.json don't match
**Solution**: Update both to same version
### Tag Already Exists
**Problem**: Tag already exists
**Solution**: Delete tag and recreate
```bash
git tag -d v0.X.Y
git push origin :refs/tags/v0.X.Y
```
### Push Rejected
**Problem**: Push rejected (not up to date)
**Solution**: Pull latest changes first
```bash
git pull origin main
git push origin main
git push origin v0.X.Y
```
---
## Related Files
- **Version management**: `scripts/versioning/bump-version.sh`
- **CHANGELOG**: `CHANGELOG.md`
- **VERSION**: `VERSION`
---
**Last Updated**: 2025-12-10
**Version**: 0.5.0
@@ -1,399 +0,0 @@
# Guide: Debugging Common Issues
**Purpose**: Troubleshooting guide for common problems
---
## Quick Diagnostics
```bash
# Check system health
./scripts/registry/validate-registry.sh
./scripts/validation/validate-test-suites.sh
# Check version consistency
cat VERSION && cat package.json | jq '.version'
# Test core agents
cd evals/framework && npm run eval:sdk -- --agent=core/openagent --pattern="smoke-test.yaml"
```
---
## Registry Issues
### Registry Validation Fails
**Symptoms**:
```
ERROR: Path does not exist: .opencode/agent/core/missing.md
```
**Diagnosis**:
```bash
./scripts/registry/validate-registry.sh -v
```
**Solutions**:
1. **Path doesn't exist**: Remove entry or create file
2. **Duplicate ID**: Rename one component
3. **Invalid category**: Use valid category
**Fix**:
```bash
# Re-run auto-detect
./scripts/registry/auto-detect-components.sh --auto-add
# Validate
./scripts/registry/validate-registry.sh
```
---
### Component Not in Registry
**Symptoms**:
- Component doesn't appear in `./install.sh --list`
- Auto-detect doesn't find component
**Diagnosis**:
```bash
# Check frontmatter
head -10 .opencode/agent/{category}/{agent}.md
# Dry run auto-detect
./scripts/registry/auto-detect-components.sh --dry-run
```
**Solutions**:
1. **Missing frontmatter**: Add frontmatter
2. **Invalid YAML**: Fix YAML syntax
3. **Wrong location**: Move to correct directory
**Fix**:
```bash
# Add frontmatter
cat > .opencode/agent/{category}/{agent}.md << 'EOF'
---
description: "Brief description"
category: "category"
type: "agent"
---
# Agent Content
EOF
# Re-run auto-detect
./scripts/registry/auto-detect-components.sh --auto-add
```
---
## Test Failures
### Approval Gate Violation
**Symptoms**:
```
✗ Approval Gate: FAIL
Violation: Agent executed write tool without requesting approval
```
**Diagnosis**:
```bash
# Run with debug
cd evals/framework
npm run eval:sdk -- --agent={agent} --pattern="{test}" --debug
# Check session
ls -lt .tmp/sessions/ | head -5
cat .tmp/sessions/{session-id}/session.json | jq
```
**Solution**:
Add approval request in agent prompt:
```markdown
Before executing:
1. Present plan to user
2. Request approval
3. Execute after approval
```
---
### Context Loading Violation
**Symptoms**:
```
✗ Context Loading: FAIL
Violation: Agent executed write tool without loading required context
```
**Diagnosis**:
```bash
# Check what context was loaded
cat .tmp/sessions/{session-id}/events.json | jq '.[] | select(.type == "context_load")'
```
**Solution**:
Add context loading in agent prompt:
```markdown
Before implementing:
1. Load core/standards/code-quality.md
2. Apply standards to implementation
```
---
### Tool Usage Violation
**Symptoms**:
```
✗ Tool Usage: FAIL
Violation: Agent used bash tool for reading file instead of read tool
```
**Diagnosis**:
```bash
# Check tool usage
cat .tmp/sessions/{session-id}/events.json | jq '.[] | select(.type == "tool_call")'
```
**Solution**:
Update agent to use correct tools:
- Use `read` instead of `bash cat`
- Use `list` instead of `bash ls`
- Use `grep` instead of `bash grep`
---
## Install Issues
### Install Script Fails
**Symptoms**:
```
ERROR: Failed to fetch registry
ERROR: Component not found
```
**Diagnosis**:
```bash
# Check dependencies
which curl jq
# Test with local registry
REGISTRY_URL="file://$(pwd)/registry.json" ./install.sh --list
```
**Solutions**:
1. **Missing dependencies**: Install curl and jq
2. **Registry not found**: Check registry.json exists
3. **Component not found**: Verify component in registry
**Fix**:
```bash
# Install dependencies (macOS)
brew install curl jq
# Install dependencies (Linux)
sudo apt-get install curl jq
# Test locally
REGISTRY_URL="file://$(pwd)/registry.json" ./install.sh --list
```
---
### Collision Handling
**Symptoms**:
```
File exists: .opencode/agent/core/openagent.md
```
**Solutions**:
1. **Skip**: Keep existing file
2. **Overwrite**: Replace with new file
3. **Backup**: Backup existing, install new
**Fix**:
```bash
# Skip all collisions
./install.sh developer --skip-existing
# Overwrite all collisions
./install.sh developer --force
# Backup all collisions
./install.sh developer --backup
```
---
## Path Resolution Issues
### Agent Not Found
**Symptoms**:
```
ERROR: Agent not found: development/frontend-specialist
```
**Diagnosis**:
```bash
# Check file exists
ls -la .opencode/agent/development/frontend-specialist.md
# Check registry
cat registry.json | jq '.components.agents[] | select(.id == "frontend-specialist")'
```
**Solutions**:
1. **File doesn't exist**: Create file
2. **Wrong path**: Fix path in registry
3. **Not in registry**: Run auto-detect
**Fix**:
```bash
# Re-run auto-detect
./scripts/registry/auto-detect-components.sh --auto-add
# Validate
./scripts/registry/validate-registry.sh
```
---
## Version Issues
### Version Mismatch
**Symptoms**:
```
VERSION: 0.5.0
package.json: 0.4.0
registry.json: 0.5.0
```
**Diagnosis**:
```bash
cat VERSION
cat package.json | jq '.version'
cat registry.json | jq '.version'
```
**Solution**:
Update all to same version:
```bash
echo "0.5.0" > VERSION
jq '.version = "0.5.0"' package.json > tmp && mv tmp package.json
jq '.version = "0.5.0"' registry.json > tmp && mv tmp registry.json
```
---
## CI/CD Issues
### Workflow Fails
**Symptoms**:
- Registry validation fails in CI
- Tests fail in CI but pass locally
**Diagnosis**:
```bash
# Run same commands as CI
./scripts/registry/validate-registry.sh
./scripts/validation/validate-test-suites.sh
cd evals/framework && npm run eval:sdk
```
**Solutions**:
1. **Registry invalid**: Fix registry
2. **Tests fail**: Fix tests
3. **Dependencies missing**: Update CI config
---
## Performance Issues
### Tests Timeout
**Symptoms**:
```
ERROR: Test timeout after 60000ms
```
**Solution**:
Increase timeout in config.yaml:
```yaml
timeout: 120000 # 2 minutes
```
---
### Slow Auto-Detect
**Symptoms**:
Auto-detect takes too long
**Solution**:
Limit scope:
```bash
# Only scan specific directory
./scripts/registry/auto-detect-components.sh --path .opencode/agent/development/
```
---
## Getting Help
### Check Logs
```bash
# Session logs
ls -lt .tmp/sessions/ | head -5
cat .tmp/sessions/{session-id}/session.json | jq
# Event timeline
cat .tmp/sessions/{session-id}/events.json | jq
```
### Run Diagnostics
```bash
# Full system check
./scripts/registry/validate-registry.sh -v
./scripts/validation/validate-test-suites.sh
cd evals/framework && npm run eval:sdk -- --agent=core/openagent
```
### Common Commands
```bash
# Validate everything
./scripts/registry/validate-registry.sh && \
./scripts/validation/validate-test-suites.sh && \
cd evals/framework && npm run eval:sdk
# Reset and rebuild
./scripts/registry/auto-detect-components.sh --auto-add --force
./scripts/registry/validate-registry.sh
# Test installation
REGISTRY_URL="file://$(pwd)/registry.json" ./install.sh --list
```
---
## Related Files
- **Testing guide**: `guides/testing-agent.md`
- **Registry guide**: `guides/updating-registry.md`
- **Eval concepts**: `core-concepts/evals.md`
---
**Last Updated**: 2025-12-10
**Version**: 0.5.0
@@ -1,223 +0,0 @@
<!-- Context: openagents-repo/guides/external-libraries-workflow | Priority: high | Version: 1.0 | Updated: 2026-01-29 -->
# Guide: External Libraries Workflow
**Purpose**: Fetch current documentation for external packages when adding agents or skills
**When to Use**: Any time you're working with external libraries (Drizzle, Better Auth, Next.js, etc.)
**Time to Read**: 5 minutes
---
## Quick Start
**Golden Rule**: NEVER rely on training data for external libraries → ALWAYS fetch current docs
**Process**:
1. Detect external package in your task
2. Check for install scripts (if first-time setup)
3. Use **ExternalScout** to fetch current documentation
4. Implement with fresh, version-specific knowledge
---
## When to Use ExternalScout (MANDATORY)
✅ **Use ExternalScout when**:
- Adding new agents that depend on external packages
- Adding new skills that integrate with external libraries
- First-time package setup in your implementation
- Package/dependency errors occur
- Version upgrades are needed
- ANY external library work
❌ **Don't rely on**:
- Training data (outdated, often wrong)
- Old documentation (APIs change)
- Assumptions about package behavior
---
## Why This Matters
**Example**: Next.js Evolution
```
Training data (2023): Next.js 13 uses pages/ directory
Current (2025): Next.js 15 uses app/ directory (App Router)
Training data = broken code ❌
ExternalScout = working code ✅
```
**Real Impact**:
- APIs change (new methods, deprecated features)
- Configuration patterns evolve
- Breaking changes happen frequently
- Version-specific features differ
---
## Workflow Steps
### Step 1: Detect External Package
**Triggers**:
- User mentions a library name
- You see imports in code
- package.json has new dependencies
- Build errors reference external packages
**Action**: Identify which external packages are involved
**Example**:
```
User: "Add authentication with Better Auth"
→ External package detected: Better Auth
→ Proceed to Step 2
```
---
### Step 2: Check Install Scripts (First-Time Only)
**For first-time package setup**, check if there are install scripts:
```bash
# Look for install scripts
ls scripts/install/ scripts/setup/ bin/install* setup.sh install.sh
# Check package-specific requirements
grep -r "postinstall\|preinstall" package.json
```
**If scripts exist**:
- Read them to understand setup order
- Check for environment variables needed
- Identify prerequisites (database, services)
- Follow their guidance before implementing
**Why**: Scripts may set up databases, generate files, or configure services in a specific order
---
### Step 3: Fetch Current Documentation (MANDATORY)
**Use ExternalScout** to get live, version-specific documentation:
```bash
# Invoke ExternalScout via task tool
task(
subagent_type="ExternalScout",
description="Fetch Drizzle ORM documentation",
prompt="Fetch current documentation for Drizzle ORM focusing on:
- Modular schema patterns
- Next.js integration
- Database setup
- Migration strategies"
)
```
**What ExternalScout Returns**:
- Live documentation from official sources
- Version-specific features
- Integration patterns
- Setup requirements
- Code examples
**Supported Libraries** (18+):
- Drizzle ORM
- Better Auth
- Next.js
- TanStack Query/Router/Start
- Cloudflare Workers
- AWS Lambda
- Vercel
- Shadcn/ui
- Radix UI
- Tailwind CSS
- Zustand
- Jotai
- Zod
- React Hook Form
- Vitest
- Playwright
- And more...
---
### Step 4: Implement with Fresh Knowledge
**Now implement** using the documentation from ExternalScout:
- Follow current best practices
- Use version-specific APIs
- Apply recommended patterns
- Reference the fetched docs in your code
---
## Integration with Agent/Skill Creation
### When Adding an Agent
1. Read: `guides/adding-agent.md`
2. **If agent uses external packages**:
- Use ExternalScout to fetch docs
- Document dependencies in agent metadata
- Add to registry with correct versions
3. Test: `guides/testing-agent.md`
### When Adding a Skill
1. Read: `guides/adding-skill.md`
2. **If skill uses external packages**:
- Use ExternalScout to fetch docs
- Document dependencies in skill metadata
- Add to registry with correct versions
3. Test: `guides/testing-subagents.md`
---
## Common Packages in OpenAgents
| Package | Use Case | Priority |
|---------|----------|----------|
| **Drizzle ORM** | Database schemas & queries | ⭐⭐⭐⭐⭐ |
| **Better Auth** | Authentication & authorization | ⭐⭐⭐⭐⭐ |
| **Next.js** | Full-stack web framework | ⭐⭐⭐⭐⭐ |
| **TanStack Query** | Server state management | ⭐⭐⭐⭐ |
| **Zod** | Schema validation | ⭐⭐⭐⭐ |
| **Tailwind CSS** | Styling | ⭐⭐⭐⭐ |
| **Shadcn/ui** | UI components | ⭐⭐⭐ |
| **Vitest** | Testing framework | ⭐⭐⭐ |
---
## Checklist
Before implementing with external libraries:
- [ ] Identified all external packages involved
- [ ] Checked for install scripts (if first-time)
- [ ] Used ExternalScout to fetch current docs
- [ ] Reviewed version-specific features
- [ ] Documented dependencies in metadata
- [ ] Added to registry with correct versions
- [ ] Tested implementation thoroughly
- [ ] Referenced ExternalScout docs in code comments
---
## Related Guides
- `guides/adding-agent.md` - Creating new agents
- `guides/adding-skill.md` - Creating new skills
- `guides/debugging.md` - Troubleshooting (includes dependency issues)
- `guides/updating-registry.md` - Registry management
---
## Key Principle
> **External libraries change constantly. Your training data is outdated. Always fetch current documentation before implementing.**
This is not optional - it's the difference between working code and broken code.
@@ -1,471 +0,0 @@
# Guide: GitHub Issues and Project Board Workflow
**Prerequisites**: Basic understanding of GitHub issues and projects
**Purpose**: Step-by-step workflow for managing issues and project board
---
## Overview
This guide covers how to work with GitHub issues and the project board to track and process different requests, features, and improvements.
**Project Board**: https://github.com/users/darrenhinde/projects/2/views/2
**Time**: Varies by task
---
## Quick Commands Reference
```bash
# List issues
gh issue list --repo darrenhinde/OpenAgentsControl
# Create issue
gh issue create --repo darrenhinde/OpenAgentsControl --title "Title" --body "Body" --label "label1,label2"
# Add issue to project
gh project item-add 2 --owner darrenhinde --url https://github.com/darrenhinde/OpenAgentsControl/issues/NUMBER
# View issue
gh issue view NUMBER --repo darrenhinde/OpenAgentsControl
# Update issue
gh issue edit NUMBER --repo darrenhinde/OpenAgentsControl --add-label "new-label"
# Close issue
gh issue close NUMBER --repo darrenhinde/OpenAgentsControl
```
---
## Step 1: Creating Issues
### Issue Types
**Feature Request**
- Labels: `feature`, `enhancement`
- Include: Goals, key features, success criteria
- Template: See "Feature Issue Template" below
**Bug Report**
- Labels: `bug`
- Include: Steps to reproduce, expected vs actual behavior
- Template: See "Bug Issue Template" below
**Improvement**
- Labels: `enhancement`, `framework`
- Include: Current state, proposed improvement, impact
**Question**
- Labels: `question`
- Include: Context, specific question, use case
### Priority Labels
- `priority-high` - Critical, blocking work
- `priority-medium` - Important, not blocking
- `priority-low` - Nice to have
### Category Labels
- `agents` - Agent system related
- `framework` - Core framework changes
- `evals` - Evaluation framework
- `idea` - High-level proposal
### Creating an Issue
```bash
# Basic issue
gh issue create \
--repo darrenhinde/OpenAgentsControl \
--title "Add new feature X" \
--body "Description of feature" \
--label "feature,priority-medium"
# Feature with detailed body
gh issue create \
--repo darrenhinde/OpenAgentsControl \
--title "Build plugin system" \
--label "feature,framework,priority-high" \
--body "$(cat <<'EOF'
## Overview
Brief description
## Goals
- Goal 1
- Goal 2
## Key Features
- Feature 1
- Feature 2
## Success Criteria
- [ ] Criterion 1
- [ ] Criterion 2
EOF
)"
```
---
## Step 2: Adding Issues to Project Board
### Add Single Issue
```bash
# Add issue to project
gh project item-add 2 \
--owner darrenhinde \
--url https://github.com/darrenhinde/OpenAgentsControl/issues/NUMBER
```
### Add Multiple Issues
```bash
# Add issues 137-142 to project
for i in {137..142}; do
gh project item-add 2 \
--owner darrenhinde \
--url https://github.com/darrenhinde/OpenAgentsControl/issues/$i
done
```
### Verify Issues on Board
```bash
# View project items
gh project item-list 2 --owner darrenhinde --format json | jq '.items[] | {title, status}'
```
---
## Step 3: Processing Issues
### Workflow States
1. **Backlog** - New issues, not yet prioritized
2. **Todo** - Prioritized, ready to work on
3. **In Progress** - Currently being worked on
4. **In Review** - PR submitted, awaiting review
5. **Done** - Completed and merged
### Moving Issues
```bash
# Update issue status (via project board UI or gh CLI)
# Note: Status updates are typically done via web UI
```
### Assigning Issues
```bash
# Assign to yourself
gh issue edit NUMBER \
--repo darrenhinde/OpenAgentsControl \
--add-assignee @me
# Assign to someone else
gh issue edit NUMBER \
--repo darrenhinde/OpenAgentsControl \
--add-assignee username
```
---
## Step 4: Working on Issues
### Start Work
1. **Assign issue to yourself**
```bash
gh issue edit NUMBER --repo darrenhinde/OpenAgentsControl --add-assignee @me
```
2. **Move to "In Progress"** (via web UI)
3. **Create branch** (optional)
```bash
git checkout -b feature/issue-NUMBER-description
```
4. **Reference issue in commits**
```bash
git commit -m "feat: implement X (#NUMBER)"
```
### Update Progress
```bash
# Add comment to issue
gh issue comment NUMBER \
--repo darrenhinde/OpenAgentsControl \
--body "Progress update: Completed X, working on Y"
```
### Complete Work
1. **Create PR**
```bash
gh pr create \
--repo darrenhinde/OpenAgentsControl \
--title "Fix #NUMBER: Description" \
--body "Closes #NUMBER\n\nChanges:\n- Change 1\n- Change 2"
```
2. **Move to "In Review"** (via web UI)
3. **After merge, issue auto-closes** (if PR uses "Closes #NUMBER")
---
## Step 5: Using Issues for Request Processing
### Request Types
**User Feature Request**
1. Create issue with `feature` label
2. Add to project board
3. Prioritize based on impact
4. Break down into subtasks if needed
5. Assign to appropriate person/team
**Bug Report**
1. Create issue with `bug` label
2. Add reproduction steps
3. Prioritize based on severity
4. Assign for investigation
5. Link to related issues if applicable
**Improvement Suggestion**
1. Create issue with `enhancement` label
2. Discuss approach in comments
3. Get consensus before implementation
4. Create implementation plan
5. Execute and track progress
### Breaking Down Large Issues
For complex features, create parent issue and subtasks:
```bash
# Parent issue
gh issue create \
--repo darrenhinde/OpenAgentsControl \
--title "[EPIC] Plugin System" \
--label "feature,framework,priority-high" \
--body "Parent issue for plugin system work"
# Subtask issues
gh issue create \
--repo darrenhinde/OpenAgentsControl \
--title "Plugin manifest system" \
--label "feature" \
--body "Part of #PARENT_NUMBER\n\nImplement plugin.json manifest"
```
---
## Step 6: Issue Templates
### Feature Issue Template
```markdown
## Overview
Brief description of the feature
## Goals
- Goal 1
- Goal 2
- Goal 3
## Key Features
- Feature 1
- Feature 2
- Feature 3
## Related Issues
- #123 (related issue)
## Success Criteria
- [ ] Criterion 1
- [ ] Criterion 2
- [ ] Criterion 3
```
### Bug Issue Template
```markdown
## Description
Brief description of the bug
## Steps to Reproduce
1. Step 1
2. Step 2
3. Step 3
## Expected Behavior
What should happen
## Actual Behavior
What actually happens
## Environment
- OS: macOS/Linux/Windows
- Version: 0.5.2
- Node: v20.x
## Additional Context
Any other relevant information
```
### Improvement Issue Template
```markdown
## Current State
Description of current implementation
## Proposed Improvement
What should be improved and why
## Impact
- Performance improvement
- Developer experience
- User experience
## Implementation Approach
High-level approach to implementation
## Success Criteria
- [ ] Criterion 1
- [ ] Criterion 2
```
---
## Step 7: Automation and Integration
### Auto-Close Issues
Use keywords in PR descriptions:
- `Closes #123`
- `Fixes #123`
- `Resolves #123`
### Link Issues to PRs
```bash
# In PR description
gh pr create \
--title "Add feature X" \
--body "Implements #123\n\nChanges:\n- Change 1"
```
### Issue References in Commits
```bash
# Reference issue in commit
git commit -m "feat: add plugin system (#137)"
# Close issue in commit
git commit -m "fix: resolve permission error (closes #140)"
```
---
## Best Practices
### Issue Creation
✅ **Clear titles** - Descriptive and specific
✅ **Detailed descriptions** - Include context and goals
✅ **Proper labels** - Use consistent labeling
✅ **Success criteria** - Define what "done" means
✅ **Link related issues** - Show dependencies
### Issue Management
✅ **Regular triage** - Review and prioritize weekly
✅ **Keep updated** - Add comments on progress
✅ **Close stale issues** - Clean up old/irrelevant issues
✅ **Use milestones** - Group related issues
✅ **Assign owners** - Clear responsibility
### Project Board
✅ **Update status** - Keep board current
✅ **Limit WIP** - Don't overload "In Progress"
✅ **Review regularly** - Weekly board review
✅ **Archive completed** - Keep board clean
---
## Common Workflows
### Processing User Request
1. **Receive request** (via issue, email, chat)
2. **Create issue** with appropriate labels
3. **Add to project board**
4. **Triage and prioritize**
5. **Assign to team member**
6. **Track progress** via status updates
7. **Review and merge** PR
8. **Close issue** and notify requester
### Planning New Feature
1. **Create epic issue** for overall feature
2. **Break down into subtasks**
3. **Add all to project board**
4. **Prioritize subtasks**
5. **Assign to team members**
6. **Track progress** across subtasks
7. **Complete and close** when all subtasks done
### Bug Triage
1. **Create bug issue** with reproduction steps
2. **Label with severity** (critical, high, medium, low)
3. **Add to project board**
4. **Assign for investigation**
5. **Reproduce and diagnose**
6. **Fix and test**
7. **Create PR** with fix
8. **Close issue** after merge
---
## Checklist
Before closing an issue:
- [ ] All success criteria met
- [ ] Tests passing
- [ ] Documentation updated
- [ ] PR merged (if applicable)
- [ ] Related issues updated
- [ ] Stakeholders notified
---
## Related Files
- **Registry guide**: `guides/updating-registry.md`
- **Release guide**: `guides/creating-release.md`
- **Testing guide**: `guides/testing-agent.md`
- **Debugging**: `guides/debugging.md`
---
## External Resources
- [GitHub Issues Documentation](https://docs.github.com/en/issues)
- [GitHub Projects Documentation](https://docs.github.com/en/issues/planning-and-tracking-with-projects)
- [GitHub CLI Documentation](https://cli.github.com/manual/)
---
**Last Updated**: 2026-01-30
**Version**: 0.5.2
@@ -1,42 +0,0 @@
# OpenAgents Guides
**Purpose**: Step-by-step guides for working with OpenAgents Control
---
## Structure
```
openagents-repo/guides/
├── navigation.md (this file)
└── [guide files]
```
---
## Quick Routes
| Task | Path |
|------|------|
| **Adding agents (basics)** | `./adding-agent-basics.md` |
| **Adding agents (tests)** | `./adding-agent-testing.md` |
| **Adding OpenCode skills** | `./adding-skill-basics.md` |
| **Creating Claude Code skills** | `./creating-skills.md` |
| **Creating Claude Code subagents** | `./creating-subagents.md` |
| **Testing subagents** | `./testing-subagents.md` |
---
## By Type
**Implementation Guides** → How to implement features
**Agent Guides** → How to work with agents
**Testing Guides** → How to test implementations
---
## Related Context
- **OpenAgents Navigation** → `../navigation.md`
- **Examples** → `../examples/navigation.md`
- **Core Concepts** → `../core-concepts/navigation.md`
@@ -1,153 +0,0 @@
# NPM Publishing Guide
**Purpose**: Quick reference for publishing OpenAgents Control to npm
**Time to Read**: 3 minutes
---
## Core Concept
OpenAgents Control is published as `@nextsystems/oac` on npm. Users install globally and run `oac [profile]` to set up their projects.
**Key files**:
- `package.json` - Package configuration
- `bin/oac.js` - CLI entry point
- `.npmignore` - Exclude dev files
- `install.sh` - Main installer (runs when user executes `oac`)
---
## Publishing Workflow
### 1. Prepare Release
```bash
# Update version
npm version patch # 0.7.0 -> 0.7.1
npm version minor # 0.7.0 -> 0.8.0
# Update VERSION file
node -p "require('./package.json').version" > VERSION
# Update CHANGELOG.md with changes
```
### 2. Test Locally
```bash
# Create package
npm pack
# Install globally from tarball
npm install -g ./nextsystems-oac-0.7.1.tgz
# Test CLI
oac --version
oac --help
# Uninstall
npm uninstall -g @nextsystems/oac
```
### 3. Publish
```bash
# Login (one-time)
npm login
# Publish (scoped packages need --access public)
npm publish --access public
```
### 4. Verify
```bash
# Check it's live
npm view @nextsystems/oac
# Test installation
npm install -g @nextsystems/oac
oac --version
```
### 5. Create GitHub Release
```bash
git tag v0.7.1
git push --tags
# Create release on GitHub with changelog
```
---
## User Installation
Once published, users can:
```bash
# Global install (recommended)
npm install -g @nextsystems/oac
oac developer
# Or use npx (no install)
npx @nextsystems/oac developer
```
---
## Common Issues
**"You do not have permission to publish"**
```bash
npm whoami # Check you're logged in
npm publish --access public # Scoped packages need public access
```
**"Version already exists"**
```bash
npm version patch # Bump version first
```
**"You must verify your email"**
```bash
npm profile get # Check email verification status
```
---
## Package Configuration
**What's included** (see `package.json` → `files`):
- `.opencode/` - Agents, commands, context, profiles, skills, tools
- `scripts/` - Installation scripts
- `bin/` - CLI entry point
- `registry.json` - Component registry
- `install.sh` - Main installer
- Docs (README, CHANGELOG, LICENSE)
**What's excluded** (see `.npmignore`):
- `node_modules/`
- `evals/`
- `.tmp/`
- Dev files
---
## Security
- ✅ Enable 2FA: `npm profile enable-2fa auth-and-writes`
- ✅ Use strong npm password
- ✅ `@nextsystems` scope is protected (only you can publish)
---
## References
- **Package**: https://www.npmjs.com/package/@nextsystems/oac
- **Stats**: https://npm-stat.com/charts.html?package=@nextsystems/oac
- **Codebase**: `package.json`, `bin/oac.js`, `.npmignore`
---
**Last Updated**: 2026-01-30
@@ -1,368 +0,0 @@
# Guide: Profile Validation
**Purpose**: Ensure installation profiles include all appropriate components
**Priority**: HIGH - Check this when adding new agents or updating registry
---
## What Are Profiles?
Profiles are pre-configured component bundles in `registry.json` that users install:
- **essential** - Minimal setup (openagent + core subagents)
- **developer** - Full dev environment (all dev agents + tools)
- **business** - Content/product focus (content agents + tools)
- **full** - Everything (all agents, subagents, tools)
- **advanced** - Full + meta-level (system-builder, repo-manager)
---
## The Problem
**Issue**: New agents added to `components.agents[]` but NOT added to profiles
**Result**: Users install a profile but don't get the new agents
**Example** (v0.5.0 bug):
```json
// ✅ Agent exists in components
{
"id": "devops-specialist",
"path": ".opencode/agent/development/devops-specialist.md"
}
// ❌ But NOT in developer profile
"developer": {
"components": [
"agent:openagent",
"agent:opencoder"
// Missing: "agent:devops-specialist"
]
}
```
---
## Validation Checklist
When adding a new agent, **ALWAYS** check:
### 1. Agent Added to Components
```bash
# Check agent exists in registry
cat registry.json | jq '.components.agents[] | select(.id == "your-agent")'
```
### 2. Agent Added to Appropriate Profiles
**Development agents** → Add to:
- ✅ `developer` profile
- ✅ `full` profile
- ✅ `advanced` profile
**Content agents** → Add to:
- ✅ `business` profile
- ✅ `full` profile
- ✅ `advanced` profile
**Data agents** → Add to:
- ✅ `business` profile (if business-focused)
- ✅ `full` profile
- ✅ `advanced` profile
**Meta agents** → Add to:
- ✅ `advanced` profile only
**Core agents** → Add to:
- ✅ `essential` profile
- ✅ All other profiles
### 3. Verify Profile Includes Agent
```bash
# Check if agent is in developer profile
cat registry.json | jq '.profiles.developer.components[] | select(. == "agent:your-agent")'
# Check if agent is in business profile
cat registry.json | jq '.profiles.business.components[] | select(. == "agent:your-agent")'
# Check if agent is in full profile
cat registry.json | jq '.profiles.full.components[] | select(. == "agent:your-agent")'
```
---
## Profile Assignment Rules
### Developer Profile
**Include**:
- Core agents (openagent, opencoder)
- Development specialist subagents (frontend, devops)
- All code subagents (tester, reviewer, coder-agent, build-agent)
- Dev commands (commit, test, validate-repo, analyze-patterns)
- Dev context (standards/code, standards/tests, workflows/*)
- Utility subagents (image-specialist for website images)
- Tools (env, gemini for image generation)
**Exclude**:
- Content agents (copywriter, technical-writer)
- Data agents (data-analyst)
- Meta agents (system-builder, repo-manager)
### Business Profile
**Include**:
- Core agent (openagent)
- Content specialists (copywriter, technical-writer)
- Data specialists (data-analyst)
- Image tools (gemini, image-specialist)
- Notification tools (notify)
**Exclude**:
- Development specialists
- Code subagents
- Meta agents
### Full Profile
**Include**:
- Everything from developer profile
- Everything from business profile
- All agents except meta agents
**Exclude**:
- Meta agents (system-builder, repo-manager)
### Advanced Profile
**Include**:
- Everything from full profile
- Meta agents (system-builder, repo-manager)
- Meta subagents (domain-analyzer, agent-generator, etc.)
- Meta commands (build-context-system)
---
## Automated Validation
### Script to Check Profile Coverage
```bash
#!/bin/bash
# Check if all agents are in appropriate profiles
echo "Checking profile coverage..."
# Get all agent IDs
agents=$(cat registry.json | jq -r '.components.agents[].id')
for agent in $agents; do
# Get agent category
category=$(cat registry.json | jq -r ".components.agents[] | select(.id == \"$agent\") | .category")
# Check which profiles include this agent
in_developer=$(cat registry.json | jq ".profiles.developer.components[] | select(. == \"agent:$agent\")" 2>/dev/null)
in_business=$(cat registry.json | jq ".profiles.business.components[] | select(. == \"agent:$agent\")" 2>/dev/null)
in_full=$(cat registry.json | jq ".profiles.full.components[] | select(. == \"agent:$agent\")" 2>/dev/null)
in_advanced=$(cat registry.json | jq ".profiles.advanced.components[] | select(. == \"agent:$agent\")" 2>/dev/null)
# Validate based on category
case $category in
"development")
if [[ -z "$in_developer" ]]; then
echo "❌ $agent (development) missing from developer profile"
fi
if [[ -z "$in_full" ]]; then
echo "❌ $agent (development) missing from full profile"
fi
if [[ -z "$in_advanced" ]]; then
echo "❌ $agent (development) missing from advanced profile"
fi
;;
"content"|"data")
if [[ -z "$in_business" ]]; then
echo "❌ $agent ($category) missing from business profile"
fi
if [[ -z "$in_full" ]]; then
echo "❌ $agent ($category) missing from full profile"
fi
if [[ -z "$in_advanced" ]]; then
echo "❌ $agent ($category) missing from advanced profile"
fi
;;
"meta")
if [[ -z "$in_advanced" ]]; then
echo "❌ $agent (meta) missing from advanced profile"
fi
;;
"essential"|"standard")
if [[ -z "$in_full" ]]; then
echo "❌ $agent ($category) missing from full profile"
fi
if [[ -z "$in_advanced" ]]; then
echo "❌ $agent ($category) missing from advanced profile"
fi
;;
esac
done
echo "✅ Profile coverage check complete"
```
Save this as: `scripts/registry/validate-profile-coverage.sh`
---
## Manual Validation Steps
### After Adding a New Agent
1. **Add agent to components**:
```bash
./scripts/registry/auto-detect-components.sh --auto-add
```
2. **Manually add to profiles**:
Edit `registry.json` and add `"agent:your-agent"` to appropriate profiles
3. **Validate registry**:
```bash
./scripts/registry/validate-registry.sh
```
4. **Test local install**:
```bash
# Test developer profile
REGISTRY_URL="file://$(pwd)/registry.json" ./install.sh --list
# Verify agent appears in profile
REGISTRY_URL="file://$(pwd)/registry.json" ./install.sh --list | grep "your-agent"
```
5. **Test actual install**:
```bash
# Install to temp directory
mkdir -p /tmp/test-install
cd /tmp/test-install
REGISTRY_URL="file://$(pwd)/registry.json" bash <(curl -s https://raw.githubusercontent.com/darrenhinde/OpenAgentsControl/main/install.sh) developer
# Check if agent was installed
ls .opencode/agent/category/your-agent.md
```
---
## Common Mistakes
### ❌ Mistake 1: Only Adding to Components
```json
// Added to components
"components": {
"agents": [
{"id": "new-agent", ...}
]
}
// But forgot to add to profiles
"profiles": {
"developer": {
"components": [
// Missing: "agent:new-agent"
]
}
}
```
### ❌ Mistake 2: Wrong Profile Assignment
```json
// Development agent added to business profile
"business": {
"components": [
"agent:devops-specialist" // ❌ Should be in developer
]
}
```
### ❌ Mistake 3: Inconsistent Profile Coverage
```json
// Added to full but not advanced
"full": {
"components": ["agent:new-agent"]
},
"advanced": {
"components": [
// ❌ Missing: "agent:new-agent"
]
}
```
---
## Best Practices
✅ **Use auto-detect** - Adds to components automatically
✅ **Check all profiles** - Verify agent in correct profiles
✅ **Test locally** - Install and verify before pushing
✅ **Validate** - Run validation script after changes
✅ **Document** - Update CHANGELOG with profile changes
---
## CI/CD Integration
Add profile validation to CI:
```yaml
# .github/workflows/validate-registry.yml
- name: Validate Registry
run: ./scripts/registry/validate-registry.sh
- name: Validate Profile Coverage
run: ./scripts/registry/validate-profile-coverage.sh
```
---
## Quick Reference
| Agent Category | Essential | Developer | Business | Full | Advanced |
|---------------|-----------|-----------|----------|------|----------|
| core | ✅ | ✅ | ✅ | ✅ | ✅ |
| development* | ❌ | ✅ | ❌ | ✅ | ✅ |
| content | ❌ | ❌ | ✅ | ✅ | ✅ |
| data | ❌ | ❌ | ✅ | ✅ | ✅ |
| meta | ❌ | ❌ | ❌ | ❌ | ✅ |
*Note: Development category includes agents (opencoder) and specialist subagents (frontend, devops)
---
## Development Profile Changes (v2.0.0)
**What Changed**:
- frontend-specialist: Agent → Subagent (specialized executor)
- devops-specialist: Agent → Subagent (specialized executor)
- backend-specialist: Removed (functionality covered by opencoder)
- codebase-pattern-analyst: Removed (replaced by analyze-patterns command)
- analyze-patterns: New command for pattern analysis
**Why**:
- Streamlined main agents to 2 (openagent, opencoder)
- Specialist subagents provide focused expertise when needed
- Reduced cognitive load for new users
- Clearer separation between main agents and specialized tools
**Impact**:
- Developer profile now has 2 main agents + 8 subagents
- Smaller, more focused profile
- Same capabilities, better organization
- No breaking changes for existing workflows
---
## Related Files
- **Registry concepts**: `core-concepts/registry.md`
- **Updating registry**: `guides/updating-registry.md`
- **Adding agents**: `guides/adding-agent.md`
---
**Last Updated**: 2025-01-28
**Version**: 0.5.2
@@ -1,57 +0,0 @@
# Guide: Resolving Installer Wildcard Failures
**Purpose**: Capture the root cause, fix, and lessons from wildcard context install failures.
**Last Updated**: 2026-01-12
---
## Prerequisites
- Installer changes scoped to `install.sh`
- Registry entries validated (`./scripts/registry/validate-registry.sh`)
**Estimated time**: 10 min
## Steps
### 1. Identify the failure mode
**Symptom**:
```
curl: (3) URL rejected: Malformed input to a URL function
```
**Cause**: Wildcard expansion returned context IDs that weren’t path-aligned (e.g., `standards-code` mapped to `.opencode/context/core/standards/code-quality.md`). Installer treated IDs as paths.
### 2. Expand wildcards to path-based IDs
**Goal**: Make wildcard expansion output `core/...` IDs that map directly to a path.
**Update**:
- Expand `context:core/*` to `core/standards/code-quality` style IDs
### 3. Resolve context paths deterministically
**Goal**: Avoid ambiguous matches and ensure one registry entry is used.
**Update**:
- Add `resolve_component_path` to map context IDs to the registry path
- Use `first(...)` in jq queries for deterministic selection
### 4. Verify installation
```bash
bash scripts/tests/test-e2e-install.sh
```
**Expected**: All E2E tests pass on macOS and Ubuntu.
## Verification
```bash
REGISTRY_URL="file://$(pwd)/registry.json" ./install.sh --list
```
## Troubleshooting
| Issue | Solution |
|-------|----------|
| `Malformed input to a URL function` | Ensure wildcard expansion returns `core/...` IDs and uses `resolve_component_path` |
| Multiple context entries for one path | Use `first(...)` in jq lookups |
## Related
- guides/debugging.md
- guides/updating-registry.md
- core-concepts/registry.md
@@ -1,371 +0,0 @@
# Guide: Subagent Invocation
**Purpose**: How to correctly invoke subagents using the task tool
**Priority**: HIGH - Critical for agent delegation
---
## The Problem
**Issue**: Agents trying to invoke subagents with incorrect `subagent_type` format
**Error**:
```
Unknown agent type: ContextScout is not a valid agent type
```
**Root Cause**: The `subagent_type` parameter in the task tool must match the registered agent type in the OpenCode CLI, not the file path.
---
## Correct Subagent Invocation
### Available Subagent Types
Based on the OpenCode CLI registration, use these exact strings for `subagent_type`:
**Core Subagents**:
- `"Task Manager"` - Task breakdown and planning
- `"Documentation"` - Documentation generation
- `"ContextScout"` - Context file discovery
**Code Subagents**:
- `"Coder Agent"` - Code implementation
- `"TestEngineer"` - Test authoring
- `"Reviewer"` - Code review
- `"Build Agent"` - Build validation
**System Builder Subagents**:
- `"Domain Analyzer"` - Domain analysis
- `"Agent Generator"` - Agent generation
- `"Context Organizer"` - Context organization
- `"Workflow Designer"` - Workflow design
- `"Command Creator"` - Command creation
**Utility Subagents**:
- `"Image Specialist"` - Image generation/editing
---
## Invocation Syntax
### ✅ Correct Format
```javascript
task(
subagent_type="Task Manager",
description="Break down feature into subtasks",
prompt="Detailed instructions..."
)
```
### ❌ Incorrect Formats
```javascript
// ❌ Using file path
task(
subagent_type="TaskManager",
...
)
// ❌ Using kebab-case ID
task(
subagent_type="task-manager",
...
)
// ❌ Using registry path
task(
subagent_type=".opencode/agent/TaskManager.md",
...
)
```
---
## How to Find the Correct Type
### Method 1: Check Registry
```bash
# List all subagent names
cat registry.json | jq -r '.components.subagents[] | "\(.name)"'
```
**Output**:
```
Task Manager
Image Specialist
Reviewer
TestEngineer
Documentation Writer
Coder Agent
Build Agent
Domain Analyzer
Agent Generator
Context Organizer
Workflow Designer
Command Creator
ContextScout
```
### Method 2: Check OpenCode CLI
```bash
# List available agents (if CLI supports it)
opencode list agents
```
### Method 3: Check Agent Frontmatter
Look at the `name` field in the subagent's frontmatter:
```yaml
---
id: task-manager
name: Task Manager # ← Use this for subagent_type
type: subagent
---
```
---
## Common Subagent Invocations
### Task Manager
```javascript
task(
subagent_type="Task Manager",
description="Break down complex feature",
prompt="Break down the following feature into atomic subtasks:
Feature: {feature description}
Requirements:
- {requirement 1}
- {requirement 2}
Create subtask files in tasks/subtasks/{feature}/"
)
```
### Documentation
```javascript
task(
subagent_type="Documentation",
description="Update documentation for feature",
prompt="Update documentation for {feature}:
What changed:
- {change 1}
- {change 2}
Files to update:
- {doc 1}
- {doc 2}"
)
```
### TestEngineer
```javascript
task(
subagent_type="TestEngineer",
description="Write tests for feature",
prompt="Write comprehensive tests for {feature}:
Files to test:
- {file 1}
- {file 2}
Test coverage:
- Positive cases
- Negative cases
- Edge cases"
)
```
### Reviewer
```javascript
task(
subagent_type="Reviewer",
description="Review implementation",
prompt="Review the following implementation:
Files:
- {file 1}
- {file 2}
Focus areas:
- Security
- Performance
- Code quality"
)
```
### Coder Agent
```javascript
task(
subagent_type="Coder Agent",
description="Implement subtask",
prompt="Implement the following subtask:
Subtask: {subtask description}
Files to create/modify:
- {file 1}
Requirements:
- {requirement 1}
- {requirement 2}"
)
```
---
## ContextScout Special Case
**Status**: ⚠️ May not be registered in OpenCode CLI yet
The `ContextScout` subagent exists in the repository but may not be registered in the OpenCode CLI's available agent types.
### Workaround
Until ContextScout is properly registered, use direct file operations instead:
```javascript
// ❌ This may fail
task(
subagent_type="ContextScout",
description="Find context files",
prompt="Search for context related to {topic}"
)
// ✅ Use direct operations instead
// 1. Use glob to find context files
glob(pattern="**/*.md", path=".opencode/context")
// 2. Use grep to search content
grep(pattern="registry", path=".opencode/context")
// 3. Read relevant files directly
read(filePath=".opencode/context/openagents-repo/core-concepts/registry.md")
```
---
## Fixing Existing Agents
### Agents That Need Fixing
1. **repo-manager.md** - Uses `ContextScout`
2. **opencoder.md** - Check if uses incorrect format
### Fix Process
1. **Find incorrect invocations**:
```bash
grep -r 'subagent_type="subagents/' .opencode/agent --include="*.md"
```
2. **Replace with correct format**:
```bash
# Example: Fix task-manager invocation
# Old: subagent_type="TaskManager"
# New: subagent_type="Task Manager"
```
3. **Test the fix**:
```bash
# Run agent with test prompt
# Verify subagent delegation works
```
---
## Validation
### Check Subagent Type Before Using
```javascript
// Pseudo-code for validation
available_types = [
"Task Manager",
"Documentation",
"TestEngineer",
"Reviewer",
"Coder Agent",
"Build Agent",
"Image Specialist",
"Domain Analyzer",
"Agent Generator",
"Context Organizer",
"Workflow Designer",
"Command Creator"
]
if subagent_type not in available_types:
error("Invalid subagent type: {subagent_type}")
```
---
## Best Practices
✅ **Use exact names** - Match registry `name` field exactly
✅ **Check registry first** - Verify subagent exists before using
✅ **Test invocations** - Test delegation before committing
✅ **Document dependencies** - List required subagents in agent frontmatter
❌ **Don't use paths** - Never use file paths as subagent_type
❌ **Don't use IDs** - Don't use kebab-case IDs
❌ **Don't assume** - Always verify subagent is registered
---
## Troubleshooting
### Error: "Unknown agent type"
**Cause**: Subagent type not registered in CLI or incorrect format
**Solutions**:
1. Check registry for correct name
2. Verify subagent exists in `.opencode/agent/subagents/`
3. Use exact name from registry `name` field
4. If subagent not registered, use direct operations instead
### Error: "Subagent not found"
**Cause**: Subagent file doesn't exist
**Solutions**:
1. Check file exists at expected path
2. Verify registry entry is correct
3. Run `./scripts/registry/validate-registry.sh`
### Delegation Fails Silently
**Cause**: Subagent invoked but doesn't execute
**Solutions**:
1. Check subagent has required tools enabled
2. Verify subagent permissions allow operation
3. Check subagent prompt is clear and actionable
---
## Related Files
- **Registry**: `registry.json` - Component catalog
- **Subagents**: `.opencode/agent/subagents/` - Subagent definitions
- **Validation**: `scripts/registry/validate-registry.sh`
---
**Last Updated**: 2025-12-29
**Version**: 0.5.1
@@ -1,303 +0,0 @@
# Guide: Testing an Agent
**Prerequisites**: Load `core-concepts/evals.md` first
**Purpose**: Step-by-step workflow for testing agents
---
## Quick Start
```bash
# Run smoke test
cd evals/framework
npm run eval:sdk -- --agent={category}/{agent} --pattern="smoke-test.yaml"
# Run all tests for agent
npm run eval:sdk -- --agent={category}/{agent}
# Run with debug
npm run eval:sdk -- --agent={category}/{agent} --debug
```
---
## Test Types
### 1. Smoke Test
**Purpose**: Basic functionality check
```yaml
name: Smoke Test
description: Verify agent responds correctly
agent: {category}/{agent}
model: anthropic/claude-sonnet-4-5
conversation:
- role: user
content: "Hello, can you help me?"
expectations:
- type: no_violations
```
**Run**:
```bash
npm run eval:sdk -- --agent={agent} --pattern="smoke-test.yaml"
```
---
### 2. Approval Gate Test
**Purpose**: Verify agent requests approval
```yaml
name: Approval Gate Test
description: Verify agent requests approval before execution
agent: {category}/{agent}
model: anthropic/claude-sonnet-4-5
conversation:
- role: user
content: "Create a new file called test.js"
expectations:
- type: specific_evaluator
evaluator: approval_gate
should_pass: true
```
---
### 3. Context Loading Test
**Purpose**: Verify agent loads required context
```yaml
name: Context Loading Test
description: Verify agent loads required context
agent: {category}/{agent}
model: anthropic/claude-sonnet-4-5
conversation:
- role: user
content: "Write a new function"
expectations:
- type: context_loaded
contexts: ["core/standards/code-quality.md"]
```
---
### 4. Tool Usage Test
**Purpose**: Verify agent uses correct tools
```yaml
name: Tool Usage Test
description: Verify agent uses appropriate tools
agent: {category}/{agent}
model: anthropic/claude-sonnet-4-5
conversation:
- role: user
content: "Read the package.json file"
expectations:
- type: tool_usage
tools: ["read"]
min_count: 1
```
---
## Running Tests
### Single Test
```bash
cd evals/framework
npm run eval:sdk -- --agent={category}/{agent} --pattern="{test-name}.yaml"
```
### All Tests for Agent
```bash
cd evals/framework
npm run eval:sdk -- --agent={category}/{agent}
```
### All Tests (All Agents)
```bash
cd evals/framework
npm run eval:sdk
```
### With Debug Output
```bash
cd evals/framework
npm run eval:sdk -- --agent={agent} --pattern="{test}" --debug
```
---
## Interpreting Results
### Pass Example
```
✓ Test: smoke-test.yaml
Status: PASS
Duration: 5.2s
Evaluators:
✓ Approval Gate: PASS
✓ Context Loading: PASS
✓ Tool Usage: PASS
✓ Stop on Failure: PASS
✓ Execution Balance: PASS
```
### Fail Example
```
✗ Test: approval-gate.yaml
Status: FAIL
Duration: 4.8s
Evaluators:
✗ Approval Gate: FAIL
Violation: Agent executed write tool without requesting approval
Location: Message #3, Tool call #1
✓ Context Loading: PASS
✓ Tool Usage: PASS
```
---
## Debugging Failures
### Step 1: Run with Debug
```bash
npm run eval:sdk -- --agent={agent} --pattern="{test}" --debug
```
### Step 2: Check Session
```bash
# Find recent session
ls -lt .tmp/sessions/ | head -5
# View session
cat .tmp/sessions/{session-id}/session.json | jq
```
### Step 3: Analyze Events
```bash
# View event timeline
cat .tmp/sessions/{session-id}/events.json | jq
```
### Step 4: Identify Issue
Common issues:
- **Approval Gate Violation**: Agent executed without approval
- **Context Loading Violation**: Agent didn't load required context
- **Tool Usage Violation**: Agent used wrong tool (bash instead of read)
- **Stop on Failure Violation**: Agent auto-fixed instead of stopping
### Step 5: Fix Agent
Update agent prompt to address the issue, then re-test.
---
## Writing New Tests
### Test Template
```yaml
name: Test Name
description: What this test validates
agent: {category}/{agent}
model: anthropic/claude-sonnet-4-5
conversation:
- role: user
content: "User message"
- role: assistant
content: "Expected response (optional)"
expectations:
- type: no_violations
```
### Best Practices
✅ **Clear name** - Descriptive test name
✅ **Good description** - Explain what's being tested
✅ **Realistic scenario** - Test real-world usage
✅ **Specific expectations** - Clear pass/fail criteria
✅ **Fast execution** - Keep under 10 seconds
---
## Common Test Patterns
### Test Approval Workflow
```yaml
conversation:
- role: user
content: "Create a new file"
expectations:
- type: specific_evaluator
evaluator: approval_gate
should_pass: true
```
### Test Context Loading
```yaml
conversation:
- role: user
content: "Write new code"
expectations:
- type: context_loaded
contexts: ["core/standards/code-quality.md"]
```
### Test Tool Selection
```yaml
conversation:
- role: user
content: "Read the README file"
expectations:
- type: tool_usage
tools: ["read"]
min_count: 1
```
---
## Continuous Testing
### Pre-Commit Hook
```bash
# Setup pre-commit hook
./scripts/validation/setup-pre-commit-hook.sh
```
### CI/CD Integration
Tests run automatically on:
- Pull requests
- Merges to main
- Release tags
---
## Related Files
- **Eval concepts**: `core-concepts/evals.md`
- **Debugging guide**: `guides/debugging.md`
- **Adding agents**: `guides/adding-agent.md`
---
**Last Updated**: 2025-12-10
**Version**: 0.5.0
@@ -1,137 +0,0 @@
---
description: "Guide for testing subagents and handling approval gates"
type: "context"
category: "openagents-repo"
tags: [testing, subagents, approval-gates]
---
# Testing Subagents: Approval Gates
**Context**: openagents-repo/guides | **Priority**: HIGH | **Updated**: 2026-01-09
---
## Critical Rule: Subagents Don't Need Approval Gates
**IMPORTANT**: When writing tests for subagents, DO NOT include `expectedViolations` for `approval-gate`.
### Why?
Subagents are **delegated to** by parent agents (OpenAgent, OpenCoder, etc.). The parent agent already requested and received approval before delegating. Therefore:
- ✅ Subagents can execute tools directly without asking for approval
- ✅ Subagents inherit approval from their parent
- ❌ Subagents should NOT be tested for approval gate violations
### Test Configuration for Subagents
**Correct** (no approval gate expectations):
```yaml
category: developer
agent: ContextScout
approvalStrategy:
type: auto-approve
behavior:
mustUseTools:
- read
- glob
forbiddenTools:
- write
- edit
minToolCalls: 2
maxToolCalls: 15
# NO expectedViolations for approval-gate!
```
**Incorrect** (don't do this):
```yaml
expectedViolations:
- rule: approval-gate # ❌ WRONG for subagents
shouldViolate: false
severity: error
```
---
## When to Test Approval Gates
**Test approval gates for**:
- ✅ Primary agents (OpenAgent, OpenCoder, System Builder)
- ✅ Category agents (frontend-specialist, data-analyst, etc.)
**Don't test approval gates for**:
- ❌ Subagents (contextscout, tester, reviewer, coder-agent, etc.)
- ❌ Any agent with `mode: subagent` in frontmatter
---
## Approval Strategy for Subagents
Always use `auto-approve` for subagent tests:
```yaml
approvalStrategy:
type: auto-approve
```
This simulates the parent agent having already approved the delegation.
---
## Example: ContextScout Test
```yaml
id: contextscout-code-standards
name: "ContextScout: Code Standards Discovery"
description: Tests that ContextScout discovers code-related context files
category: developer
agent: ContextScout
prompts:
- text: |
Search for context files related to: coding standards
Task type: code
Return:
- Exact file paths
- Priority order
- Key findings
approvalStrategy:
type: auto-approve
behavior:
mustUseTools:
- read
- glob
forbiddenTools:
- write
- edit
minToolCalls: 2
maxToolCalls: 15
timeout: 60000
tags:
- contextscout
- discovery
- subagent
```
---
## Related Files
- **Testing subagents**: `.opencode/context/openagents-repo/guides/testing-subagents.md`
- **Subagent invocation**: `.opencode/context/openagents-repo/guides/subagent-invocation.md`
- **Agent concepts**: `.opencode/context/openagents-repo/core-concepts/agents.md`
---
**Last Updated**: 2026-01-09
**Version**: 1.0.0
@@ -1,282 +0,0 @@
# Testing Subagents - Step-by-Step Guide
**Purpose**: How to test subagents in standalone mode
**Last Updated**: 2026-01-09
---
## ⚠️ CRITICAL: Adding New Subagent to Framework
**Before testing**, you MUST update THREE locations in framework code:
### 1. `evals/framework/src/sdk/run-sdk-tests.ts` (~line 336)
Add to `subagentParentMap`:
```typescript
'contextscout': 'openagent', // Maps subagent → parent
```
### 2. `evals/framework/src/sdk/run-sdk-tests.ts` (~line 414)
Add to `subagentPathMap`:
```typescript
'contextscout': 'ContextScout', // Maps name → path
```
### 3. `evals/framework/src/sdk/test-runner.ts` (~line 238)
Add to `agentMap`:
```typescript
'contextscout': 'ContextScout.md', // Maps name → file
```
**If missing from ANY map**: Tests will fail with "No test files found" or "Unknown subagent"
---
## Quick Start
```bash
# Test subagent directly (standalone mode)
cd evals/framework
npm run eval:sdk -- --subagent=contextscout --pattern="01-test.yaml"
# Test via delegation (integration mode)
npm run eval:sdk -- --subagent=contextscout --delegate --pattern="01-test.yaml"
# Debug mode
npm run eval:sdk -- --subagent=contextscout --pattern="01-test.yaml" --debug
```
---
## Step 1: Verify Agent File
**Check agent exists and has correct structure**:
```bash
# Check agent file
cat .opencode/agent/ContextScout.md | head -20
# Verify frontmatter
grep -A 5 "^id:" .opencode/agent/ContextScout.md
```
**Expected**:
```yaml
id: contextscout
name: ContextScout
category: subagents/core
type: subagent
mode: subagent # ← Will be forced to 'primary' in standalone tests
```
---
## Step 2: Verify Test Configuration
**Check test config points to correct agent**:
```bash
cat evals/agents/ContextScout/config/config.yaml
```
**Expected**:
```yaml
agent: ContextScout # ← Full path
model: anthropic/claude-sonnet-4-5
timeout: 60000
```
---
## Step 3: Run Standalone Test
**Use `--subagent` flag** (not `--agent`):
```bash
cd evals/framework
npm run eval:sdk -- --subagent=ContextScout --pattern="standalone/01-simple-discovery.yaml"
```
**What to Look For**:
```
⚡ Standalone Test Mode
Subagent: contextscout
Mode: Forced to 'primary' for direct testing
Testing agent: contextscout # ← Should show subagent name
```
---
## Step 4: Verify Agent Loaded Correctly
**Check test results**:
```bash
# View latest results
cat evals/results/latest.json | jq '.meta'
```
**Expected**:
```json
{
"agent": "ContextScout", // ← Correct agent
"model": "opencode/grok-code-fast",
"timestamp": "2026-01-07T..."
}
```
**Red Flags**:
- `"agent": "core/openagent"` ← Wrong! OpenAgent is running instead
- `"agent": "contextscout"` ← Missing category prefix
---
## Step 5: Check Tool Usage
**Verify subagent used tools**:
```bash
# Check tool calls in output
cat evals/results/latest.json | jq '.tests[0]' | grep -A 5 "Tool Calls"
```
**Expected** (for ContextScout):
```
Tool Calls: 1
Tools Used: glob
Tool Call Details:
1. glob: {"pattern":"*.md","path":".opencode/context/core"}
```
**Red Flags**:
- `Tool Calls: 0` ← Agent didn't use any tools
- `Tools Used: task` ← Parent agent delegating (wrong mode)
---
## Step 6: Analyze Failures
**If test fails, check violations**:
```bash
cat evals/results/latest.json | jq '.tests[0].violations'
```
**Common Issues**:
### Issue 1: No Tool Calls
```json
{
"type": "missing-required-tool",
"message": "Required tool 'glob' was not used"
}
```
**Cause**: Agent prompt doesn't emphasize tool usage
**Fix**: Add critical rules section emphasizing tools (see `examples/subagent-prompt-structure.md`)
### Issue 2: Wrong Agent Running
```
Agent: OpenAgent
```
**Cause**: Used `--agent` instead of `--subagent`
**Fix**: Use `--subagent=ContextScout`
### Issue 3: Tool Permission Denied
```json
{
"type": "missing-approval",
"message": "Execution tool 'bash' called without requesting approval"
}
```
**Cause**: Agent tried to use restricted tool
**Fix**: See `errors/tool-permission-errors.md`
---
## Step 7: Validate Results
**Check test passed**:
```bash
# View summary
cat evals/results/latest.json | jq '.summary'
```
**Expected**:
```json
{
"total": 1,
"passed": 1, // ← Should be 1
"failed": 0,
"pass_rate": 1.0
}
```
---
## Test File Organization
**Best Practice**: Organize by mode
```
evals/agents/ContextScout/tests/
├── standalone/ # Unit tests (--subagent flag)
│ ├── 01-simple-discovery.yaml
│ ├── 02-search-test.yaml
│ └── 03-extraction-test.yaml
└── delegation/ # Integration tests (--agent flag)
├── 01-openagent-delegates.yaml
└── 02-context-loading.yaml
```
---
## Writing Good Test Prompts
**Be explicit about tool usage**:
❌ **Vague** (may not work):
```yaml
prompts:
- text: |
List all markdown files in .opencode/context/core/
```
✅ **Explicit** (works):
```yaml
prompts:
- text: |
Use the glob tool to find all markdown files in .opencode/context/core/
You MUST use the glob tool like this:
glob(pattern="*.md", path=".opencode/context/core")
Then list the files you found.
```
---
## Quick Troubleshooting
| Symptom | Cause | Fix |
|---------|-------|-----|
| OpenAgent runs instead | Used `--agent` flag | Use `--subagent` flag |
| Tool calls: 0 | Prompt doesn't emphasize tools | Add critical rules section |
| Permission denied | Tool restricted in frontmatter | Check `tools:` and `permissions:` |
| Test timeout | Agent stuck/looping | Check prompt logic, add timeout |
---
## Related
- `concepts/subagent-testing-modes.md` - Understand standalone vs delegation
- `lookup/subagent-test-commands.md` - Quick command reference
- `errors/tool-permission-errors.md` - Common permission issues
- `examples/subagent-prompt-structure.md` - Optimized prompt structure
**Reference**: `evals/framework/src/sdk/run-sdk-tests.ts`
@@ -1,481 +0,0 @@
# Guide: Updating Registry
**Prerequisites**: Load `core-concepts/registry.md` first
**Purpose**: How to update the component registry
---
## Quick Commands
```bash
# Auto-detect and add new components
./scripts/registry/auto-detect-components.sh --auto-add
# Validate registry
./scripts/registry/validate-registry.sh
# Dry run (see what would change)
./scripts/registry/auto-detect-components.sh --dry-run
```
---
## When to Update Registry
Update the registry when you:
- ✅ Add a new agent
- ✅ Add a new command
- ✅ Add a new tool
- ✅ Add a new context file
- ✅ Change component metadata
- ✅ Move or rename components
---
## Auto-Detect (Recommended)
### Step 1: Dry Run
```bash
# See what would be added/updated
./scripts/registry/auto-detect-components.sh --dry-run
```
**Output**:
```
Scanning .opencode/ for components...
Would add:
- agent: development/api-specialist
- context: development/api-patterns.md
Would update:
- agent: core/openagent (description changed)
```
### Step 2: Apply Changes
```bash
# Actually update registry
./scripts/registry/auto-detect-components.sh --auto-add
```
### Step 3: Validate
```bash
# Validate registry
./scripts/registry/validate-registry.sh
```
---
## Frontmatter Metadata (Auto-Extracted)
The auto-detect script automatically extracts `tags` and `dependencies` from component frontmatter. This is the **recommended way** to add metadata.
### Supported Formats
**Multi-line arrays** (recommended for readability):
```yaml
---
description: Your component description
tags:
- tag1
- tag2
- tag3
dependencies:
- subagent:coder-agent
- context:core/standards/code
- command:context
---
```
**Inline arrays** (compact format):
```yaml
---
description: Your component description
tags: [tag1, tag2, tag3]
dependencies: [subagent:coder-agent, context:core/standards/code]
---
```
### Component-Specific Examples
**Command** (`.opencode/command/your-command.md`):
```yaml
---
description: Brief description of what this command does
tags:
- category
- feature
- use-case
dependencies:
- subagent:context-organizer
- subagent:contextscout
---
```
**Subagent** (`.opencode/agent/subagents/category/your-agent.md`):
```yaml
---
id: your-agent
name: Your Agent Name
description: What this agent does
category: specialist
type: specialist
tags:
- domain
- capability
dependencies:
- subagent:coder-agent
- context:core/standards/code
---
```
**Context** (`.opencode/context/category/your-context.md`):
```yaml
---
description: What knowledge this context provides
tags:
- domain
- topic
dependencies:
- context:core/standards/code
---
```
### Dependency Format
Dependencies use the format: `type:id`
**Valid types**:
- `subagent:` - References a subagent (e.g., `subagent:coder-agent`)
- `command:` - References a command (e.g., `command:context`)
- `context:` - References a context file (e.g., `context:core/standards/code`)
- `agent:` - References a main agent (e.g., `agent:openagent`)
**Examples**:
```yaml
dependencies:
- subagent:coder-agent # Depends on coder-agent subagent
- context:core/standards/code # Requires code standards context
- command:context # Uses context command
```
### How It Works
1. **Create component** with frontmatter (tags + dependencies)
2. **Run auto-detect**: `./scripts/registry/auto-detect-components.sh --dry-run`
3. **Verify extraction**: Check that tags/dependencies appear in output
4. **Apply changes**: `./scripts/registry/auto-detect-components.sh --auto-add`
5. **Validate**: `./scripts/registry/validate-registry.sh`
The script automatically:
- ✅ Extracts `description`, `tags`, `dependencies` from frontmatter
- ✅ Handles both inline and multi-line array formats
- ✅ Converts to proper JSON arrays in registry
- ✅ Validates dependency references exist
---
## Manual Updates (Not Recommended)
Only edit `registry.json` manually if auto-detect doesn't work.
**Prefer frontmatter**: Add tags/dependencies to component frontmatter instead of editing registry directly.
### Adding Component Manually
```json
{
"id": "agent-name",
"name": "Agent Name",
"type": "agent",
"path": ".opencode/agent/category/agent-name.md",
"description": "Brief description",
"category": "category",
"tags": ["tag1", "tag2"],
"dependencies": [],
"version": "0.5.0"
}
```
### Validate After Manual Edit
```bash
./scripts/registry/validate-registry.sh
```
---
## Validation
### What Gets Validated
✅ **Schema** - Correct JSON structure
✅ **Paths** - All paths exist
✅ **IDs** - Unique IDs
✅ **Categories** - Valid categories
✅ **Dependencies** - Dependencies exist
### Validation Errors
```bash
# Example errors
ERROR: Path does not exist: .opencode/agent/core/missing.md
ERROR: Duplicate ID: frontend-specialist
ERROR: Invalid category: invalid-category
ERROR: Missing dependency: subagent:nonexistent
```
### Fixing Errors
1. **Path not found**: Fix path or remove entry
2. **Duplicate ID**: Rename one component
3. **Invalid category**: Use valid category
4. **Missing dependency**: Add dependency or remove reference
---
## Testing Registry Changes
### Test Locally
```bash
# Test with local registry
REGISTRY_URL="file://$(pwd)/registry.json" ./install.sh --list
# Try installing a component
REGISTRY_URL="file://$(pwd)/registry.json" ./install.sh --component agent:your-agent
```
### Verify Component Appears
```bash
# List all agents
cat registry.json | jq '.components.agents[].id'
# Check specific component
cat registry.json | jq '.components.agents[] | select(.id == "your-agent")'
```
---
## Common Tasks
### Add New Component to Registry
```bash
# 1. Create component file with frontmatter (including tags/dependencies)
# 2. Run auto-detect
./scripts/registry/auto-detect-components.sh --auto-add
# 3. Validate
./scripts/registry/validate-registry.sh
```
**Example**: Adding a new command with tags/dependencies:
```bash
# 1. Create .opencode/command/my-command.md with frontmatter:
cat > .opencode/command/my-command.md << 'EOF'
---
description: My custom command description
tags: [automation, workflow]
dependencies: [subagent:coder-agent]
---
# My Command
...
EOF
# 2. Auto-detect extracts metadata
./scripts/registry/auto-detect-components.sh --dry-run
# 3. Apply changes
./scripts/registry/auto-detect-components.sh --auto-add
# 4. Validate
./scripts/registry/validate-registry.sh
```
### Update Component Metadata
```bash
# 1. Update frontmatter in component file (tags, dependencies, description)
# 2. Run auto-detect
./scripts/registry/auto-detect-components.sh --auto-add
# 3. Validate
./scripts/registry/validate-registry.sh
```
**Example**: Adding tags to existing component:
```bash
# 1. Edit .opencode/command/existing-command.md frontmatter:
# Add or update:
# tags: [new-tag, another-tag]
# dependencies: [subagent:new-dependency]
# 2. Auto-detect picks up changes
./scripts/registry/auto-detect-components.sh --dry-run
# 3. Apply
./scripts/registry/auto-detect-components.sh --auto-add
```
### Remove Component
```bash
# 1. Delete component file
# 2. Run auto-detect (will remove from registry)
./scripts/registry/auto-detect-components.sh --auto-add
# 3. Validate
./scripts/registry/validate-registry.sh
```
---
## CI/CD Integration
### Automatic Validation
Registry is validated on:
- Pull requests (`.github/workflows/validate-registry.yml`)
- Merges to main
- Release tags
### Auto-Update on Merge
Registry can be auto-updated after merge:
```yaml
# .github/workflows/update-registry.yml
- name: Update Registry
run: ./scripts/registry/auto-detect-components.sh --auto-add
```
---
## Best Practices
✅ **Use frontmatter** - Add tags/dependencies to component files, not registry
✅ **Use auto-detect** - Don't manually edit registry
✅ **Validate often** - Catch issues early
✅ **Test locally** - Use local registry for testing
✅ **Dry run first** - See changes before applying
✅ **Version consistency** - Keep versions in sync
✅ **Multi-line arrays** - More readable than inline format
✅ **Meaningful tags** - Use descriptive, searchable tags
✅ **Declare dependencies** - Helps with component discovery and validation
---
## Related Files
- **Registry concepts**: `core-concepts/registry.md`
- **Adding agents**: `guides/adding-agent.md`
- **Debugging**: `guides/debugging.md`
---
## Troubleshooting
### Tags/Dependencies Not Extracted
**Problem**: Auto-detect doesn't extract tags or dependencies from frontmatter.
**Solutions**:
1. **Check frontmatter format**:
- Must be at top of file
- Must start/end with `---`
- Must use valid YAML syntax
2. **Verify array format**:
```yaml
# ✅ Valid formats
tags: [tag1, tag2]
tags:
- tag1
- tag2
# ❌ Invalid
tags: tag1, tag2 # Missing brackets
```
3. **Check dependency format**:
```yaml
# ✅ Valid
dependencies: [subagent:coder-agent, context:core/standards/code]
# ❌ Invalid
dependencies: [coder-agent] # Missing type prefix
```
4. **Run dry-run to debug**:
```bash
./scripts/registry/auto-detect-components.sh --dry-run
# Check output shows extracted tags/dependencies
```
### Dependency Validation Errors
**Problem**: Validation fails with "Missing dependency" error.
**Solution**: Ensure referenced component exists in registry:
```bash
# Check if dependency exists
jq '.components.subagents[] | select(.id == "coder-agent")' registry.json
# If missing, add the dependency component first
```
### Context Not Found (Aliases)
**Problem**: Error `Could not find path for context:old-name` even though file exists.
**Cause**: The context file might have been renamed or the ID in registry doesn't match the requested name.
**Solution**: Add an alias to the component in `registry.json`.
1. Find the component in `registry.json`
2. Add `"aliases": ["old-name", "alternative-name"]`
3. Validate registry
---
## Managing Aliases
Aliases allow components to be referenced by multiple names. This is useful for:
- Backward compatibility (renamed files)
- Shorthand references
- Alternative naming conventions
### Adding Aliases
Currently, aliases must be added **manually** to `registry.json` (auto-detect does not yet support them).
```json
{
"id": "session-management",
"name": "Session Management",
"type": "context",
"path": ".opencode/context/core/workflows/session-management.md",
"aliases": [
"workflows-sessions",
"sessions"
],
...
}
```
**Note**: Always validate the registry after manual edits:
```bash
./scripts/registry/validate-registry.sh
```
---
**Last Updated**: 2025-01-06
**Version**: 2.0.0
@@ -1,400 +0,0 @@
# Lookup: Command Reference
**Purpose**: Quick reference for common commands
---
## Registry Commands
### Validate Registry
```bash
# Basic validation
./scripts/registry/validate-registry.sh
# Verbose output
./scripts/registry/validate-registry.sh -v
```
### Auto-Detect Components
```bash
# Dry run (see what would change)
./scripts/registry/auto-detect-components.sh --dry-run
# Add new components
./scripts/registry/auto-detect-components.sh --auto-add
# Force update existing
./scripts/registry/auto-detect-components.sh --auto-add --force
```
### Validate Component Structure
```bash
./scripts/registry/validate-component.sh
```
---
## Testing Commands
### Run Tests
```bash
# Single test
cd evals/framework
npm run eval:sdk -- --agent={category}/{agent} --pattern="{test}.yaml"
# All tests for agent
npm run eval:sdk -- --agent={category}/{agent}
# All tests (all agents)
npm run eval:sdk
# With debug
npm run eval:sdk -- --agent={agent} --debug
```
### Validate Test Suites
```bash
./scripts/validation/validate-test-suites.sh
```
---
## Installation Commands
### Install Components
```bash
# List available components
./install.sh --list
# Install profile
./install.sh {profile}
# Profiles: essential, developer, business
# Install specific component
./install.sh --component agent:{agent-name}
# Test with local registry
REGISTRY_URL="file://$(pwd)/registry.json" ./install.sh --list
```
### Collision Handling
```bash
# Skip existing files
./install.sh developer --skip-existing
# Overwrite all
./install.sh developer --force
# Backup existing
./install.sh developer --backup
```
---
## Version Commands
### Check Version
```bash
# Check all version files
cat VERSION
cat package.json | jq '.version'
cat registry.json | jq '.version'
```
### Update Version
```bash
# Update VERSION
echo "0.X.Y" > VERSION
# Update package.json
jq '.version = "0.X.Y"' package.json > tmp && mv tmp package.json
# Update registry.json
jq '.version = "0.X.Y"' registry.json > tmp && mv tmp registry.json
```
### Bump Version Script
```bash
./scripts/versioning/bump-version.sh 0.X.Y
```
---
## Git Commands
### Create Release
```bash
# Commit version changes
git add VERSION package.json CHANGELOG.md
git commit -m "chore: bump version to 0.X.Y"
# Create tag
git tag -a v0.X.Y -m "Release v0.X.Y"
# Push
git push origin main
git push origin v0.X.Y
```
### Create GitHub Release
```bash
# Via GitHub CLI
gh release create v0.X.Y \
--title "v0.X.Y" \
--notes "See CHANGELOG.md for details"
```
---
## Validation Commands
### Full Validation
```bash
# Validate everything
./scripts/registry/validate-registry.sh && \
./scripts/validation/validate-test-suites.sh && \
cd evals/framework && npm run eval:sdk
```
### Check Context Dependencies
```bash
# Analyze all agents
/check-context-deps
# Analyze specific agent
/check-context-deps contextscout
# Auto-fix missing dependencies
/check-context-deps --fix
```
### Validate Context References
```bash
./scripts/validation/validate-context-refs.sh
```
### Setup Pre-Commit Hook
```bash
./scripts/validation/setup-pre-commit-hook.sh
```
---
## Development Commands
### Run Demo
```bash
./scripts/development/demo.sh
```
### Run Dashboard
```bash
./scripts/development/dashboard.sh
```
---
## Maintenance Commands
### Cleanup Stale Sessions
```bash
./scripts/maintenance/cleanup-stale-sessions.sh
```
### Uninstall
```bash
./scripts/maintenance/uninstall.sh
```
---
## Debugging Commands
### Check Sessions
```bash
# List recent sessions
ls -lt .tmp/sessions/ | head -5
# View session
cat .tmp/sessions/{session-id}/session.json | jq
# View events
cat .tmp/sessions/{session-id}/events.json | jq
```
### Check Context Logs
```bash
# Check session cache
./scripts/check-context-logs/check-session-cache.sh
# Count agent tokens
./scripts/check-context-logs/count-agent-tokens.sh
# Show API payload
./scripts/check-context-logs/show-api-payload.sh
# Show cached data
./scripts/check-context-logs/show-cached-data.sh
```
---
## Quick Workflows
### Adding a New Agent
```bash
# 1. Create agent file
touch .opencode/agent/{category}/{agent-name}.md
# (Add frontmatter and content)
# 2. Create test structure
mkdir -p evals/agents/{category}/{agent-name}/{config,tests}
# (Create config.yaml and smoke-test.yaml)
# 3. Update registry
./scripts/registry/auto-detect-components.sh --auto-add
# 4. Validate
./scripts/registry/validate-registry.sh
cd evals/framework && npm run eval:sdk -- --agent={category}/{agent-name}
```
### Testing an Agent
```bash
# 1. Run smoke test
cd evals/framework
npm run eval:sdk -- --agent={category}/{agent} --pattern="smoke-test.yaml"
# 2. If fails, debug
npm run eval:sdk -- --agent={category}/{agent} --debug
# 3. Check session
ls -lt .tmp/sessions/ | head -1
cat .tmp/sessions/{session-id}/session.json | jq
```
### Creating a Release
```bash
# 1. Update version
echo "0.X.Y" > VERSION
jq '.version = "0.X.Y"' package.json > tmp && mv tmp package.json
# 2. Update CHANGELOG
# (Edit CHANGELOG.md)
# 3. Commit and tag
git add VERSION package.json CHANGELOG.md
git commit -m "chore: bump version to 0.X.Y"
git tag -a v0.X.Y -m "Release v0.X.Y"
# 4. Push
git push origin main
git push origin v0.X.Y
# 5. Create GitHub release
gh release create v0.X.Y --title "v0.X.Y" --notes "See CHANGELOG.md"
```
---
## Common Patterns
### Find Files
```bash
# Find agent
find .opencode/agent -name "{agent-name}.md"
# Find tests
find evals/agents -name "*.yaml"
# Find context
find .opencode/context -name "*.md"
# Find scripts
find scripts -name "*.sh"
```
### Check Registry
```bash
# List all agents
cat registry.json | jq '.components.agents[].id'
# Check specific component
cat registry.json | jq '.components.agents[] | select(.id == "{agent-name}")'
# Count components
cat registry.json | jq '.components.agents | length'
```
### Test Locally
```bash
# Test with local registry
REGISTRY_URL="file://$(pwd)/registry.json" ./install.sh --list
# Install locally
REGISTRY_URL="file://$(pwd)/registry.json" ./install.sh developer
```
---
## NPM Commands (Eval Framework)
```bash
cd evals/framework
# Install dependencies
npm install
# Run tests
npm test
# Run eval SDK
npm run eval:sdk
# Build
npm run build
# Lint
npm run lint
```
---
## Related Files
- **Quick start**: `quick-start.md`
- **File locations**: `lookup/file-locations.md`
- **Guides**: `guides/`
---
**Last Updated**: 2025-12-10
**Version**: 0.5.0
@@ -1,314 +0,0 @@
# Lookup: File Locations
**Purpose**: Quick reference for finding files
---
## Directory Tree
```
opencode-agents/
├── .opencode/
│ ├── agent/
│ │ ├── core/ # Core system agents
│ │ ├── development/ # Dev specialists
│ │ ├── content/ # Content creators
│ │ ├── data/ # Data analysts
│ │ ├── product/ # Product managers (ready)
│ │ ├── learning/ # Educators (ready)
│ │ └── subagents/ # Delegated specialists
│ │ ├── code/ # Code-related
│ │ ├── core/ # Core workflows
│ │ ├── system-builder/ # System generation
│ │ └── utils/ # Utilities
│ ├── command/ # Slash commands
│ ├── context/ # Shared knowledge
│ │ ├── core/ # Core standards & workflows
│ │ ├── development/ # Dev context
│ │ ├── content-creation/ # Content creation context
│ │ ├── data/ # Data context
│ │ ├── product/ # Product context
│ │ ├── learning/ # Learning context
│ │ └── openagents-repo/ # Repo-specific context
│ ├── prompts/ # Model-specific variants
│ ├── tool/ # Custom tools
│ └── plugin/ # Plugins
├── evals/
│ ├── framework/ # Eval framework (TypeScript)
│ │ ├── src/ # Source code
│ │ ├── scripts/ # Test utilities
│ │ └── docs/ # Framework docs
│ └── agents/ # Agent test suites
│ ├── core/ # Core agent tests
│ ├── development/ # Dev agent tests
│ └── content/ # Content agent tests
├── scripts/
│ ├── registry/ # Registry management
│ ├── validation/ # Validation tools
│ ├── testing/ # Test utilities
│ ├── versioning/ # Version management
│ ├── docs/ # Doc tools
│ └── maintenance/ # Maintenance
├── docs/ # Documentation
│ ├── agents/ # Agent docs
│ ├── contributing/ # Contribution guides
│ ├── features/ # Feature docs
│ └── getting-started/ # User guides
├── registry.json # Component catalog
├── install.sh # Installer
├── VERSION # Current version
└── package.json # Node dependencies
```
---
## Where Is...?
| Component | Location |
|-----------|----------|
| **Core agents** | `.opencode/agent/core/` |
| **Category agents** | `.opencode/agent/{category}/` |
| **Subagents** | `.opencode/agent/subagents/` |
| **Commands** | `.opencode/command/` |
| **Context files** | `.opencode/context/` |
| **Prompt variants** | `.opencode/prompts/{category}/{agent}/` |
| **Tools** | `.opencode/tool/` |
| **Plugins** | `.opencode/plugin/` |
| **Agent tests** | `evals/agents/{category}/{agent}/` |
| **Eval framework** | `evals/framework/src/` |
| **Registry scripts** | `scripts/registry/` |
| **Validation scripts** | `scripts/validation/` |
| **Documentation** | `docs/` |
| **Registry** | `registry.json` |
| **Installer** | `install.sh` |
| **Version** | `VERSION` |
---
## Where Do I Add...?
| What | Where |
|------|-------|
| **New core agent** | `.opencode/agent/core/{name}.md` |
| **New category agent** | `.opencode/agent/{category}/{name}.md` |
| **New subagent** | `.opencode/agent/subagents/{category}/{name}.md` |
| **New command** | `.opencode/command/{name}.md` |
| **New context** | `.opencode/context/{category}/{name}.md` |
| **Agent tests** | `evals/agents/{category}/{agent}/tests/` |
| **Test config** | `evals/agents/{category}/{agent}/config/config.yaml` |
| **Documentation** | `docs/{section}/{topic}.md` |
| **Script** | `scripts/{purpose}/{name}.sh` |
---
## Specific File Paths
### Core Files
```
registry.json # Component catalog
install.sh # Main installer
update.sh # Update script
VERSION # Current version (0.5.0)
package.json # Node dependencies
CHANGELOG.md # Release notes
README.md # Main documentation
```
### Core Agents
```
.opencode/agent/core/openagent.md
.opencode/agent/core/opencoder.md
.opencode/agent/meta/system-builder.md
```
### Development Agents
```
.opencode/agent/development/frontend-specialist.md
.opencode/agent/development/devops-specialist.md
```
.opencode/agent/development/frontend-specialist.md
.opencode/agent/development/devops-specialist.md
```
### Content Agents
```
.opencode/agent/content/copywriter.md
.opencode/agent/content/technical-writer.md
```
### Key Subagents
```
.opencode/agent/TestEngineer.md
.opencode/agent/CodeReviewer.md
.opencode/agent/CoderAgent.md
.opencode/agent/TaskManager.md
.opencode/agent/DocWriter.md
```
### Core Context
```
.opencode/context/core/standards/code-quality.md
.opencode/context/core/standards/documentation.md
.opencode/context/core/standards/test-coverage.md
.opencode/context/core/standards/security-patterns.md
.opencode/context/core/workflows/task-delegation-basics.md
.opencode/context/core/workflows/code-review.md
```
### Registry Scripts
```
scripts/registry/validate-registry.sh
scripts/registry/auto-detect-components.sh
scripts/registry/register-component.sh
scripts/registry/validate-component.sh
```
### Validation Scripts
```
scripts/validation/validate-context-refs.sh
scripts/validation/validate-test-suites.sh
scripts/validation/setup-pre-commit-hook.sh
```
### Eval Framework
```
evals/framework/src/sdk/ # Test runner
evals/framework/src/evaluators/ # Rule evaluators
evals/framework/src/collector/ # Session collection
evals/framework/src/types/ # TypeScript types
```
---
## Path Patterns
### Agents
```
.opencode/agent/{category}/{agent-name}.md
```
**Examples**:
- `.opencode/agent/core/openagent.md`
- `.opencode/agent/development/frontend-specialist.md`
- `.opencode/agent/TestEngineer.md`
### Context
```
.opencode/context/{category}/{topic}.md
```
**Examples**:
- `.opencode/context/core/standards/code-quality.md`
- `.opencode/context/development/frontend/react/react-patterns.md`
- `.opencode/context/content-creation/principles/copywriting-frameworks.md`
### Tests
```
evals/agents/{category}/{agent-name}/
├── config/config.yaml
└── tests/{test-name}.yaml
```
**Examples**:
- `evals/agents/core/openagent/tests/smoke-test.yaml`
- `evals/agents/development/frontend-specialist/tests/approval-gate.yaml`
### Scripts
```
scripts/{purpose}/{action}-{target}.sh
```
**Examples**:
- `scripts/registry/validate-registry.sh`
- `scripts/validation/validate-test-suites.sh`
- `scripts/versioning/bump-version.sh`
---
## Naming Conventions
### Files
- **Agents**: `{name}.md` or `{domain}-specialist.md`
- **Context**: `{topic}.md`
- **Tests**: `{test-name}.yaml`
- **Scripts**: `{action}-{target}.sh`
- **Docs**: `{topic}.md`
### Directories
- **Categories**: lowercase, singular (e.g., `development`, `content`)
- **Purposes**: lowercase, descriptive (e.g., `registry`, `validation`)
---
## Quick Lookups
### Find Agent File
```bash
# By name
find .opencode/agent -name "{agent-name}.md"
# By category
ls .opencode/agent/{category}/
# All agents
find .opencode/agent -name "*.md" -not -path "*/subagents/*"
```
### Find Test File
```bash
# By agent
ls evals/agents/{category}/{agent}/tests/
# All tests
find evals/agents -name "*.yaml"
```
### Find Context File
```bash
# By category
ls .opencode/context/{category}/
# All context
find .opencode/context -name "*.md"
```
### Find Script
```bash
# By purpose
ls scripts/{purpose}/
# All scripts
find scripts -name "*.sh"
```
---
## Related Files
- **Quick start**: `quick-start.md`
- **Categories**: `core-concepts/categories.md`
- **Commands**: `lookup/commands.md`
---
**Last Updated**: 2025-12-10
**Version**: 0.5.0
@@ -1,38 +0,0 @@
# OpenAgents Lookup
**Purpose**: Quick reference and lookup tables for OpenAgents Control
---
## Structure
```
openagents-repo/lookup/
├── navigation.md (this file)
└── [lookup reference files]
```
---
## Quick Routes
| Task | Path |
|------|------|
| **View lookups** | `./` |
| **Guides** | `../guides/navigation.md` |
| **Core Concepts** | `../core-concepts/navigation.md` |
---
## By Type
**Quick Reference** → Fast lookup tables and commands
**Checklists** → Verification and validation checklists
---
## Related Context
- **OpenAgents Navigation** → `../navigation.md`
- **Guides** → `../guides/navigation.md`
- **Core Concepts** → `../core-concepts/navigation.md`
@@ -1,76 +0,0 @@
# Lookup: Subagent Framework Maps
**Purpose**: Quick reference for adding subagents to eval framework
**Last Updated**: 2026-01-09
---
## Critical: THREE Maps Must Be Updated
When adding a new subagent, update these THREE locations:
### 1. Parent Map (run-sdk-tests.ts ~line 336)
**Purpose**: Maps subagent → parent agent for delegation testing
```typescript
const subagentParentMap: Record<string, string> = {
'contextscout': 'openagent', // Core subagents → openagent
'task-manager': 'openagent',
'documentation': 'openagent',
'coder-agent': 'opencoder', // Code subagents → opencoder
'tester': 'opencoder',
'reviewer': 'opencoder',
};
```
### 2. Path Map (run-sdk-tests.ts ~line 414)
**Purpose**: Maps subagent name → file path for test discovery
```typescript
const subagentPathMap: Record<string, string> = {
'contextscout': 'ContextScout',
'task-manager': 'TaskManager',
'coder-agent': 'CoderAgent',
};
```
### 3. Agent Map (test-runner.ts ~line 238)
**Purpose**: Maps subagent name → agent file for eval-runner
```typescript
const agentMap: Record<string, string> = {
'contextscout': 'ContextScout.md',
'task-manager': 'TaskManager.md',
'coder-agent': 'CoderAgent.md',
};
```
---
## Error Messages
| Error | Missing From | Fix |
|-------|--------------|-----|
| "No test files found" | Path Map (#2) | Add to `subagentPathMap` |
| "Unknown subagent" | Parent Map (#1) | Add to `subagentParentMap` |
| "Agent file not found" | Agent Map (#3) | Add to `agentMap` |
---
## Testing Commands
```bash
# Standalone mode (forces mode: primary)
npm run eval:sdk -- --subagent=contextscout
# Delegation mode (tests via parent)
npm run eval:sdk -- --subagent=contextscout --delegate
```
---
## Related
- `guides/testing-subagents.md` - Full testing guide
- `guides/adding-agent.md` - Creating new agents
@@ -1,192 +0,0 @@
# Subagent Testing Commands - Quick Reference
**Purpose**: Quick command reference for testing subagents
**Last Updated**: 2026-01-07
---
## Standalone Mode (Unit Testing)
### Run All Standalone Tests
```bash
cd evals/framework
npm run eval:sdk -- --subagent=ContextScout --pattern="standalone/*.yaml"
```
### Run Single Test
```bash
npm run eval:sdk -- --subagent=ContextScout --pattern="standalone/01-simple-discovery.yaml"
```
### Debug Mode
```bash
npm run eval:sdk -- --subagent=ContextScout --pattern="standalone/*.yaml" --debug
```
---
## Delegation Mode (Integration Testing)
### Run Delegation Tests
```bash
npm run eval:sdk -- --agent=core/openagent --pattern="delegation/*.yaml"
```
### Test Specific Delegation
```bash
npm run eval:sdk -- --agent=core/openagent --pattern="delegation/01-contextscout-delegation.yaml"
```
---
## Verification Commands
### Check Agent File
```bash
# View agent frontmatter
head -30 .opencode/agent/ContextScout.md
# Check tool permissions
grep -A 10 "^tools:" .opencode/agent/ContextScout.md
```
### Check Test Config
```bash
cat evals/agents/ContextScout/config/config.yaml
```
### View Latest Results
```bash
# Summary
cat evals/results/latest.json | jq '.summary'
# Agent loaded
cat evals/results/latest.json | jq '.meta.agent'
# Tool calls
cat evals/results/latest.json | jq '.tests[0]' | grep -A 5 "Tool"
# Violations
cat evals/results/latest.json | jq '.tests[0].violations'
```
---
## Common Test Patterns
### Smoke Test
```bash
npm run eval:sdk -- --subagent=ContextScout --pattern="smoke-test.yaml"
```
### Specific Test Suite
```bash
npm run eval:sdk -- --subagent=ContextScout --pattern="discovery/*.yaml"
```
### All Tests for Subagent
```bash
npm run eval:sdk -- --subagent=ContextScout
```
---
## Flag Reference
| Flag | Purpose | Example |
|------|---------|---------|
| `--subagent` | Test subagent in standalone mode | `--subagent=ContextScout` |
| `--agent` | Test primary agent (or delegation) | `--agent=core/openagent` |
| `--pattern` | Filter test files | `--pattern="standalone/*.yaml"` |
| `--debug` | Show detailed output | `--debug` |
| `--timeout` | Override timeout | `--timeout=120000` |
---
## Troubleshooting Commands
### Check Which Agent Ran
```bash
# Should show subagent name for standalone mode
cat evals/results/latest.json | jq '.meta.agent'
```
### Check Tool Usage
```bash
# Should show tool calls > 0
cat evals/results/latest.json | jq '.tests[0]' | grep "Tool Calls"
```
### View Test Timeline
```bash
# See full conversation
cat evals/results/history/2026-01/07-*.json | jq '.tests[0].timeline'
```
### Check for Errors
```bash
# View violations
cat evals/results/latest.json | jq '.tests[0].violations.details'
```
---
## File Locations
### Agent Files
```
.opencode/agent/subagents/core/{subagent}.md
```
### Test Files
```
evals/agents/subagents/core/{subagent}/
├── config/config.yaml
└── tests/
├── standalone/
│ ├── 01-simple-discovery.yaml
│ └── 02-advanced-test.yaml
└── delegation/
└── 01-delegation-test.yaml
```
### Results
```
evals/results/
├── latest.json # Latest test run
└── history/2026-01/ # Historical results
└── 07-HHMMSS-{agent}.json
```
---
## Quick Checks
### Is Agent Loaded Correctly?
```bash
# Should show: "agent": "ContextScout"
cat evals/results/latest.json | jq '.meta.agent'
```
### Did Agent Use Tools?
```bash
# Should show: Tool Calls: 1 (or more)
cat evals/results/latest.json | jq '.tests[0]' | grep "Tool Calls"
```
### Did Test Pass?
```bash
# Should show: "passed": 1, "failed": 0
cat evals/results/latest.json | jq '.summary'
```
---
## Related
- `concepts/subagent-testing-modes.md` - Understand testing modes
- `guides/testing-subagents.md` - Step-by-step testing guide
- `errors/tool-permission-errors.md` - Fix common issues
**Reference**: `evals/framework/src/sdk/run-sdk-tests.ts`
@@ -1,187 +0,0 @@
# OpenAgents Control Repository Context
**Purpose**: Context files specific to the OpenAgents Control repository
**Last Updated**: 2026-02-04
---
## Quick Navigation
| Function | Files | Purpose |
|----------|-------|---------|
| **Standards** | 2 files | Agent creation standards |
| **Concepts** | 6 files | Core ideas and principles |
| **Examples** | 9 files | Working code samples |
| **Guides** | 14 files | Step-by-step workflows |
| **Lookup** | 11 files | Quick reference tables |
| **Errors** | 2 files | Common issues + solutions |
---
## Standards (Agent Creation)
| File | Topic | Priority |
|------|-------|----------|
| `standards/agent-frontmatter.md` | Valid OpenCode YAML frontmatter | ⭐⭐⭐⭐⭐ |
| `standards/subagent-structure.md` | Standard subagent file structure | ⭐⭐⭐⭐⭐ |
**When to read**: Before creating or modifying any agent files
---
## Concepts (Core Ideas)
| File | Topic | Priority |
|------|-------|----------|
| `concepts/compatibility-layer.md` | Adapter pattern for AI coding tools | ⭐⭐⭐⭐⭐ |
| `concepts/subagent-testing-modes.md` | Standalone vs delegation testing | ⭐⭐⭐⭐⭐ |
| `concepts/hooks-system.md` | User-defined lifecycle commands | ⭐⭐⭐⭐ |
| `concepts/agent-skills.md` | Skills that teach Claude tasks | ⭐⭐⭐⭐ |
| `concepts/subagents-system.md` | Specialized AI assistants | ⭐⭐⭐⭐ |
**When to read**: Before testing any subagent or working with tool adapters
---
## Examples (Working Code)
| File | Topic | Priority |
|------|-------|----------|
| `examples/baseadapter-pattern.md` | Template Method pattern for tool adapters | ⭐⭐⭐⭐⭐ |
| `examples/zod-schema-migration.md` | Migrating TypeScript to Zod schemas | ⭐⭐⭐⭐ |
| `examples/subagent-prompt-structure.md` | Optimized subagent prompt template | ⭐⭐⭐⭐ |
**When to read**: When creating adapters, schemas, or optimizing subagent prompts
---
## Guides (Step-by-Step)
| File | Topic | Priority |
|------|-------|----------|
| `guides/compatibility-layer-workflow.md` | Developing compatibility layer for AI tools | ⭐⭐⭐⭐⭐ |
| `guides/testing-subagents.md` | How to test subagents standalone | ⭐⭐⭐⭐⭐ |
| `guides/adding-agent-basics.md` | How to add new agents (basics) | ⭐⭐⭐⭐ |
| `guides/adding-agent-testing.md` | How to add agent tests | ⭐⭐⭐⭐ |
| `guides/adding-skill-basics.md` | How to add OpenCode skills | ⭐⭐⭐⭐ |
| `guides/creating-skills.md` | How to create Claude Code skills | ⭐⭐⭐⭐ |
| `guides/creating-subagents.md` | How to create Claude Code subagents | ⭐⭐⭐⭐ |
| `guides/testing-agent.md` | How to test agents | ⭐⭐⭐⭐ |
| `guides/external-libraries-workflow.md` | How to handle external library dependencies | ⭐⭐⭐⭐ |
| `guides/github-issues-workflow.md` | How to work with GitHub issues and project board | ⭐⭐⭐⭐ |
| `guides/npm-publishing.md` | How to publish package to npm | ⭐⭐⭐ |
| `guides/updating-registry.md` | How to update registry | ⭐⭐⭐ |
| `guides/debugging.md` | How to debug issues | ⭐⭐⭐ |
| `guides/resolving-installer-wildcard-failures.md` | Fix wildcard context install failures | ⭐⭐⭐ |
| `guides/creating-release.md` | How to create releases | ⭐⭐ |
**When to read**: When performing specific tasks
---
## Lookup (Quick Reference)
| File | Topic | Priority |
|------|-------|----------|
| `lookup/tool-feature-parity.md` | AI coding tool feature comparison | ⭐⭐⭐⭐⭐ |
| `lookup/compatibility-layer-structure.md` | Compatibility package file structure | ⭐⭐⭐⭐⭐ |
| `lookup/subagent-test-commands.md` | Subagent testing commands | ⭐⭐⭐⭐⭐ |
| `lookup/hook-events.md` | All hook events reference | ⭐⭐⭐⭐ |
| `lookup/skill-metadata.md` | SKILL.md frontmatter fields | ⭐⭐⭐⭐ |
| `lookup/skills-comparison.md` | Skills vs other options | ⭐⭐⭐⭐ |
| `lookup/builtin-subagents.md` | Default subagents (Explore, Plan) | ⭐⭐⭐⭐ |
| `lookup/subagent-frontmatter.md` | Subagent configuration fields | ⭐⭐⭐⭐ |
| `lookup/file-locations.md` | Where files are located | ⭐⭐⭐⭐ |
| `lookup/commands.md` | Available slash commands | ⭐⭐⭐ |
**When to read**: Quick command lookups and feature comparisons
---
## Errors (Troubleshooting)
| File | Topic | Priority |
|------|-------|----------|
| `errors/tool-permission-errors.md` | Tool permission issues | ⭐⭐⭐⭐⭐ |
| `errors/skills-errors.md` | Skills not triggering/loading | ⭐⭐⭐⭐ |
**When to read**: When tests fail with permission errors
---
## Core Concepts (Foundational)
| File | Topic | Priority |
|------|-------|----------|
| `core-concepts/agents.md` | How agents work | ⭐⭐⭐⭐⭐ |
| `core-concepts/evals.md` | How testing works | ⭐⭐⭐⭐⭐ |
| `core-concepts/registry.md` | How registry works | ⭐⭐⭐⭐ |
| `core-concepts/categories.md` | How organization works | ⭐⭐⭐ |
**When to read**: First time working in this repo
---
## Loading Strategy
### For Subagent Testing:
1. Load `concepts/subagent-testing-modes.md` (understand modes)
2. Load `guides/testing-subagents.md` (step-by-step)
3. Reference `lookup/subagent-test-commands.md` (commands)
4. If errors: Load `errors/tool-permission-errors.md`
### For Agent Creation:
1. Load `standards/agent-frontmatter.md` (valid YAML frontmatter)
2. Load `standards/subagent-structure.md` (file structure)
3. Load `core-concepts/agents.md` (understand system)
4. Load `guides/adding-agent-basics.md` (step-by-step)
5. **If using external libraries**: Load `guides/external-libraries-workflow.md` (fetch docs)
6. Load `examples/subagent-prompt-structure.md` (if subagent)
7. Load `guides/testing-agent.md` (validate)
### For Issue Management:
1. Load `guides/github-issues-workflow.md` (understand workflow)
2. Create issues with proper labels and templates
3. Add to project board for tracking
4. Process requests systematically
### For Debugging:
1. Load `guides/debugging.md` (general approach)
2. Load specific error file from `errors/`
3. Reference `lookup/file-locations.md` (find files)
---
## File Size Compliance
All files follow MVI principle (<200 lines):
- ✅ Standards: <200 lines
- ✅ Concepts: <100 lines
- ✅ Examples: <100 lines
- ✅ Guides: <150 lines
- ✅ Lookup: <100 lines
- ✅ Errors: <150 lines
---
## Related Context
- `../core/` - Core system context (standards, patterns)
- `../core/context-system/` - Context management system
- `quick-start.md` - 2-minute repo orientation
- `../content-creation/navigation.md` - Content creation principles
- `plugins/context/context-overview.md` - Plugin system context
---
## Contributing
When adding new context files:
1. Follow MVI principle (<200 lines)
2. Use function-based organization (concepts/, examples/, guides/, lookup/, errors/)
3. Update this README.md navigation
4. Add cross-references to related files
5. Validate with `/context validate`
@@ -1,38 +0,0 @@
# Plugin Lifecycle & Packaging
## File Structure for Complex Plugins
For larger plugins, follow this recommended structure:
```
my-plugin/
├── .claude-plugin/
│ └── plugin.json # Manifest (required for packaging)
├── commands/ # Custom slash commands
├── agents/ # Custom agents
├── hooks/ # Event handlers
└── README.md # Documentation
```
## The Manifest (`plugin.json`)
```json
{
"name": "my-plugin",
"description": "A custom plugin",
"version": "1.0.0",
"author": {
"name": "Your Name"
}
}
```
The `name` becomes the namespace prefix for commands: `/my-plugin:command`.
## SDK Access
Plugins have full access to the OpenCode SDK via `context.client`. This allows:
- Sending prompts programmatically: `client.session.prompt()`
- Managing sessions: `client.session.list()`, `client.session.get()`
- Showing UI elements: `client.tui.showToast()`
- Appending to prompt: `client.tui.appendPrompt()`
@@ -1,58 +0,0 @@
# OpenCode Plugins Overview
OpenCode plugins are JavaScript or TypeScript modules that hook into **25+ events** across the entire OpenCode lifecycle—from when you type a prompt, to when tools execute, to when sessions complete.
## Key Concepts
- **Zero-Config**: No build step or compilation required. Just drop `.ts` or `.js` files into the plugin folder.
- **Middleware Pattern**: Plugins subscribe to events and execute logic, similar to Express.js middleware.
- **Access**: Plugins receive a `context` object with:
- `project`: Current project metadata.
- `client`: OpenCode SDK client for programmatic control.
- `$`: Bun's shell API for running commands.
- `directory`: Current working directory.
- `worktree`: Git worktree path.
## Plugin Registration
OpenCode looks for plugins in:
1. **Project-level**: `.opencode/plugin/` (project root)
2. **Global**: `~/.config/opencode/plugin/` (home directory)
## Basic Structure
```typescript
export const MyPlugin = async (context) => {
const { project, client, $, directory, worktree } = context;
return {
event: async ({ event }) => {
// Handle events here
}
};
};
```
Each exported function becomes a separate plugin instance. The name of the export is used as the plugin name.
## Build and Development
OpenCode plugins are typically written in TypeScript and bundled into a single JavaScript file for execution.
### Build Command
Use Bun to bundle the plugin into the `dist` directory:
```bash
bun build src/index.ts --outdir dist --target bun --format esm
```
The output will be a single file (e.g., `./index.js`) containing all dependencies.
### Development Workflow
1. **Source Code**: Write your plugin in `src/index.ts`.
2. **Bundle**: Run the build command to generate `dist/index.js`.
3. **Load**: Point OpenCode to the bundled file or the directory containing the manifest.
4. **Watch Mode**: For rapid development, use the `--watch` flag with Bun build:
```bash
bun build src/index.ts --outdir dist --target bun --format esm --watch
```
@@ -1,44 +0,0 @@
# Custom Agents in OpenCode
Plugins can register custom AI agents that have specific roles, instructions, and toolsets.
## Agent Definition
Custom agents are configured in the plugin's `config` function.
```typescript
export const registerCustomAgents = (config) => {
return {
...config,
agents: [
{
name: "my-helper",
description: "A friendly assistant for this project",
instructions: "You are a helpful assistant. Use your tools to help the user.",
model: "claude-3-5-sonnet-latest", // Specify the model
tools: ["say_hello", "read", "write"] // Reference built-in or custom tools
}
]
};
};
```
## Integrating into Plugin
The `config` method in the plugin return object is used to register agents.
```typescript
export const MyPlugin: Plugin = async (context) => {
return {
config: async (currentConfig) => {
return registerCustomAgents(currentConfig);
},
// ... other properties
};
};
```
## Agent Capabilities
- **Model Choice**: You can select specific models for different agents.
- **Scoped Tools**: Limit what tools an agent can use to ensure safety or focus.
- **System Instructions**: Define the "personality" and rules for the agent.
@@ -1,42 +0,0 @@
# OpenCode Plugin Events
OpenCode fires over 25 events that you can hook into. These are categorized below:
## Command Events
- `command.executed`: Fired when a user or plugin runs a command.
## File Events
- `file.edited`: Fired when a file is modified via OpenCode tools.
- `file.watcher.updated`: Fired when the file watcher detects changes.
## Message Events (Read-Only)
- `message.updated`: Fired when a message in the session updates.
- `message.part.updated`: Fired when individual parts of a message update.
- `message.part.removed`: Fired when a part is removed.
- `message.removed`: Fired when entire message is removed.
## Session Events
- `session.created`: New session started.
- `session.updated`: Session state changed.
- `session.idle`: Session completed (no more activity expected).
- `session.status`: Session status changed.
- `session.error`: Error occurred in session.
- `session.compacted`: Session was compacted (context summarized).
## Tool Events (Interception)
- `tool.execute.before`: Fired before a tool runs. **Can block execution** by throwing an error.
- `tool.execute.after`: Fired after a tool completes with result.
## TUI Events
- `tui.prompt.append`: Text appended to prompt input.
- `tui.command.execute`: Command executed from TUI.
- `tui.toast.show`: Toast notification shown.
## Mapping from Claude Code Hooks
| Claude Hook | OpenCode Event |
|---|---|
| PreToolUse | tool.execute.before |
| PostToolUse | tool.execute.after |
| UserPromptSubmit | message.* events |
| SessionEnd | session.idle |
@@ -1,596 +0,0 @@
# OpenCode Events: Skills Plugin Implementation
## Overview
This document explains how the OpenCode Skills Plugin uses event hooks (`tool.execute.before` and `tool.execute.after`) to implement skill delivery and output enhancement. This is a practical example of the event system described in `events.md`.
---
## Event Hooks Used
### tool.execute.before
**Event Type:** Tool Execution Interception
**When it fires:** Before a tool function executes
**Purpose in Skills Plugin:** Inject skill content into the conversation
**Implementation:**
```typescript
const beforeHook = async (input: any, output: any) => {
// Check if this is a skill tool
if (input.tool.startsWith("skills_")) {
// Look up skill from map
const skill = skillMap.get(input.tool)
if (skill) {
// Inject skill content as silent prompt
await ctx.client.session.prompt({
path: { id: input.sessionID },
body: {
agent: input.agent,
noReply: true, // Don't trigger AI response
parts: [
{
type: "text",
text: `📚 Skill: ${skill.name}\nBase directory: ${skill.fullPath}\n\n${skill.content}`,
},
],
},
})
}
}
}
```
**Why use this hook?**
- Runs before tool execution, perfect for context injection
- Can access tool name and session ID
- Can inject content without triggering AI response
- Skill content persists in conversation history
**Input Parameters:**
- `input.tool` - Tool name (e.g., "skills_brand_guidelines")
- `input.sessionID` - Current session ID
- `input.agent` - Agent name that called the tool
- `output.args` - Tool arguments
**What you can do:**
- ✅ Inject context (skill content)
- ✅ Validate inputs
- ✅ Preprocess arguments
- ✅ Log tool calls
- ✅ Implement security checks
**What you can't do:**
- ❌ Modify tool output (tool hasn't run yet)
- ❌ Access tool results
---
### tool.execute.after
**Event Type:** Tool Execution Interception
**When it fires:** After a tool function completes
**Purpose in Skills Plugin:** Enhance output with visual feedback
**Implementation:**
```typescript
const afterHook = async (input: any, output: any) => {
// Check if this is a skill tool
if (input.tool.startsWith("skills_")) {
// Look up skill from map
const skill = skillMap.get(input.tool)
if (skill && output.output) {
// Add emoji title for visual feedback
output.title = `📚 ${skill.name}`
}
}
}
```
**Why use this hook?**
- Runs after tool execution, perfect for output enhancement
- Can modify output properties
- Can add visual feedback (emoji titles)
- Can implement logging/analytics
**Input Parameters:**
- `input.tool` - Tool name (e.g., "skills_brand_guidelines")
- `input.sessionID` - Current session ID
- `output.output` - Tool result/output
- `output.title` - Output title (can be modified)
**What you can do:**
- ✅ Modify output
- ✅ Add titles/formatting
- ✅ Log completion
- ✅ Add analytics
- ✅ Transform results
**What you can't do:**
- ❌ Modify tool arguments (already executed)
- ❌ Prevent tool execution (already happened)
---
## Event Lifecycle in Skills Plugin
```
┌─────────────────────────────────────────────────────────────────┐
│ AGENT CALLS SKILL TOOL │
│ (e.g., skills_brand_guidelines) │
└─────────────────────────────────────────────────────────────────┘
↓
┌─────────────────────────────────────────────────────────────────┐
│ EVENT: tool.execute.before fires │
│ │
│ Hook Function: beforeHook(input, output) │
│ ┌─────────────────────────────────────────────────────────┐ │
│ │ 1. Check: input.tool.startsWith("skills_") │ │
│ │ 2. Lookup: skillMap.get(input.tool) │ │
│ │ 3. Inject: ctx.client.session.prompt({ │ │
│ │ path: { id: input.sessionID }, │ │
│ │ body: { │ │
│ │ agent: input.agent, │ │
│ │ noReply: true, │ │
│ │ parts: [{ type: "text", text: skill.content }] │ │
│ │ } │ │
│ │ }) │ │
│ │ 4. Result: Skill content added to conversation │ │
│ └─────────────────────────────────────────────────────────┘ │
│ │
│ Effect: Skill content persists in conversation history │
│ No AI response triggered (noReply: true) │
└─────────────────────────────────────────────────────────────────┘
↓
┌─────────────────────────────────────────────────────────────────┐
│ TOOL.EXECUTE() RUNS │
│ │
│ async execute(args, toolCtx) { │
│ return `Skill activated: ${skill.name}` │
│ } │
│ │
│ Effect: Minimal confirmation returned │
└─────────────────────────────────────────────────────────────────┘
↓
┌─────────────────────────────────────────────────────────────────┐
│ EVENT: tool.execute.after fires │
│ │
│ Hook Function: afterHook(input, output) │
│ ┌─────────────────────────────────────────────────────────┐ │
│ │ 1. Check: input.tool.startsWith("skills_") │ │
│ │ 2. Lookup: skillMap.get(input.tool) │ │
│ │ 3. Verify: output.output exists │ │
│ │ 4. Enhance: output.title = `📚 ${skill.name}` │ │
│ │ 5. Result: Output title modified with emoji │ │
│ └─────────────────────────────────────────────────────────┘ │
│ │
│ Effect: Visual feedback added to output │
│ Could add logging/analytics here │
└─────────────────────────────────────────────────────────────────┘
↓
┌─────────────────────────────────────────────────────────────────┐
│ RESULT RETURNED TO AGENT │
│ │
│ - Tool confirmation message │
│ - Skill content in conversation history │
│ - Enhanced output with emoji title │
│ - Agent can now use skill content in reasoning │
└─────────────────────────────────────────────────────────────────┘
```
---
## Why Hooks Instead of Embedded Logic?
### Problem: Embedded Delivery (Anti-Pattern)
```typescript
// ❌ OLD: Skill delivery inside tool.execute()
async execute(args, toolCtx) {
const sendSilentPrompt = (text: string) =>
ctx.client.session.prompt({...})
await sendSilentPrompt(`The "${skill.name}" skill is loading...`)
await sendSilentPrompt(`Base directory: ${skill.fullPath}\n\n${skill.content}`)
return `Launching skill: ${skill.name}`
}
```
**Issues:**
1. **Tight Coupling**: Tool logic and delivery are inseparable
2. **Hard to Test**: Can't test tool without testing delivery
3. **Violates SOLID**: Single Responsibility Principle broken
4. **No Reusability**: Delivery logic can't be extracted
5. **Difficult to Monitor**: Can't track delivery separately
---
### Solution: Hook-Based Delivery (Best Practice)
```typescript
// ✅ NEW: Separated concerns using hooks
// Tool: Minimal and focused
async execute(args, toolCtx) {
return `Skill activated: ${skill.name}`
}
// Hook: Handles delivery
const beforeHook = async (input, output) => {
if (input.tool.startsWith("skills_")) {
const skill = skillMap.get(input.tool)
if (skill) {
await ctx.client.session.prompt({...})
}
}
}
```
**Benefits:**
1. ✅ **Loose Coupling**: Tool and delivery are independent
2. ✅ **Easy to Test**: Each component tested separately
3. ✅ **SOLID Compliant**: Single Responsibility Principle
4. ✅ **Reusable**: Hooks can be composed with other plugins
5. ✅ **Monitorable**: Can add logging/analytics independently
---
## Skill Lookup Map: Performance Optimization
### Why a Map?
The skill lookup map enables O(1) access instead of O(n) search:
```typescript
// ✅ EFFICIENT: O(1) lookup
const skillMap = new Map<string, Skill>()
for (const skill of skills) {
skillMap.set(skill.toolName, skill)
}
const beforeHook = async (input, output) => {
if (input.tool.startsWith("skills_")) {
const skill = skillMap.get(input.tool) // O(1) constant time
if (skill) {
// Use skill
}
}
}
```
### Performance Impact
| Number of Skills | Array Search (O(n)) | Map Lookup (O(1)) | Speedup |
|------------------|-------------------|------------------|---------|
| 10 | 10 comparisons | 1 lookup | 10x |
| 100 | 100 comparisons | 1 lookup | 100x |
| 1000 | 1000 comparisons | 1 lookup | 1000x |
**Conclusion:** Map lookup is essential for scalability
---
## Integration with OpenCode Event System
### Event Mapping
| OpenCode Event | Skills Plugin Hook | Purpose |
|---|---|---|
| `tool.execute.before` | `beforeHook` | Skill content injection |
| `tool.execute.after` | `afterHook` | Output enhancement |
### Plugin Return Object
```typescript
return {
// Custom tools
tool: tools,
// Hook: Runs before tool execution
"tool.execute.before": beforeHook,
// Hook: Runs after tool execution
"tool.execute.after": afterHook,
}
```
**Key Points:**
- Hooks apply to ALL tools (use `if` statements to filter)
- Multiple plugins can register hooks without conflict
- Hooks run in registration order
- Hooks can be async
---
## Comparison with Other Event Hooks
### Available Tool Execution Hooks
| Hook | When | Use Case |
|------|------|----------|
| `tool.execute.before` | Before tool runs | Input validation, context injection, preprocessing |
| `tool.execute.after` | After tool completes | Output formatting, logging, analytics |
### Other Event Hooks (Not Used in Skills Plugin)
| Hook | When | Use Case |
|------|------|----------|
| `session.created` | Session starts | Welcome messages, initialization |
| `message.updated` | Message changes | Monitoring, logging |
| `session.idle` | Session completes | Cleanup, background tasks |
| `session.error` | Error occurs | Error handling, logging |
---
## Real-World Example: Skill Delivery Flow
### Step 1: Agent Calls Skill Tool
```
Agent: "Use the brand-guidelines skill"
↓
OpenCode: Calls skills_brand_guidelines tool
```
### Step 2: Before Hook Fires
```typescript
const beforeHook = async (input, output) => {
// input.tool = "skills_brand_guidelines"
// input.sessionID = "ses_abc123"
// input.agent = "my-helper"
if (input.tool.startsWith("skills_")) {
const skill = skillMap.get("skills_brand_guidelines")
// skill = {
// name: "brand-guidelines",
// description: "Brand guidelines for the project",
// content: "# Brand Guidelines\n\n...",
// fullPath: "/path/to/skill"
// }
await ctx.client.session.prompt({
path: { id: "ses_abc123" },
body: {
agent: "my-helper",
noReply: true,
parts: [
{
type: "text",
text: "📚 Skill: brand-guidelines\nBase directory: /path/to/skill\n\n# Brand Guidelines\n\n..."
}
]
}
})
}
}
```
**Result:** Skill content added to conversation, no AI response
### Step 3: Tool Executes
```typescript
async execute(args, toolCtx) {
// Minimal logic
return `Skill activated: brand-guidelines`
}
```
**Result:** Simple confirmation returned
### Step 4: After Hook Fires
```typescript
const afterHook = async (input, output) => {
// input.tool = "skills_brand_guidelines"
// output.output = "Skill activated: brand-guidelines"
if (input.tool.startsWith("skills_")) {
const skill = skillMap.get("skills_brand_guidelines")
if (skill && output.output) {
output.title = `📚 brand-guidelines`
}
}
}
```
**Result:** Output title enhanced with emoji
### Step 5: Agent Receives Result
```
Conversation History:
├─ User: "Use the brand-guidelines skill"
├─ Tool Call: skills_brand_guidelines
├─ Silent Message: "📚 Skill: brand-guidelines\n..."
├─ Tool Result: "Skill activated: brand-guidelines"
│ (with title: "📚 brand-guidelines")
└─ Agent: "I now have the brand guidelines. I can help with..."
```
---
## Testing Hooks
### Testing Before Hook
```typescript
describe("beforeHook", () => {
it("should inject skill content for skill tools", async () => {
const input = {
tool: "skills_brand_guidelines",
sessionID: "ses_test",
agent: "test-agent"
}
const output = { args: {} }
const mockPrompt = jest.fn()
ctx.client.session.prompt = mockPrompt
await beforeHook(input, output)
expect(mockPrompt).toHaveBeenCalledWith(
expect.objectContaining({
path: { id: "ses_test" },
body: expect.objectContaining({
agent: "test-agent",
noReply: true,
parts: expect.arrayContaining([
expect.objectContaining({
type: "text",
text: expect.stringContaining("brand-guidelines")
})
])
})
})
)
})
it("should skip non-skill tools", async () => {
const input = { tool: "read_file", sessionID: "ses_test" }
const output = { args: {} }
const mockPrompt = jest.fn()
ctx.client.session.prompt = mockPrompt
await beforeHook(input, output)
expect(mockPrompt).not.toHaveBeenCalled()
})
})
```
### Testing After Hook
```typescript
describe("afterHook", () => {
it("should add emoji title for skill tools", async () => {
const input = { tool: "skills_brand_guidelines" }
const output = { output: "Skill activated" }
await afterHook(input, output)
expect(output.title).toBe("📚 brand-guidelines")
})
it("should skip non-skill tools", async () => {
const input = { tool: "read_file" }
const output = { output: "File content" }
await afterHook(input, output)
expect(output.title).toBeUndefined()
})
it("should skip if output is missing", async () => {
const input = { tool: "skills_brand_guidelines" }
const output = { output: null }
await afterHook(input, output)
expect(output.title).toBeUndefined()
})
})
```
---
## Common Patterns
### Pattern 1: Tool-Specific Hooks
```typescript
const beforeHook = async (input, output) => {
switch (input.tool) {
case "skills_brand_guidelines":
// Handle brand guidelines
break
case "skills_api_reference":
// Handle API reference
break
default:
// Skip non-skill tools
}
}
```
### Pattern 2: Conditional Processing
```typescript
const beforeHook = async (input, output) => {
if (input.tool.startsWith("skills_")) {
const skill = skillMap.get(input.tool)
if (skill && skill.allowedTools?.includes(input.agent)) {
// Process only if allowed
}
}
}
```
### Pattern 3: Logging & Monitoring
```typescript
const beforeHook = async (input, output) => {
if (input.tool.startsWith("skills_")) {
console.log(`[BEFORE] Skill tool called: ${input.tool}`)
console.log(`[BEFORE] Session: ${input.sessionID}`)
}
}
const afterHook = async (input, output) => {
if (input.tool.startsWith("skills_")) {
console.log(`[AFTER] Skill tool completed: ${input.tool}`)
console.log(`[AFTER] Output length: ${output.output?.length || 0}`)
}
}
```
### Pattern 4: Error Handling
```typescript
const beforeHook = async (input, output) => {
try {
if (input.tool.startsWith("skills_")) {
const skill = skillMap.get(input.tool)
if (!skill) {
throw new Error(`Skill not found: ${input.tool}`)
}
// Process skill
}
} catch (error) {
console.error(`Hook error:`, error)
// Don't rethrow - let tool execute anyway
}
}
```
---
## Key Takeaways
1. **Hooks are middleware**: They intercept tool execution at specific points
2. **Before hook**: For preprocessing, validation, context injection
3. **After hook**: For output enhancement, logging, analytics
4. **Lookup maps**: Enable O(1) access instead of O(n) search
5. **Separation of concerns**: Tools do one thing, hooks do another
6. **Composability**: Multiple plugins can register hooks without conflict
7. **Testability**: Each component can be tested independently
8. **Maintainability**: Changes are isolated to specific hooks
---
## References
- **OpenCode Events**: `context/capabilities/events.md`
- **Tool Definition**: `context/capabilities/tools.md`
- **Best Practices**: `context/reference/best-practices.md`
- **Skills Plugin Example**: `skills-plugin/example.ts`
- **Hook Lifecycle**: `skills-plugin/hook-lifecycle-and-patterns.md`
- **Implementation Pattern**: `skills-plugin/implementation-pattern.md`
@@ -1,51 +0,0 @@
# Building Custom Tools
Plugins can add custom tools that OpenCode agents can call autonomously.
## Tool Definition
Custom tools use Zod for schema definition and the `tool` helper from `@opencode-ai/plugin`.
```typescript
import { z } from 'zod';
import { tool } from '@opencode-ai/plugin';
export const MyCustomTool = tool(
z.object({
query: z.string().describe('Search query'),
limit: z.number().default(10).describe('Results limit')
}),
async (args, context) => {
const { query, limit } = args;
// Implementation logic
return { success: true, data: [] };
}
).describe('Search your database');
```
## Shell-based Tools
You can leverage Bun's shell API (`$`) to run commands in any language.
```typescript
export const PythonCalculatorTool = tool(
z.object({ expression: z.string() }),
async (args, context) => {
const { $ } = context;
const result = await $`python3 -c 'print(eval("${args.expression}"))'`;
return { result: result.stdout };
}
).describe('Calculate mathematical expressions');
```
## Integration
To register tools in your plugin:
```typescript
export const MyPlugin = async (context) => {
return {
tool: [MyCustomTool, PythonCalculatorTool]
};
};
```
@@ -1,34 +0,0 @@
# OpenCode Plugin Context Library
This library provides structured context for AI coding assistants to understand, build, and extend OpenCode plugins. Depending on your task, you can load specific parts of this library.
## 📚 Library Map
### 🏗️ Architecture
Foundational concepts of how plugins are registered and executed.
- [Overview](./architecture/overview.md): Basic structure, registration, and context object.
- [Lifecycle](./architecture/lifecycle.md): Packaging, manifest, and session lifecycle.
### 🛠️ Capabilities
Deep dives into specific plugin features.
- [Events](./capabilities/events.md): Detailed list of all 25+ hookable events.
- [Events: Skills Plugin](./capabilities/events_skills.md): Practical example of event hooks in the Skills Plugin.
- [Tools](./capabilities/tools.md): How to build and register custom tools using Zod.
- [Agents](./capabilities/agents.md): Creating and configuring custom AI agents.
### 📖 Reference
Guidelines and troubleshooting.
- [Best Practices](./reference/best-practices.md): Message injection workarounds, security, and performance.
### 🧩 Claude Code Plugins (External)
Claude Code plugin system documentation (harvested from external docs).
- [Concepts: Plugin Architecture](./concepts/plugin-architecture.md): Core concepts and structure
- [Guides: Creating Plugins](./guides/creating-plugins.md): Step-by-step creation
- [Guides: Migrating to Plugins](./guides/migrating-to-plugins.md): Convert standalone to plugin
- [Lookup: Plugin Structure](./lookup/plugin-structure.md): Directory reference
## 🚀 How to use this library
If you are asking an AI to build a new feature:
1. **For a new tool**: Provide `architecture/overview.md` and `capabilities/tools.md`.
2. **For reacting to events**: Provide `capabilities/events.md`.
3. **For overall plugin architecture**: Provide `architecture/overview.md` and `architecture/lifecycle.md`.
@@ -1,26 +0,0 @@
# Best Practices & Limitations
## Message Injection Workarounds
**The Reality**: The message system is largely read-only. You cannot mutate messages mid-stream or inject text directly into an existing message part.
### What Doesn't Work
- Modifying `event.data.content` in `message.updated`.
- Retroactively changing AI responses.
### What Works
1. **Initial Context**: Use `session.created` to inject a starting message using `client.session.prompt()`.
2. **Prompt Decoration**: Use `client.tui.appendPrompt()` to add text to the user's input box before they hit enter.
3. **Tool Interception**: Use `tool.execute.before` to modify arguments *before* the tool runs.
4. **On-Demand Context**: Provide custom tools that the AI can call when it needs more information.
## Security
- Always validate tool inputs in `tool.execute.before`.
- Use environment variables for sensitive data; do not hardcode API keys.
- Be careful with the `$` shell API to prevent command injection.
## Performance
- Avoid heavy synchronous operations in event handlers as they can block the TUI.
- Use the `session.idle` event for cleanup or background sync tasks.
@@ -1,40 +0,0 @@
# OpenAgents Plugins
**Purpose**: Plugin architecture and documentation for OpenAgents Control
---
## Structure
```
openagents-repo/plugins/
├── navigation.md (this file)
├── context/
│ └── [context plugin files]
└── [plugin files]
```
---
## Quick Routes
| Task | Path |
|------|------|
| **Context plugin** | `./context/` |
| **View plugins** | `./` |
| **Guides** | `../guides/navigation.md` |
---
## By Type
**Context Plugin** → Context system plugin documentation
**Plugin Architecture** → How plugins work in OpenAgents
---
## Related Context
- **OpenAgents Navigation** → `../navigation.md`
- **Guides** → `../guides/navigation.md`
- **Core Concepts** → `../core-concepts/navigation.md`
@@ -1,39 +0,0 @@
# OpenAgents Quality
**Purpose**: Quality assurance and standards for OpenAgents Control
---
## Structure
```
openagents-repo/quality/
├── navigation.md (this file)
└── [quality documentation]
```
---
## Quick Routes
| Task | Path |
|------|------|
| **View quality docs** | `./` |
| **Guides** | `../guides/navigation.md` |
| **Errors** | `../errors/navigation.md` |
---
## By Type
**Quality Standards** → Standards for code quality
**Testing** → Testing requirements and patterns
**Validation** → Validation procedures
---
## Related Context
- **OpenAgents Navigation** → `../navigation.md`
- **Guides** → `../guides/navigation.md`
- **Errors** → `../errors/navigation.md`
@@ -1,586 +0,0 @@
---
description: Maintain registry quality through dependency validation and consistency checks
tags:
- registry
- quality
- validation
- dependencies
dependencies: []
---
<!-- Context: quality/registry-dependencies | Priority: high | Version: 1.0 | Updated: 2026-01-06 -->
# Registry Dependency Validation
**Purpose**: Maintain registry quality through dependency validation and consistency checks
**Audience**: Contributors, maintainers, CI/CD processes
---
## Quick Reference
**Golden Rule**: All component dependencies must be declared in frontmatter and validated before commits.
**Critical Commands**:
```bash
# Check context file dependencies
/check-context-deps
# Auto-fix missing dependencies
/check-context-deps --fix
# Validate entire registry
./scripts/registry/validate-registry.sh
# Update registry after changes
./scripts/registry/auto-detect-components.sh --auto-add
```
---
## Dependency System
### Dependency Types
Components can depend on other components using the `type:id` format:
| Type | Format | Example | Description |
|------|--------|---------|-------------|
| **agent** | `agent:id` | `agent:opencoder` | Core agent profile |
| **subagent** | `subagent:id` | `subagent:coder-agent` | Delegatable subagent |
| **command** | `command:id` | `command:context` | Slash command |
| **tool** | `tool:id` | `tool:gemini` | External tool integration |
| **plugin** | `plugin:id` | `plugin:context` | Plugin component |
| **context** | `context:path` | `context:core/standards/code` | Context file |
| **config** | `config:id` | `config:defaults` | Configuration file |
### Declaring Dependencies
**In component frontmatter** (example):
```
id: opencoder
name: OpenCoder
description: Multi-language implementation agent
dependencies:
- subagent:task-manager # Can delegate to task-manager
- subagent:coder-agent # Can delegate to coder-agent
- subagent:tester # Can delegate to tester
- context:core/standards/code # Requires code standards context
```
**Why declare dependencies?**
- ✅ **Validation**: Catch missing components before runtime
- ✅ **Documentation**: Clear visibility of what each component needs
- ✅ **Installation**: Installers can fetch all required dependencies
- ✅ **Dependency graphs**: Visualize component relationships
- ✅ **Breaking change detection**: Know what's affected by changes
---
## Context File Dependencies
### The Problem
Agents reference context files in their prompts but often don't declare them as dependencies:
```markdown
<!-- In agent prompt -->
BEFORE any code implementation, ALWAYS load:
- Code tasks → .opencode/context/core/standards/code-quality.md (MANDATORY)
```
**Without dependency declaration**:
- ❌ No validation that context file exists
- ❌ Can't track which agents use which context files
- ❌ Breaking changes when context files are moved/deleted
- ❌ Installers don't know to fetch context files
### The Solution
**Declare context dependencies in frontmatter** (example):
```
id: opencoder
dependencies:
- context:core/standards/code # ← ADD THIS
```
**Use `/check-context-deps` to find missing declarations**:
```bash
# Analyze all agents
/check-context-deps
# Auto-fix missing context dependencies
/check-context-deps --fix
```
### Context Dependency Format
**Path normalization**:
```
File path: .opencode/context/core/standards/code-quality.md
Dependency: context:core/standards/code
^^^^^^^ ^^^^^^^^^^^^^^^^^^^
type path (no .opencode/, no .md)
```
**Examples**:
```
dependencies:
- context:core/standards/code # .opencode/context/core/standards/code-quality.md
- context:core/standards/docs # .opencode/context/core/standards/documentation.md
- context:core/workflows/delegation # .opencode/context/core/workflows/task-delegation-basics.md
- context:openagents-repo/guides/adding-agent # Project-specific context
```
---
## Validation Workflow
### Pre-Commit Checklist
Before committing changes to agents, commands, or context files:
1. **Check context dependencies**:
```bash
/check-context-deps
```
- Identifies agents using context files without declaring them
- Reports unused context files
- Validates context file paths
2. **Fix missing dependencies** (if needed):
```bash
/check-context-deps --fix
```
- Automatically adds missing `context:` dependencies to frontmatter
- Preserves existing dependencies
3. **Update registry**:
```bash
./scripts/registry/auto-detect-components.sh --auto-add
```
- Extracts dependencies from frontmatter
- Updates registry.json
4. **Validate registry**:
```bash
./scripts/registry/validate-registry.sh
```
- Checks all dependencies exist
- Validates component paths
- Reports missing dependencies
### Validation Tools
#### 1. `/check-context-deps` Command
**Purpose**: Analyze context file usage and validate dependencies
**What it checks**:
- ✅ Agents referencing context files in prompts
- ✅ Context dependencies declared in frontmatter
- ✅ Context files exist on disk
- ✅ Context files in registry
- ✅ Unused context files
**Usage**:
```bash
# Full analysis
/check-context-deps
# Specific agent
/check-context-deps opencoder
# Auto-fix
/check-context-deps --fix
# Verbose (show line numbers)
/check-context-deps --verbose
```
**Example output**:
```
# Context Dependency Analysis Report
## Summary
- Agents scanned: 25
- Context files referenced: 12
- Missing dependencies: 8
- Unused context files: 2
## Missing Dependencies
### opencoder
Uses but not declared:
- context:core/standards/code (referenced 3 times)
- Line 64: "Code tasks → .opencode/context/core/standards/code-quality.md"
Recommended fix:
dependencies:
- context:core/standards/code
```
#### 2. `auto-detect-components.sh` Script
**Purpose**: Scan for new components and update registry
**Dependency validation**:
- Checks dependencies during component scanning
- Logs warnings for missing dependencies
- Non-blocking (warnings only)
**Usage**:
```bash
# See what would be added
./scripts/registry/auto-detect-components.sh --dry-run
# Add new components
./scripts/registry/auto-detect-components.sh --auto-add
```
**Example warning**:
```
⚠ New command: Demo (demo)
Dependencies: subagent:coder-agent,subagent:missing-agent
⚠ Dependency not found in registry: subagent:missing-agent
```
#### 3. `validate-registry.sh` Script
**Purpose**: Comprehensive registry validation
**Checks**:
- ✅ All component paths exist
- ✅ All dependencies exist in registry
- ✅ No duplicate IDs
- ✅ Valid JSON structure
- ✅ Required fields present
**Usage**:
```bash
./scripts/registry/validate-registry.sh
```
**Example output**:
```
Validating registry.json...
✗ Dependency not found: opencoder → context:core/standards/code
Missing dependencies: 1
- opencoder (agent) → context:core/standards/code
Fix: Add missing component to registry or remove from dependencies
```
---
## Quality Standards
### Well-Maintained Registry
A high-quality registry has:
✅ **Complete dependencies**: All component dependencies declared
✅ **Validated dependencies**: All dependencies exist in registry
✅ **No orphans**: All context files used by at least one component
✅ **Consistent format**: Dependencies use `type:id` format
✅ **Up-to-date**: Registry reflects current component state
✅ **No broken paths**: All component paths valid
### Dependency Declaration Standards
**DO**:
- ✅ Declare all subagents you delegate to
- ✅ Declare all context files you reference
- ✅ Declare all commands you invoke
- ✅ Use correct format: `type:id`
- ✅ Keep dependencies in frontmatter (not hardcoded in prompts)
**DON'T**:
- ❌ Reference context files without declaring dependency
- ❌ Use invalid dependency formats
- ❌ Declare dependencies you don't actually use
- ❌ Forget to update registry after adding dependencies
---
## Commit Guidelines
### When Adding/Modifying Components
**1. Add component with proper frontmatter** (example):
```
id: my-agent
name: My Agent
description: Does something useful
tags:
- development
- coding
dependencies:
- subagent:coder-agent
- context:core/standards/code
```
**2. Validate dependencies**:
```bash
/check-context-deps my-agent
```
**3. Update registry**:
```bash
./scripts/registry/auto-detect-components.sh --auto-add
```
**4. Validate registry**:
```bash
./scripts/registry/validate-registry.sh
```
**5. Commit with descriptive message**:
```bash
git add .opencode/agent/my-agent.md registry.json
git commit -m "Add my-agent with coder-agent and code standards dependencies"
```
### When Modifying Context Files
**1. Check which agents depend on it**:
```bash
jq '.components[] | .[] | select(.dependencies[]? | contains("context:core/standards/code")) | {id, name}' registry.json
```
**2. Update context file**:
```bash
# Make your changes
vim .opencode/context/core/standards/code-quality.md
```
**3. Validate no broken references**:
```bash
/check-context-deps --verbose
```
**4. Update registry if needed**:
```bash
./scripts/registry/auto-detect-components.sh --auto-add
```
**5. Commit with impact note**:
```bash
git commit -m "Update code standards - affects opencoder, openagent, reviewer"
```
### When Deleting Components
**1. Check dependencies first**:
```bash
# Find what depends on this component
jq '.components[] | .[] | select(.dependencies[]? == "subagent:old-agent") | {id, name}' registry.json
```
**2. Remove from dependents**:
```bash
# Update agents that depend on it
# Remove the dependency from their frontmatter
```
**3. Delete component**:
```bash
rm .opencode/agent/subagents/old-agent.md
```
**4. Update registry**:
```bash
./scripts/registry/auto-detect-components.sh --auto-add
```
**5. Validate**:
```bash
./scripts/registry/validate-registry.sh
```
---
## Troubleshooting
### Missing Context Dependencies
**Symptom**:
```
/check-context-deps reports:
opencoder: missing context:core/standards/code
```
**Fix**:
```bash
# Option 1: Auto-fix
/check-context-deps --fix
# Option 2: Manual fix
# Edit .opencode/agent/core/opencoder.md
# Add to frontmatter:
dependencies:
- context:core/standards/code
# Then update registry
./scripts/registry/auto-detect-components.sh --auto-add
```
### Dependency Not Found in Registry
**Symptom**:
```
⚠ Dependency not found in registry: context:core/standards/code
```
**Causes**:
1. Context file doesn't exist
2. Context file exists but not in registry
3. Wrong dependency format
**Fix**:
```bash
# Check if file exists
ls -la .opencode/context/core/standards/code-quality.md
# If exists, add to registry
./scripts/registry/auto-detect-components.sh --auto-add
# If doesn't exist, remove dependency or create file
```
### Unused Context Files
**Symptom**:
```
/check-context-deps reports:
Unused: context:core/standards/analysis (0 references)
```
**Fix**:
```bash
# Option 1: Add to an agent that should use it
# Edit agent frontmatter to add dependency
# Option 2: Remove if truly unused
rm .opencode/context/core/standards/code-analysis.md
./scripts/registry/auto-detect-components.sh --auto-add
```
### Circular Dependencies
**Symptom**:
```
Agent A depends on Agent B
Agent B depends on Agent A
```
**Fix**:
- Refactor to remove circular dependency
- Extract shared logic to a third component
- Use dependency injection instead
---
## CI/CD Integration
### Pre-Commit Hook
```bash
#!/bin/bash
# .git/hooks/pre-commit
echo "Validating registry dependencies..."
# Check context dependencies
/check-context-deps || {
echo "❌ Context dependency validation failed"
echo "Run: /check-context-deps --fix"
exit 1
}
# Validate registry
./scripts/registry/validate-registry.sh || {
echo "❌ Registry validation failed"
exit 1
}
echo "✅ Registry validation passed"
```
### GitHub Actions
```yaml
name: Validate Registry
on: [push, pull_request]
jobs:
validate:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Validate registry
run: ./scripts/registry/validate-registry.sh
- name: Check context dependencies
run: /check-context-deps
```
---
## Best Practices
### For Component Authors
1. **Always declare dependencies** in frontmatter
2. **Use `/check-context-deps`** before committing
3. **Update registry** after adding components
4. **Validate** before pushing
5. **Document** why dependencies are needed
### For Maintainers
1. **Review dependencies** in PRs
2. **Run validation** in CI/CD
3. **Keep context files** organized and documented
4. **Monitor unused** context files
5. **Refactor** when dependency graphs get complex
### For CI/CD
1. **Fail builds** on validation errors
2. **Report** missing dependencies
3. **Track** dependency changes over time
4. **Alert** on circular dependencies
5. **Enforce** dependency declaration standards
---
## Related Documentation
- **Registry Guide**: `.opencode/context/openagents-repo/guides/updating-registry.md`
- **Registry Concepts**: `.opencode/context/openagents-repo/core-concepts/registry.md`
- **Adding Agents**: `.opencode/context/openagents-repo/guides/adding-agent.md`
- **Command Reference**: `/check-context-deps` command
---
## Summary
**Key Takeaways**:
1. Declare all dependencies in frontmatter (subagents, context files, etc.)
2. Use `/check-context-deps` to find missing context dependencies
3. Validate registry before commits
4. Keep registry in sync with component changes
5. Follow dependency format: `type:id`
**Quality Checklist**:
- [ ] All context files referenced have dependencies declared
- [ ] All dependencies exist in registry
- [ ] No unused context files (or documented why)
- [ ] Registry validates without errors
- [ ] Dependency format is consistent
**Remember**: Dependencies are documentation. They help users understand what components need and help the system validate integrity.
@@ -1,169 +0,0 @@
# OpenAgents Control Repository - Quick Start
**Purpose**: Get oriented in this repo in 2 minutes
---
## What Is This Repo?
OpenAgents Control is an AI agent framework with:
- **Category-based agents** (core, development, content, data, product, learning)
- **Eval framework** for testing agent behavior
- **Registry system** for component distribution
- **Install system** for easy setup
---
## Core Concepts (Load These First)
Before working on this repo, understand these 4 systems:
1. **Agents** → Load: `core-concepts/agents.md`
- How agents are structured
- Category system
- Prompt variants
- Subagents vs category agents
2. **Evals** → Load: `core-concepts/evals.md`
- How testing works
- Running tests
- Evaluators
- Session collection
3. **Registry** → Load: `core-concepts/registry.md`
- How components are tracked
- Auto-detect system
- Validation
- Install system
4. **Categories** → Load: `core-concepts/categories.md`
- How organization works
- Naming conventions
- Path patterns
---
## I Need To...
| Task | Load These Files |
|------|------------------|
| Add a new agent | `core-concepts/agents.md` + `guides/adding-agent.md` |
| Test an agent | `core-concepts/evals.md` + `guides/testing-agent.md` |
| Fix registry | `core-concepts/registry.md` + `guides/updating-registry.md` |
| Debug issue | `guides/debugging.md` |
| Find files | `lookup/file-locations.md` |
| Create release | `guides/creating-release.md` |
| Write content or copy | `core-concepts/categories.md` + `../content-creation/principles/navigation.md` |
| Use Claude Code helpers | `core-concepts/agents.md` + `guides/adding-agent.md` + `../to-be-consumed/claude-code-docs/create-subagents.md` |
---
## Essential Paths (Top 15)
```
.opencode/agent/core/ # Core agents (openagent, opencoder)
.opencode/agent/{category}/ # Category agents
.opencode/agent/subagents/ # Subagents
evals/agents/{category}/{agent}/ # Agent tests
evals/framework/src/ # Eval framework code
registry.json # Component catalog
install.sh # Installer
scripts/registry/validate-registry.sh # Validate registry
scripts/registry/auto-detect-components.sh # Auto-detect components
scripts/validation/validate-test-suites.sh # Validate tests
.opencode/context/ # Context files
.opencode/command/ # Slash commands
docs/ # Documentation
VERSION # Current version
package.json # Node dependencies
```
---
## Common Commands (Top 10)
```bash
# Add new agent (auto-detect)
./scripts/registry/auto-detect-components.sh --auto-add
# Validate registry
./scripts/registry/validate-registry.sh
# Test agent
cd evals/framework && npm run eval:sdk -- --agent={category}/{agent}
# Run smoke test
cd evals/framework && npm run eval:sdk -- --agent={agent} --pattern="smoke-test.yaml"
# Test with debug
cd evals/framework && npm run eval:sdk -- --agent={agent} --debug
# Validate test suites
./scripts/validation/validate-test-suites.sh
# Install locally (test)
REGISTRY_URL="file://$(pwd)/registry.json" ./install.sh --list
# Bump version
echo "0.X.Y" > VERSION && jq '.version = "0.X.Y"' package.json > tmp && mv tmp package.json
# Check version consistency
cat VERSION && cat package.json | jq '.version'
# Run full validation
./scripts/registry/validate-registry.sh && ./scripts/validation/validate-test-suites.sh
```
---
## Repository Structure (Quick View)
```
opencode-agents/
├── .opencode/
│ ├── agent/{category}/ # Agents by domain
│ │ ├── core/ # Core system agents
│ │ ├── development/ # Dev specialists
│ │ ├── content/ # Content creators
│ │ ├── data/ # Data analysts
│ │ ├── product/ # Product managers
│ │ ├── learning/ # Educators
│ │ └── subagents/ # Delegated specialists
│ ├── command/ # Slash commands
│ └── context/ # Shared knowledge
├── evals/
│ ├── agents/{category}/ # Test suites
│ └── framework/ # Eval framework
├── scripts/
│ ├── registry/ # Registry tools
│ └── validation/ # Validation tools
├── docs/ # Documentation
├── registry.json # Component catalog
└── install.sh # Installer
```
---
## Quick Troubleshooting
| Problem | Solution |
|---------|----------|
| Registry validation fails | `./scripts/registry/auto-detect-components.sh --auto-add` |
| Test fails | Load `guides/debugging.md` |
| Can't find file | Load `lookup/file-locations.md` |
| Install fails | Check: `which curl jq` |
| Path resolution issues | Check `core-concepts/categories.md` |
---
## Next Steps
1. **First time?** → Read `core-concepts/agents.md`, `evals.md`, `registry.md`
2. **Adding agent?** → Load `guides/adding-agent.md`
3. **Testing?** → Load `guides/testing-agent.md`
4. **Need details?** → Load specific files from `core-concepts/` or `guides/`
---
**Last Updated**: 2026-01-13
**Version**: 0.5.1
@@ -1,248 +0,0 @@
# Context Bundle Template
**Purpose**: Template for creating context bundles when delegating tasks to subagents
**Location**: `.tmp/context/{session-id}/bundle.md`
**Used by**: repo-manager agent when delegating to subagents
---
## Template
```markdown
# Context Bundle: {Task Name}
Session: {session-id}
Created: {ISO timestamp}
For: {subagent-name}
Status: in_progress
## Task Overview
{Brief description of what we're building/doing}
## User Request
{Original user request - what they asked for}
## Relevant Standards (Load These Before Starting)
**Core Standards**:
- `.opencode/context/core/standards/code.md` → Modular, functional code patterns
- `.opencode/context/core/standards/tests.md` → Testing requirements and TDD
- `.opencode/context/core/standards/docs.md` → Documentation standards
- `.opencode/context/core/standards/patterns.md` → Error handling, security patterns
**Core Workflows**:
- `.opencode/context/core/workflows/delegation.md` → Delegation process
- `.opencode/context/core/workflows/task-breakdown.md` → Task breakdown methodology
- `.opencode/context/core/workflows/review.md` → Code review guidelines
## Repository-Specific Context (Load These Before Starting)
**Quick Start** (ALWAYS load first):
- `.opencode/context/openagents-repo/quick-start.md` → Repo orientation and common commands
**Core Concepts** (Load based on task type):
- `.opencode/context/openagents-repo/core-concepts/agents.md` → How agents work
- `.opencode/context/openagents-repo/core-concepts/evals.md` → How testing works
- `.opencode/context/openagents-repo/core-concepts/registry.md` → How registry works
- `.opencode/context/openagents-repo/core-concepts/categories.md` → How organization works
**Guides** (Load for specific workflows):
- `.opencode/context/openagents-repo/guides/adding-agent.md` → Step-by-step agent creation
- `.opencode/context/openagents-repo/guides/testing-agent.md` → Testing workflow
- `.opencode/context/openagents-repo/guides/updating-registry.md` → Registry workflow
- `.opencode/context/openagents-repo/guides/debugging.md` → Troubleshooting
**Lookup** (Quick reference):
- `.opencode/context/openagents-repo/lookup/file-locations.md` → Where everything is
- `.opencode/context/openagents-repo/lookup/commands.md` → Command reference
## Key Requirements
{Extract key requirements from loaded context}
**From Standards**:
- {requirement 1 from standards/code.md}
- {requirement 2 from standards/tests.md}
- {requirement 3 from standards/docs.md}
**From Repository Context**:
- {requirement 1 from repo context}
- {requirement 2 from repo context}
- {requirement 3 from repo context}
**Naming Conventions**:
- {convention 1}
- {convention 2}
**File Structure**:
- {structure requirement 1}
- {structure requirement 2}
## Technical Constraints
{List technical constraints and limitations}
- {constraint 1 - e.g., "Must use TypeScript"}
- {constraint 2 - e.g., "Must follow category-based organization"}
- {constraint 3 - e.g., "Must include proper frontmatter metadata"}
## Files to Create/Modify
{List all files that need to be created or modified}
**Create**:
- `{file-path-1}` - {purpose and what it should contain}
- `{file-path-2}` - {purpose and what it should contain}
**Modify**:
- `{file-path-3}` - {what needs to be changed}
- `{file-path-4}` - {what needs to be changed}
## Success Criteria
{Define what "done" looks like - binary pass/fail conditions}
- [ ] {criteria 1 - e.g., "Agent file created with proper frontmatter"}
- [ ] {criteria 2 - e.g., "Eval tests pass"}
- [ ] {criteria 3 - e.g., "Registry validation passes"}
- [ ] {criteria 4 - e.g., "Documentation updated"}
## Validation Requirements
{How to validate the work}
**Scripts to Run**:
- `{validation-script-1}` - {what it validates}
- `{validation-script-2}` - {what it validates}
**Tests to Run**:
- `{test-command-1}` - {what it tests}
- `{test-command-2}` - {what it tests}
**Manual Checks**:
- {check 1}
- {check 2}
## Expected Output
{What the subagent should produce}
**Deliverables**:
- {deliverable 1}
- {deliverable 2}
**Format**:
- {format requirement 1}
- {format requirement 2}
## Progress Tracking
{Track progress through the task}
- [ ] Context loaded and understood
- [ ] {step 1}
- [ ] {step 2}
- [ ] {step 3}
- [ ] Validation passed
- [ ] Documentation updated
---
## Instructions for Subagent
{Specific, detailed instructions for the subagent}
**IMPORTANT**:
1. Load ALL context files listed in "Relevant Standards" and "Repository-Specific Context" sections BEFORE starting work
2. Follow ALL requirements from the loaded context
3. Apply naming conventions and file structure requirements
4. Validate your work using the validation requirements
5. Update progress tracking as you complete steps
**Your Task**:
{Detailed description of what the subagent needs to do}
**Approach**:
{Suggested approach or methodology}
**Constraints**:
{Any additional constraints or notes}
**Questions/Clarifications**:
{Any questions the subagent should consider or clarifications needed}
```
---
## Usage Instructions
### When to Create a Context Bundle
Create a context bundle when:
- Delegating to any subagent
- Task requires coordination across multiple components
- Subagent needs project-specific context
- Task has complex requirements or constraints
### How to Create a Context Bundle
1. **Create session directory**:
```bash
mkdir -p .tmp/context/{session-id}
```
2. **Copy template**:
```bash
cp .opencode/context/openagents-repo/templates/context-bundle-template.md \
.tmp/context/{session-id}/bundle.md
```
3. **Fill in all sections**:
- Replace all `{placeholders}` with actual values
- List specific context files to load (with full paths)
- Extract key requirements from loaded context
- Define clear success criteria
- Provide specific instructions
4. **Pass to subagent**:
```javascript
task(
subagent_type="subagents/core/{subagent}",
description="Brief description",
prompt="Load context from .tmp/context/{session-id}/bundle.md before starting.
{Specific task instructions}
Follow all standards and requirements in the context bundle."
)
```
### Best Practices
**DO**:
- ✅ List context files with full paths (don't duplicate content)
- ✅ Extract key requirements from loaded context
- ✅ Define binary success criteria (pass/fail)
- ✅ Provide specific validation requirements
- ✅ Include clear instructions for subagent
- ✅ Track progress through the task
**DON'T**:
- ❌ Duplicate full context file content (just reference paths)
- ❌ Use vague success criteria ("make it good")
- ❌ Skip validation requirements
- ❌ Forget to list technical constraints
- ❌ Omit file paths for files to create/modify
### Example Context Bundle
See `.opencode/context/openagents-repo/examples/context-bundle-example.md` for a complete example.
---
**Last Updated**: 2025-01-21
**Version**: 1.0.0
@@ -1,38 +0,0 @@
# OpenAgents Templates
**Purpose**: Template files and patterns for OpenAgents Control
---
## Structure
```
openagents-repo/templates/
├── navigation.md (this file)
└── [template files]
```
---
## Quick Routes
| Task | Path |
|------|------|
| **View templates** | `./` |
| **Blueprints** | `../blueprints/navigation.md` |
| **Guides** | `../guides/navigation.md` |
---
## By Type
**Templates** → Reusable template files
**Patterns** → Common patterns and structures
---
## Related Context
- **OpenAgents Navigation** → `../navigation.md`
- **Blueprints** → `../blueprints/navigation.md`
- **Guides** → `../guides/navigation.md`
@@ -1,88 +0,0 @@
<!-- Context: project-intelligence/business | Priority: high | Version: 1.0 | Updated: 2025-01-12 -->
# Business Domain
> Document the business context, problems solved, and value created.
## Quick Reference
- **Purpose**: Understand why this project exists
- **Update When**: Business direction changes, new features shipped, pivot
- **Audience**: Developers needing context, stakeholders, product team
## Project Identity
```
Project Name: [Name]
Tagline: [One-line description]
Problem Statement: [What problem are we solving?]
Solution: [How we're solving it]
```
## Target Users
| User Segment | Who They Are | What They Need | Pain Points |
|--------------|--------------|----------------|-------------|
| [Primary] | [Description] | [Their needs] | [Their frustrations] |
| [Secondary] | [Description] | [Their needs] | [Their frustrations] |
## Value Proposition
**For Users**:
- [Key benefit 1]
- [Key benefit 2]
- [Key benefit 3]
**For Business**:
- [Key value 1]
- [Key value 2]
## Success Metrics
| Metric | Definition | Target | Current |
|--------|------------|--------|---------|
| [Metric 1] | [What it measures] | [Goal] | [Actual] |
| [Metric 2] | [What it measures] | [Goal] | [Actual] |
## Business Model (if applicable)
```
Revenue Model: [How the business makes money]
Pricing Strategy: [If applicable]
Unit Economics: [CAC, LTV, etc.]
Market Position: [Where we fit in the market]
```
## Key Stakeholders
| Role | Name | Responsibility | Contact |
|------|------|----------------|---------|
| [Product Owner] | [Name] | [What they own] | [Contact] |
| [Tech Lead] | [Name] | [What they own] | [Contact] |
| [Business Lead] | [Name] | [What they own] | [Contact] |
## Roadmap Context
**Current Focus**: [What we're working on now]
**Next Milestone**: [Upcoming goal]
**Long-term Vision**: [Where this is heading]
## Business Constraints
- [Constraint 1] - [Why it exists]
- [Constraint 2] - [Why it exists]
## Onboarding Checklist
- [ ] Understand the problem statement
- [ ] Identify target users and their needs
- [ ] Know the key value proposition
- [ ] Understand success metrics
- [ ] Know who the stakeholders are
- [ ] Understand current business constraints
## Related Files
- `technical-domain.md` - How this business need is solved technically
- `business-tech-bridge.md` - Mapping between business and technical
- `decisions-log.md` - Business decisions with context
Loaded 100 of 1852 files, more files were not shown because too many files have changed in this diff. Show more