Commit Graph
88 Commits
Author SHA1 Message Date
evgeny ae1682bbe2 Document XFixes dependency and translate cursor controls 2026-10-16 14:14:37 +02:00
evgeny 9cc996b775 Document Reed-Solomon module and update streaming design 2026-10-10 19:45:14 +02:00
evgeny 84a171dcac tests: keep build script regressions outside the regular check 2026-10-10 17:48:20 +02:00
evgeny 4908cb7e19 Document shared FreeBSD builds and locate Bash through PATH 2026-10-10 15:25:58 +02:00
evgeny d6c6a4d111 Use verified incremental builds instead of mandatory clean 2026-10-10 15:25:58 +02:00
evgeny a97e8bcc1f docs: update file inventory and module header contracts
Refresh both AGENTS guides and clarify module purpose, usage, ownership, threading, return values and wire formats in headers. Remove the unused timeout_heap freed_count field and unimplemented getter.
2026-10-04 13:10:01 +02:00
evgeny e3caf15f32 Guard routing loops and reconcile blocked transit against current paths
Track visited nodes outside the signed body for all routed packet types. Recheck queued transit routes without losing FIFO priority, reject unknown-group transit, and suppress BGP announcements caused only by RTT jitter.

Document routing rules and wire format. Validate graph churn, REINIT, encrypted transit, malformed paths, loop drops and queue cleanup; full suite: 106 passed, 1 skipped, plus proxy integration and sanitizer checks.
2026-09-28 15:36:47 +03:00
evgeny 904a9fbda1 docs: refresh development guides for current architecture 2026-09-28 12:10:39 +03:00
evgeny cc2d2b5f78 etcp: фикс UAF teardown (двойное закрытие линка) + вытеснение down-линка при коллизии адреса
- tcp_link_close_cb (серверная ветка): убрать преждевременный огонь link_status/on_link_down,
  etcp_link_close сам снимает линк из списка и стреляет коллбэки в безопасном порядке
  (устраняет двойное закрытие линка — UAF из callring 45502)
- etcp_connection_close: при callbacks_running откладывать закрытие через uasync_call_soon
  вместо *(int*)0=0
- callbacks_running: uint8_t-флаг -> int-счётчик вложенности (++/--)
- etcp_fire_conn_status / etcp_fire_link_status_cbk: поднимать callbacks_running вокруг огня
- insert_link_queue: при коллизии адреса с чужим conn — ERROR при одинаковом pubkey,
  отказ при up-линке, вытеснение (etcp_link_close) при down-линке

Попутно: доки/комментарии в etcp_router, ACK-интервал 100мс->10мс (ROUTER_ACK_INTERVAL_TB).
2026-09-26 13:40:33 +03:00
evgeny 75e39d2fee chat_join: модуль join-протокола, invite-ссылки v2, синхронизация мемберов 2026-09-14 19:57:27 +03:00
evgeny 1eb0566cb5 chat: enforce storage_total_size + media orphan cleanup; async_dns: fix use-after-free
- media cache: удаление сирот на старте (сверка ФС с сообщениями) + лимит
  storage_total_size (удаление самых старых файлов, только старше backfill_days)
- ma_dir_walk(): кроссплатформенный рекурсивный обход каталога (POSIX/Windows)
- async_dns: первый драйв slow-пути отложен через call_soon (исключён
  double-free при синхронном завершении запроса); cancel унифицирован
- chat: видимый WARN 'media too large' + честный счётчик started/skipped
2026-09-13 03:11:10 +03:00
evgeny e636c3a420 docs: доступ на узел colo2 2026-09-12 21:24:04 +03:00
evgeny b7da6547c1 connection: логировать только connect/reconnect/disconnect; фиксы конфига чатгуи
- категория connection теперь только жизненный цикл соединения; INIT/PING/keepalive
  переклассифицированы в ETCP/SOCKET/KEEPALIVE/REALITY/ROUTING, удалены дубли
- settingsdialog: устранён дубль my_node_name в конфиге (двухпроходная запись + только при изменении)
- storagesettingspage: канонические ключи storage_total_size/storage_unit_size с суффиксом M/G
- audiodevicesettingspage: compressor_max_gain_db min 0→1
- etcp_connections: %s-баг в логе TCP link (addr → sockaddr_storage_to_str)
- docs/sample: [chatserver], актуализированы ключи хранилища
2026-09-11 15:24:56 +03:00
evgeny 898a0a4a66 reality: реклама reality-параметров сокетов через BGP и применение к исходящим TCP-линкам 2026-09-10 11:24:17 +03:00
evgeny 7c95ddb09b chatgui: значок владельца канала (🔑) в шапке блока мемберов при наличии private key 2026-09-05 15:23:45 +03:00
evgeny f47ad7d87f tcp_proxy: fix throughput (tx_queue backpressure) + connect timeout + integration test
- Router: unified service delivery format [svc_id][src][dst][payload] (end-to-end src/dst from SVC_ROUTE header)
- tcp_proxy_client: replace single tx_buf drop-on-backpressure with bounded tx_queue + send_q waiter + retry timer (no data loss, bulk window reopen — removes 62.5ms delayed-ACK stall)
- tcp_io: configurable connect timeout (tcp_conn_set_connect_timeout)
- tcp_proxy_server: connect_timeout config option (ms, default 2000, 0=off)
- check.sh: run tcp_proxy_full integration test with sudo (skip if passwordless sudo unavailable)
- test_tcp_io: connect timeout unit test; docs (lwip_nuances.md)
2026-09-03 13:49:09 +03:00
evgeny aee44943fa topo: invite вынесен в topo_group_invite, conn_mgr упрощён до 3-фазного open; member_sync: полный пересчёт merkle-дерева при старте
- topo_group_invite.c/h: invite/join через DIRECT + INVITE_INFO_REQ/RESP (из conn_mgr)
- conn_mgr: conn_mgr_open_invite → conn_mgr_open, убран invite-режим (-480 строк)
- topo_group_connect: подключение через node_conn_direct вместо conn_mgr
- member_sync: _rebuild_all_trees сносит merkle_tree_hash канала и собирает заново из peers (чистка осиротевших листов)
- member_sync: хеш блока A/B (update_sig/adm_tags) в док-комментариях
- обновлены doc-файлы и тесты
2026-08-16 09:19:09 +03:00
evgeny b503fa4418 etcp: cyclic link_id, link close on addr change, inflight nullify; test rewrite 2026-08-11 23:10:24 +03:00
evgeny 0f1bd3d120 keepalive handshake: device_type + keepalive negotiation in UDP/TCP INIT
- Добавлен device_type (1 байт, CLIENT_TYPE_*) в INIT_REQUEST и INIT_RESPONSE
- Добавлен keepalive_interval (2 байта) в INIT_RESPONSE
- STCP handshake расширен с 43 до 46 байт (device_type+keepalive)
- negotiate_keepalive(): оба desktop/server → min, иначе (есть mobile) → max
- keepalive_interval вынесен в UTUN_INSTANCE для использования в handshake
- stcp_client_connect() принимает device_type и keepalive_interval
- peer_device_type/peer_keepalive_interval передаются через stcp_link в ETCP_LINK
2026-08-11 11:16:27 +03:00
evgeny ca176e1f24 encrypt overhead fix (33→36, noencrypt_len), mtu default 1280, connmonitor window, total_decrypted for TCP, check.sh 2026-08-08 16:56:32 +03:00
evgeny 38c9f229bc stcp: ed25519 pubkey exchange in handshake, fix instance destroy segfault, fix chat_tcp_test
- u_async.c: process immediate_queue BEFORE epoll_wait in uasync_poll
- stcp_link.c: defensive NULL etcp_conn/etcp_link in stcp_link_close
- utun_instance.c: move stcp_server_list_destroy_all to Phase J (before ETCP),
  add deferred drain in both Phase J and L
- stcp.h/c: STCP_HS_ENC_CLIENT 6→38, STCP_HS_ENC_SERVER 7→39,
  exchange ed25519_pubkey in both handshake directions
- etcp_connections.c: copy peer_ed25519 from STCP to ETCP_CONN in
  etcp_link_enter_ready_tcp; guard link->etcp in tcp_link_close_cb
- chat_sync.c: fix lk->conn NULL dereference for TCP links
- test_stcp.c: update stcp_server_create/stcp_client_connect callsites
- tools/chat_tcp_test: change transport udp→tcp
2026-08-07 00:48:27 +03:00
evgeny d773f1724a refactor: STCP connections as ETCP links (is_tcp)
- Add is_tcp, tcp_link, tcp_reconnect_* fields to ETCP_LINK
- Remove transport_link from ETCP_CONN, add tcp_link_count
- stcp_link: replace embedded etcp_conn with pointer; add etcp_link backpointer
- stcp_link: link_rx_cb routes ETCP data to etcp_conn_input
- stcp_link: call on_close_cb in close
- New TCP functions: etcp_link_enter_ready_tcp, etcp_tcp_link_start_connect,
  etcp_tcp_link_start_reconnect, tcp_link_close_cb, etcp_tcp_send
- etcp_link_new: handle NULL conn for TCP links; skip enter_init for TCP
- etcp_link_close: close stcp_link for TCP
- etcp_encrypt_send: redirect TCP to etcp_tcp_send (skips AES-CCM)
- tcp_server_on_link: create ETCP_LINK(is_tcp=1) + enter_ready_tcp
- etcp_connect: TCP addrs -> etcp_tcp_link_start_connect
- Remove ctx->tcp_link/tcp_ready
- test_stcp_traffic: find stcp_link via links->is_tcp instead of transport_link
- 66/68 tests pass; 2 STCP tests expected to fail (integration not complete)
2026-08-06 22:14:32 +03:00
evgeny dfe9d36783 wip: STCP TCP chat integration — diagnostics, invite TCP fallback, test infra 2026-08-06 18:41:41 +03:00
evgeny 14d1673f06 AGENTS.md: update structure to reflect actual codebase layout
- Debug categories: 25→27, add CONNECTIVITY and DB_SYNC
- Directory structure: reflect real subdirs (transport_layer, routing_layer, chat, proxy, lwip_tcp, media_delivery, media_async)
- File overview: reorganize by actual subdirectories, fix paths
- BBR: src/BBR/ → src/transport_layer/BBR/
- topo_node_lmdb → topo_node_sqlite
- Remove sha256.c/h (deleted), add missing lib/ modules (sqlite3, opus_codec, audio_compressor, json_flat)
- Add missing modules: ntp_time, ntp_node_time, media_delivery, media_async, nat_detection, topo_recovery, topo_group_connect, node_conn_direct, socket_monitor
- Add chatgui-android to tools/
2026-08-02 12:16:25 +03:00
evgeny 7799d54c0c conn_mgr: обновить API на единую точку входа conn_mgr_open_invite, поправить порядок destroy, обновить всех потребителей
- conn_mgr_cb_t: 5 параметров (h, node_id, group_id, event, arg)
- conn_mgr_open_invite: единая функция (ni=NULL — обычное, ni!=NULL — invite)
- CONN_EVENT_JOIN для успешного invite
- struct CONN_MGR_HANDLE: добавлен group_id
- bind/unbind в conn_mgr_init/destroy (один на инстанс, перезапись безвредна)
- utun_instance_destroy: conn_mgr_destroy до закрытия сокетов
- cm_ncd_callback: занулять ncd_handle при DOWN, защита от вызова после destroy
- topo_group_connect: свой трекинг handle'ов, conn_mgr_close для отмены
- обновлены: topo_recovery, media_download, media_delivery, chat_sync, тесты
2026-08-01 17:52:56 +03:00
evgeny 625382de34 node_conn_direct: add specific_sock param, move registry to per-instance; expose sc_hex_to_binary; add CONFIG_CONN_HANDLE 2026-08-01 12:34:53 +03:00
evgeny fe9be74e29 topo refactoring (WIP): topo_node.h + AGENTS.md safety rules 2026-07-31 16:01:47 +03:00
evgeny 51d6c782ac voice playback: miniaudio cursor ground truth + EMA smooth decay
- SoundManager: add currentPcmCursor()/isCurrentPcmPlaying() via ma_sound_get_cursor_in_pcm_frames / ma_sound_is_playing
- messagedelegate: replace wall-clock position tracking with real miniaudio cursor, remove 200ms hardcoded delay
- messagedelegate: end-of-playback detection via isCurrentPcmPlaying() instead of wall-clock comparison (was killing audio ~200ms early)
- messagedelegate: EMA smoothing of displayFrac/displayElapsed (0.15 factor, 16ms timer)
- messagedelegate: timer continues after audio stops to animate EMA decay to 0 (not mouse-move dependent)
- media_delivery: download progress callback, Ed25519 sig verify, author fallback, content hash check
- conn_mgr: already-connected returns OK with callback instead of error
- chatgui: voice download-pending auto-play, file download progress bar, media bar colors
2026-07-31 00:17:03 +03:00
evgeny df13b16fad docs: AGENTS.md chat module descriptions, improved ETCP debug logging, android AGENTS update 2026-07-30 12:20:04 +03:00
evgeny b8920cf483 chat_settings: [chat] config section + live sync + fix test segfault
- New chat_setting.c/h: unified chat settings API (int/str getters)
- Config parser supports [chat] section, calls chat_setting_set per key
- chat_core exposes chat_core_set_setting / trampoline for GUI bridge
- chatgui: AudioDevice + Storage pages persist to [chat] config + live apply
- chatgui-android: ConfigProvider, NativeLib, headless handle chat settings
- MessageList: 200 msg limit, fixed saveChannelState via trampoline
- test_media_delivery_full: fix segfault at shutdown (mon() timer vs freed conns)
2026-07-29 21:18:49 +03:00
evgeny 064bed06f2 android: fix key regeneration, Clear DB, uasync eventfd, udp log from config
- jni_on_event: fix JNI_EDETACHED — AttachCurrentThread/DetachCurrentThread
- u_async.c: pipe→eventfd for reliable cross-thread wakeup
  add process_posted_tasks in poll() path, fix EINTR (saved_errno)
- bridge_debug_hook: keep udp_log_write, remove mutex from udp_log
- memory_pool.c: alloc/free logs INFO→DEBUG (reduce spam)
- instance_lite.c: chat_event_set_handler before ensure_keys,
  KEYS_GENERATED events, fchmod 0600 on keys.conf,
  UDP log from config [log_udp] section, regenerate_trampoline
- config_parser: parse [log_udp] ip/port
- NativeLib.kt: remove callbacksSet, simplify init/stop/destroy
- ChatViewModel.kt: unified event callback in init, pubKey StateFlow,
  requestRegenerateKeys, clearState fix
- SettingsScreen.kt: pubKey.collectAsState(), Clear DB via restart()
- HeadlessService.kt: remove setUdpLogTarget (now from config)
- android_udp_log.c: simplified — no mutex, buffer+send+timer in one thread
- utun_instance.c: NULL-check tcp_connections before destroy
- chat_event.h: +CHAT_EVT_KEYS_GENERATED 11, -NODE_CHANGED
- logreceiver: UDP log receiver tool
2026-07-28 16:29:33 +03:00
evgeny 9e58160480 android: initial chatgui-android project with QR join
- libutun_lite: C99 core, invite_link_c.h/c (compatible with desktop)
- jni_bridge: android_jni_bridge.c/h with logcat logging
- app: Jetpack Compose UI, CameraX + ML Kit QR scanner
- JoinChannelDialog, QrScanScreen with proper camera lifecycle
- headless: control socket with join command
- OpenSSL prebuilt libs for arm64-v8a + x86_64
2026-07-27 22:50:29 +03:00
Evgeny 3e95cf5ca6 fix: dialog hangs on join success + remove per-packet spam logs
- chat_sync.c: move pending_invite clear from INFO_RESP to WELCOME,
  post GUI_EVT_CONNECT_RESULT on invite success in cs_handle_welcome
- etcp_connections.c: NORM_DECRYPT_TRY/CRYPTO_DECRYPT_OK DEBUG_DEBUG->TRACE
- secure_channel.c: ENCRYPT/DECRYPT DEBUG_DEBUG->TRACE
2026-07-23 16:36:40 +03:00
Evgeny 852ce401e7 Добавлен group_id в etcp_router и topo_group, удалён msg_transport
- etcp_router: SVC_ROUTE_HDR (+8B), ETCP_ROUTER_CONN/TRANSIT_QUEUE хеши расширены под group_id
- Все caller'ы (proxy, routing, conn_mgr, nat, chatgui, tests) обновлены
- topo_group: глобальный реестр TOPO_NODE по node_id, ref_count=0 при создании
- BGP-обмен активируется для всех групп (а не только дефолтной)
- Протокол: group_id добавлен в WITHDRAW/TABLE_REQ/TABLE_COMPLETE/ERR_GROUP_MISMATCH
- Per-connection коллбэки заменены на instance-level conn_status
- msg_transport полностью удалён из проекта
57/57 тестов
2026-07-22 15:51:03 +03:00
Evgeny 17202beb0a test_ntp: use compat headers instead of arpa/inet.h, add rule to AGENTS.md 2026-07-22 08:47:56 +03:00
Evgeny beed7a4e07 fix: INIT reset logic — reset only on session change or got_initial_pkt, always sync session_id 2026-07-19 13:18:42 +03:00
Evgeny 8f695cc828 AGENTS.md: add chatgui config and log file paths 2026-07-18 03:22:29 +03:00
Evgeny 4b7c7738de debug: add DEBUG_CATEGORY_DB_SYNC (26) + TRACE logs for db_sync, merkle_sync, chat_sync, member_sync 2026-07-18 03:19:46 +03:00
Evgeny 58b9f28149 fix chatgui: prevent invite deadlock on connection flap
cs_on_conn_down: remove invite state zeroing on disconnection.
Only timeout callbacks and explicit protocol end (CS_MSG_ERROR,
transition to JOIN phase) clear the invite state.

Connection flap now causes automatic retry:
  conn_up → pending_invite_ch_id != 0 && !info_req_timer → resend
  CHANNEL_INFO_REQ. No deadlock possible.
2026-07-14 18:22:05 +03:00
Evgeny f617dcd68e db_sync: LMDB→SQLite, multi-instance, etcp_api, chatgui integration
- src/db_sync: LMDB replaced with SQLite (separate tables per instance: db_sync_name_id)
  TTL index (author, timestamp), WAL mode, per-instance peers/timers
  opaque DB_SYNC_INSTANCE* handle, insert callback, select API
  new fields: author_signature, delivered_peers, delivery_chain
  transport: etcp_api direct P2P (was etcp_router)

- src/db_sync.h: user-oriented API docs, multi-instance lifecycle

- lib/Makefile.am: liblmdb sources removed
- lib/sqlite3.c+h: SQLite amalgamation added

- topo_node: LMDB→SQLite (src/topo_node_sqlite.c moved from chatgui)

- chatgui: real db_sync.c replaces db_sync_stub.c
  chat_core: db_sync_insert_signed + dual-write insert callback
  utun_node: db_sync_init/destroy replaces chat_sync_init/destroy
  CMakeLists: stubs removed, real db_sync.c added
2026-07-14 14:17:43 +03:00
Evgeny d70cdbb040 chatgui: preserve active channel selection on reload, members sync, profile settings, member list model 2026-07-14 08:52:10 +03:00
jeka 93de0aa591 agents 2026-07-13 15:46:29 +03:00
Evgeny c0935aba6c docs: update AGENTS.md and etcp_arch.md 2026-07-13 15:21:30 +03:00
Evgeny 62954ca0a9 chatgui: replace MsgClient/ChatPropagator/crypto with chat_core, update DB/messaging/layout 2026-07-08 21:15:45 +03:00
Evgeny 19a4505a03 chat_sync: новый протокол синхронизации чата вместо ChatPropagator
- gui_bridge.h/cpp — кросс-поточные асинхронные коллбэки uasync↔GUI
- chat_sync.h/c — протокол синхронизации (INIT_SYNC/RESP, SEND_DATA, PUSH/ACK, SYNC_DONE)
- DbManager полностью переписан: per-channel таблицы msg_<ch> + peers_<ch>,
  4 ключа в channels, signature, protocol/rtt в node_addresses, sync-методы
- ChatPropagator удалён, заменён на chat_sync
- db_sync_stub.c — заглушки для сборки chatgui без LMDB-версии db_sync
- db_schema.md актуализирован под новую схему
2026-07-06 16:25:52 +03:00
Evgeny 0b5a710fe6 etcp_connect + BGP_READY callbacks + router reconnect test
- etcp_connect.c: async connect API with EARLY/LATE/BGP_READY callbacks
- etcp_connect.h: declaration (via etcp_api.h)
- connect_deliver: fix to preserve unfired callback nodes
- etcp_set_routing_exchange_state: fires bgp_ready_cbk on TABLE_COMPLETE
- route_bgp: TABLE_COMPLETE handler + route_bgp_send_table_complete
- etcp.c: connections_count tracking, cleanup on close
- test_etcp_router_reconnect: a-b-c topology with b restart (etcp_connect)
- test_etcp_connect: unit tests for async connect API
- etcp_router: inflight/retrans/close improvements
- Various proxy/NAT/router cleanups
2026-07-04 18:51:01 +03:00
Evgeny 2cd38988e6 AGENTS: add ASAN build instructions 2026-07-02 15:49:10 +03:00
Evgeny 53d41d5c62 fix: use-after-free in route table — add route_delete before u_free in route_bgp_update_my_nodeinfo and route_bgp_process_nodeinfo, relax routes_overlap to allow hierarchical routes
Root cause: when bgp->local_node was reallocated (changed=1), old route
entries remained with stale v_node_info pointing to freed memory. The freed
memory got reused by a BGP peer's NODEINFO_Q, causing route_lookup to return
wrong node_id (b3b2b1b0afaeadac instead of local), which then failed at
etcp_route_send with 'no BGP route' and dropped packets to 10.23.2.1.

Changes:
- route_node.c: route_delete(old) before u_free + route_insert(new) after alloc
- utun_instance.c: remove duplicate route_insert (now atomic inside function)
- route_bgp.c: route_delete before freeing old peer NODEINFO_Q
- route_lib.c: routes_overlap now rejects only exact prefix+network duplicates
- tests/Makefile.am: add route_lib.o to test_route6_lib
2026-07-01 18:53:23 +03:00
Evgeny 231081dd2f docs: add chatgui animated emoji system and tdesktop-dev reference to AGENTS.md 2026-06-24 13:50:10 +03:00
Evgeny 73f707ef9b crypto: migrate ECDH(secp256r1) to X25519, drop TinyCrypt
- SC_PUBKEY_SIZE: 64 -> 32 bytes, SC_PUBKEY_ENC_SIZE: 72 -> 40
- sc_set_peer_public_key uses EVP_PKEY_derive with X25519
- sc_generate_keypair via EVP_PKEY_keygen(EVP_PKEY_X25519)
- sc_compute_public_key_from_private via EVP_PKEY_new_raw_private_key
- sc_validate_key: check non-zero (X25519 has no invalid keys)
- sc_obfuscate_pubkey: XOR now 32 bytes
- Remove all TinyCrypt code (~860 lines), always use OpenSSL
- Remove TinyCrypt from build system, drop test_crypto/test_ecc_encrypt
- Update all test keys (128 hex -> 64 hex)
- Fix bbr_integration.c split-line pubkeys
2026-06-17 21:28:07 +03:00