db_sync/merkle_sync/chat_sync шлют group_id (uint64 BE) вместо текстового
channel_id/namespace. Приёмная сторона находит группу/таблицу через
topo_groups_find(group_id) и требует существующую CHAT-группу. db_sync:
instance key заменён с SHA256(ch_id) на strtoull(ch_id)=group_id.
- u_async.c: process immediate_queue BEFORE epoll_wait in uasync_poll
- stcp_link.c: defensive NULL etcp_conn/etcp_link in stcp_link_close
- utun_instance.c: move stcp_server_list_destroy_all to Phase J (before ETCP),
add deferred drain in both Phase J and L
- stcp.h/c: STCP_HS_ENC_CLIENT 6→38, STCP_HS_ENC_SERVER 7→39,
exchange ed25519_pubkey in both handshake directions
- etcp_connections.c: copy peer_ed25519 from STCP to ETCP_CONN in
etcp_link_enter_ready_tcp; guard link->etcp in tcp_link_close_cb
- chat_sync.c: fix lk->conn NULL dereference for TCP links
- test_stcp.c: update stcp_server_create/stcp_client_connect callsites
- tools/chat_tcp_test: change transport udp→tcp
- member_sync apply_items: empty payload is normal sync completion, not error
- topo_node_sqlite node_load: no addresses is normal for stub/seed nodes
- conn_mgr_core: node skipped when not in routing (normal for nodes w/o addrs)
- topo_group send_nodeinfo: downgrade ERROR→WARN in DEBUG category
- db_sync INIT_RESP: protocol details belong in DEBUG, not INFO
- db_record_insert_cascade: unified insert with on_insert + PUSH cascade
- SYNC_DONE moved from INIT_RESP to DATA handler (master sends after slave response)
- db_sync_add/remove_done_cbk: global sync-completion subscription (etcp pattern)
- REQUEST_SYNC handler: sync_state != 1 instead of == 0
- SYNC_DONE mc==pc hash mismatch: accept as converged
- Removed unconditional sync_state=2 from DATA handler
- test: done_cbk for sync phases, count-only for PUSH phases, removed dead B<>C reinitiate
- Создан src/chat/ с chat_event.h/c — система нотификаций
- Перенесены chat_core, chat_msg, chat_channel, chat_profile, chat_status
- Перенесены chat_sync, member_sync, merkle_sync, db_sync
- gui_bridge_post() заменён на chat_event_post() с возможностью регистрации обработчика
- chat_event_set_handler() — chatgui регистрирует мост в gui_bridge
- Без обработчика: события логируются через DEBUG_DEBUG + log_dump
- chat интегрирован в utun_instance.c (при db_path в конфиге)
- Обновлены include-пути, Makefile.am, CMakeLists.txt
- Из chatgui удалены старые transport/ файлы, обновлён utun_node.cpp
После cascade_from в db_handle_send_data оповещаем ВСЕХ пиров
с synced_pos >= fix_from о том что цепочка изменилась — откатываем
их synced_pos. Это устраняет неэффективность конкурентных sync:
раньше только пир-отправитель обновлялся, остальные видели устаревшие
хеши и обнаруживали ложное расхождение на следующем цикле.
- si_send_data_batch: неправильный SQL SELECT с несуществующей колонкой author
вместо node_id → ломал всю передачу данных в REFINE/SEND_DATA
- Добавлена проверка si_prep и возврат int с кодом ошибки
- Все вызовы si_send_data_batch проверяют возврат, при ошибке сброс sync_state
- db_handle_push: добавлен db_cascade_from после вставки (раньше chain_hash
не пересчитывался после PUSH)
- db_handle_refine: DEBUG_WARN при scnt==0 (fm за пределами диапазона)
- Добавлена публичная db_sync_chain_verify() для верификации цепочек
- test_db_sync: фаза 4 (divergence через разныe hashes + REFINE/SEND_DATA),
фаза 5 (PUSH без хвоста + cascade verification)
- Убран мёртвый код db_sync_on_new_conn
- chatgui: адаптивная раскраска пузырей исходящих сообщений
- Unified format: 'sync [SHORT_TBL:PEER_HI] <-/-> action: key_data'
- Each log line explains what the algorithm is doing and why
- Added batch progress tracking in dump-all and tail SEND_DATA loops
- Track dropped records (duplicates, bad_sigs, parse_fails) in SEND_DATA recv
- Show synced_pos transitions (old->new) in SEND_DATA and SYNC_DONE
- Show all launched syncs in PEER CHECK log
- Convert all silent returns to ERROR logs with [TBL:NODE] context
- Convert old WARN/ERROR formats to new sync [TBL:NODE] format
- Add sync_state reset details on ERROR and timeout paths
- db_verify_author_sig: show ed_pubkey and sig first bytes on failure
- SI_PEER: add sync_retry_count field
- db_handle_sync_done: limit re-initiate to 3 retries before giving up
- db_sync_initiate_sync / db_sync_on_conn_up: reset retry_count
- db_handle_init_sync: TRACE→INFO with peer_cnt/my_cnt/tp_offset
- db_handle_send_data: detailed batch info (range, remaining count)
- db_sync_initiate_sync: add table name to log
- db_sync_on_conn_up: add first table name to summary log
- cs_on_conn_up: call cs_on_peer_status_changed(peer,1) before return
in invite path (was skipped, leaving inviter at online=0 in DB)
- cs_handle_welcome: remove unconditional set_online(0) — online
status is managed by conn_up/conn_down, not by WELCOME
- cs_handle_welcome: add cs_post_channel_online() after WELCOME
to send PEERS_ONLINE to GUI when channel is fully loaded
- cs_post_channel_online: add DEBUG log for PEERS_ONLINE event
- db_sync peer_check: suppress spam log when no peers connected
- utun_node.cpp: open topo_sqlite_db with FULLMUTEX + all PRAGMAs (WAL, FK, NORMAL sync, checkpoint)
- db_sync.c: reuse inst->topo_sqlite_db if available (shared_db flag)
- chat_core.c: reuse inst->topo_sqlite_db if available (shared_db flag)
- db_sync_destroy/chat_core_destroy: skip close on shared handle
- Only topo_groups_destroy closes the shared handle
- Fallback: each module opens own connection if topo_sqlite_db is NULL
When INIT_SYNC arrives for an unknown hash, scan all msg_* tables in
sqlite_master, compute SHA256 of channel_id, match the hash, and
auto-register via db_sync_instance_add. Fixes invite flow where
channel was created but db_sync wasn't notified.
- Add etcp_add_conn_status_cbk/remove/set — unified callback for all connections
- 4 statuses: NEW, UP, DOWN, DELETE — one subscription covers all connections
- Fire points in etcp.c: create→NEW, on_up→UP, on_down→DOWN, close→DELETE
- chat_sync.c: replace cs_on_new_conn+per-conn subs with cs_on_conn_status
- db_sync.c: same migration to instance-level callback
- cs_is_peer_online: use conn->links_up instead of link->link_status (was 0 during INIT handshake)
- db_sync: rename column author→node_id, open chats.db instead of sync file,
db_sync_instance_add takes table_name directly
- chat_core: remove CREATE TABLE msg_ch_*, remove on_db_sync_insert (74 lines),
remove fallback INSERT in submit_message, add minimal on_msg_inserted
for GUI notification only
- db_manager: update getMessages for new schema (parse JSON from data column),
fix ORDER BY to use author_signature instead of datahash,
update syncCursorNext to parse JSON
- si_register simplified: db_count from same table, no more msg vs sync gap
- Add sync_start_tb to SI_PEER, set in db_sync_initiate_sync
- peer_check_cb: after 15s timeout reset sync_state=0 with WARN
- db_handle_error: accept si, reset sync_state=0, DEBUG_ERROR with details
- DB_SYNC_SYNC_TIMEOUT=15 added to db_sync.h
- topo_sqlite_db moved from TOPO_GROUPS to UTUN_INSTANCE (direct access)
- topo_groups_set_sqlite_db() removed
- chatgui: chat_sync_connect_node → chat_core_connect_auto (direct ETCP, no conn_mgr)
- chatgui: chat_core_connect_from_invite saves to SQLite → auto connect (no BGP)
- chatgui: removed _on_node_updated BGP callback from member_sync
- chatgui: removed conn_mgr_get_status from status dump
- chatgui: removed all conn_mgr.h and topo_group.h includes
- chatgui: CONN_MGR_ERR_* constants → CC_* in chat_core.h
- removed unused cc_parallel_* code, conn_state_str/conn_type_str helpers
Root cause: db_handle_init_resp marked sync_state=2 when my_dh==peer_dh
without checking that all my records fit in the confirmed prefix.
If mc > tp+1 (I have records beyond the common prefix), the tail was
never sent — lost records.
Fix:
- Remove special-case 'sc==0 && my_dh==peer_dh' (redundant, same bug)
- In general dh_match: if mc > tp+1, send remaining records [tp+1,mc-1]
using the same inline loop pattern as 'peer empty' branch
Test: 3 phases covering all dh_match paths:
- Phase 1: B empty+late instance → dh_match at tp=0 (sc=0) → tail-send 49
- Phase 2: A=80 B=50 → dh_match at tp=49 (sc>0, sparse) → tail-send 30
- Phase 3: fresh instances → peer empty → send all 30
- db_sync_on_conn_up: per-instance synced/skipped counters
- db_sync_peer_check_cb: summary INFO of check results
- db_sync_instance_add: peers_found + peers_synced counts
- db_handle_init_sync/initiate_sync: enhanced with tbl name
- db_handle_init_resp: tp + my_mc in sync complete/dh match
- db_handle_send_data: range + SYNC_DONE details; stop reason when mc>pk
- db_handle_sync_done: both counts + datahashes in mismatch; dh in confirm
- db_sync_recv_cb: INFO when instance NOT FOUND (timing race)
- cs_handle_channel_info_resp: sync readiness after channel ready
- cs_handle_welcome: note about db_sync peer_check timer
- on_db_sync_insert: counter (#1-3 then every 10) with ch/n/ts/dh/ct
- utun_instance.h: add chat_connections queue (indexed by node_id, 8 bytes)
- chat_sync.c: queue_new in init, queue_free in destroy, queue_find_data_by_index in cs_find_conn_for_node
- chat_sync.c: remove from queue in cs_on_conn_down
- chat_core.c: add to queue in connect_from_invite and connect_auto
- chat_core.c: remove from queue in cc_parallel_cleanup and ca_cleanup
- db_sync.c: replace linear conn search with queue_find_data_by_index (db_sync_send_hash)
evgeny
Evgeny