Commit Graph
1659 Commits
Author SHA1 Message Date
Evgeny a7346c25e3 fix chatgui: members not showing (schema mismatch) and username showing JSON instead of name
- topo_node_sqlite_member_put: remove joined_at column from INSERT (not in server schema, duplicates join_ts)
- chat_core: rename peers table column name→userinfo to match server schema + add ALTER TABLE migration
- chat_sync: remove 3 redundant INSERT INTO nodes that wrote raw JSON userinfo into nodes.name, overwriting correct plain name from topo_node_sqlite_node_update_verified
2026-07-22 12:03:57 +03:00
Evgeny df809f251c ntp: use NTP-corrected time everywhere instead of unixepoch()/gettimeofday
- db_sync_next_timestamp(): ntp_time_get_us() via si->db_sync->inst
- topo_node_sqlite_node_put: +time_t now_sec param, replace unixepoch()
- topo_node_sqlite_node_update_verified: +time_t created_at param
- topo_node_sqlite_member_put: joined_at = join_ts (already NTP-corrected)
- All INSERTs (nodes, local_identity, accounts, peers):
  replace DEFAULT(unixepoch()) with explicit ntp_time_get_seconds()
- ON CONFLICT DO UPDATE: created_at = COALESCE(nodes.created_at, excluded.created_at)
- Updated callers: topo_group.c, chat_sync.c, chat_conn_mgr.c, chat_core.c
2026-07-22 11:39:10 +03:00
Evgeny b1cbd5acf9 topo_node_sqlite: dynamic idx_sql allocation instead of fixed 128-byte buffer 2026-07-22 09:18:54 +03:00
Evgeny b1bf27d023 fix Windows warnings: getpid, htole64, gettimeofday, socket_t mismatches, radix min 2026-07-22 09:08:19 +03:00
Evgeny b9a31b9fba proxy: fix Windows warnings - getsockopt cast, uasync_add_socket_t 2026-07-22 08:55:48 +03:00
Evgeny 090acc1f00 etc/rtt: UPSERT node_update_verified; gui: account delegate (rtt+nat subline), context menu member props (admin/supernode), cleanOrphanedPeers, refreshRtt timer 2s 2026-07-22 01:15:33 +03:00
Evgeny c367144776 etcp: rtt_last→ping_update_rtt раз в минуту; route_ping: ping_update_rtt на PING_RESP 2026-07-21 22:29:19 +03:00
Evgeny ad42f66df5 feat: PING/PONG RTT exchange + burst sender + TCP server; fix INIT peer_id offset
- etcp_connections: add flags byte to PING/PONG wire format (WANT_RTT/SEND_RTT)
- etcp_connections: add etcp_link_burst_start/check for BDP measurement
- etcp_connections: add tcp_server_on_link for STCP→ETCP bridge
- topo_node: add ping_request_cbk/ping_update_rtt for live RTT tracking
- topo_node: add last_ping_time field in connectivity
- topo_node_sqlite: extend DB schema for RTT data
- fix: INIT path read peer_id from offset 1 (not 2), PING offset stays 2
2026-07-21 20:50:14 +03:00
Evgeny 542694a0e9 fix: restore original source content after refactoring (accidental code changes removed) 2026-07-21 20:35:02 +03:00
Evgeny fcf3fb7f51 refactor: move transport (etcp/stcp/normalizer) to transport_layer/, routing (bgp/pings/topo*) to routing_layer/ 2026-07-21 20:25:41 +03:00
Evgeny 3dda701c67 schema: rm is_dm, rename name→userinfo (json_flat), fix join_msg format
- Remove is_dm column from channels table (dm detection by member count)
- Rename member name column to userinfo — extensible json_flat format
- Add json_flat.c/h library for compact JSON serialization
- Fix join_sig message format: use ch_x25519+ch_ed25519 instead of channel_id string
- Remove name from join_msg body (now in userinfo via json_flat)
- Update chat_core, chat_sync, member_sync, db_manager to new format
- Add topo_strategy.txt — архитектурные заметки по топологии
2026-07-21 00:11:50 +03:00
Evgeny 333acb2fe5 ntp_node_time: учёт пол-RTT в корректировке времени
offset = t4 - t_sender_us - avg_rtt/2
RTT усредняется по всем линкам соединения (rtt_last в 0.1ms).
2026-07-20 01:19:33 +03:00
Evgeny f83efda54e ntp_node_time: исправлен знак offset — корректировка шла в обратную сторону
ntp_time_get_us() вычитает offset (local_us - offset_us), поэтому
offset_us должен быть local_raw - peer_time, а не наоборот.
Было peer_time - local_raw → corrected = 2*local - peer (ошибка).
Стало local_raw - peer_time → corrected = peer_time (верно).
2026-07-20 01:07:09 +03:00
Evgeny fdaaa41010 ntp_node_time: фикс ложных докладов о дрейфе часов
check_drift() вызывался с сырой разницей часов между узлами, а не
с изменением offset со временем. При разнице часов между машинами
в 229 секунд это давало спам WARN на каждый TIME_SYNC обмен.

Теперь: первый обмен сохраняет offset без проверки; последующие
сравнивают дельту и warn только при реальном дрейфе (изменении offset).
2026-07-20 00:53:53 +03:00
Evgeny 15ca0675c0 Безопасность: pubkey check при reusing линка + node_changed при конфликте адреса чужого конна 2026-07-20 00:47:38 +03:00
Evgeny ccd35074d8 Заменил find_link_index на ll_queue с хеш-индексом: универсальный ключ port(2)+addr(16)+family(1)=19 байт, IPv4 дополнен нулями 2026-07-20 00:21:12 +03:00
Evgeny 7a5df3e196 etcp: add node_changed callback chain + chatgui subscription
- etcp.h: add node_changed_cbks field to ETCP_CONN
- etcp_api.h/c: add etcp_conn_node_changed() + add/remove_node_changed_cbk
- etcp_connections.c: fire node_changed callbacks on pubkey mismatch
  (with DEBUG_WARN(GENERAL) logging source addr, peer_id, old/new pubkey)
- etcp.c: cleanup node_changed_cbks on connection close
- chat_conn_mgr.c: subscribe to node_changed — clear DB addresses,
  reset in-memory cache, close connections (deferred for current conn),
  post GUI_EVT_NODE_CHANGED
- gui_bridge.h: GUI_EVT_NODE_CHANGED=11
- gui_bridge_impl.cpp: handle GUI_EVT_NODE_CHANGED
2026-07-19 22:09:23 +03:00
Evgeny f6f7e12192 refactor: demote high-frequency debug logs (raw_recv/dispatch→TRACE, normalizer→DEBUG), NTP drift→WARN 2026-07-19 13:53:27 +03:00
Evgeny 42041dfe84 feat: log connection status changes at general=info level (NEW/UP/DOWN/DELETE) 2026-07-19 13:41:25 +03:00
Evgeny beed7a4e07 fix: INIT reset logic — reset only on session change or got_initial_pkt, always sync session_id 2026-07-19 13:18:42 +03:00
Evgeny 58df4d8571 ntp_node_time: переход с etcp_router на raw ETCP (P2P)
- Добавлен ETCP_ID_NTP_TIME=0x04 (raw ETCP packet ID)
- Удалён ETCP_RT_ID_NTP_TIME=0x12 (router-сервис)
- ntp_node_time.c: etcp_router_bind/send → etcp_bind/etcp_send
- send_time_sync() принимает ETCP_CONN* вместо dst_node_id
- recv_cb разбирает raw dgram вместо SVC_ROUTE_HDR
- Обновлены doc-файлы
2026-07-19 12:16:37 +03:00
Evgeny 2e7c32c893 fix(db): use INSERT ... ON CONFLICT instead of INSERT OR REPLACE for nodes table
INSERT OR REPLACE deletes the old row and inserts a new one,
resetting unspecified columns (online, update_ts) to DEFAULT 0.
This was wiping online=1 set by cs_on_peer_status_changed/member_sync_set_online.

Replaced 5 INSERT OR REPLACE INTO nodes with INSERT INTO ... ON CONFLICT
DO UPDATE SET — only updates the specified columns, preserving online status.

Files:
- chat_core.c:638 (invite flow)
- chat_sync.c:1045 (CHANNEL_INFO_RESP), 1193 (JOIN), 1449 (PEER_UPSERT)
- topo_node_sqlite.c:121 (BGP node_put)
2026-07-19 11:16:26 +03:00
Evgeny 501b636d19 topo_group: disable in chatgui; fix NAT detection for matching addresses
- Add utun_instance_set_topo_group_enabled() flag (default=1, chatgui sets 0)
- Move nat_detection create/destroy from topo_group.c to utun_instance.c (unconditional)
- NAT detection: simplify third_node search (any connected node)
- NAT detection: allow NULL topo_groups in receive_cbk and handlers
- NAT detection: handle NAT_INFO without topo_group (update socket directly)
- route_ping: make TOPO_GROUP optional in send_req_addr/handle_req
- topo_node: when NAT verified and address matches interface, set entry type to NAT instead of INTERFACE + socket_id|1
- topo_node: detect nat_type change in changed computation for rebuild
- test_nat_detection: fix socket_id check (|1 for NAT entries)
- chatgui: call utun_instance_set_topo_group_enabled(0)
2026-07-19 08:33:33 +03:00
Evgeny 208a0c8ae8 fix(chat_sync): peer online status not updated in invite flow
- cs_on_conn_up: call cs_on_peer_status_changed(peer,1) before return
  in invite path (was skipped, leaving inviter at online=0 in DB)
- cs_handle_welcome: remove unconditional set_online(0) — online
  status is managed by conn_up/conn_down, not by WELCOME
- cs_handle_welcome: add cs_post_channel_online() after WELCOME
  to send PEERS_ONLINE to GUI when channel is fully loaded
- cs_post_channel_online: add DEBUG log for PEERS_ONLINE event
- db_sync peer_check: suppress spam log when no peers connected
2026-07-19 07:54:06 +03:00
Evgeny ca6a27aadb BGP: 2 addr per socket (remove TOPO_ADDR_REAL), config IP priority, private IPs in SQLite, NAT addr != INTERFACE 2026-07-19 07:00:04 +03:00
Evgeny 4df7195a58 debug: add ADDR_SYNC logs for BGP node_put DELETE and member_sync recv ac 2026-07-19 00:09:46 +03:00
Evgeny 586d46f1e6 fix: WELCOME wire format — flags+update_sig parsing, peer_needed gap, invalid join_sig skip 2026-07-18 23:53:04 +03:00
Evgeny 39a376e255 fix: BGP и чат больше не затирают адреса друг друга в node_addresses
- BGP DELETE: +AND socket_id != 0 — не трогает чат-адреса (invite/JOIN/WELCOME)
- Чат invite DELETE: +AND socket_id=0 — не трогает BGP-адреса
2026-07-18 23:23:14 +03:00
Evgeny 20f5e6c953 sqlite: add cache_size/mmap_size/temp_store PRAGMAs to db_sync and topo_group — 159x insert speedup, fix test_chat_sync_stress timeout 2026-07-18 20:16:24 +03:00
Evgeny 29e2527b6b consolidate 3 SQLite connections into 1 shared handle
- utun_node.cpp: open topo_sqlite_db with FULLMUTEX + all PRAGMAs (WAL, FK, NORMAL sync, checkpoint)
- db_sync.c: reuse inst->topo_sqlite_db if available (shared_db flag)
- chat_core.c: reuse inst->topo_sqlite_db if available (shared_db flag)
- db_sync_destroy/chat_core_destroy: skip close on shared handle
- Only topo_groups_destroy closes the shared handle
- Fallback: each module opens own connection if topo_sqlite_db is NULL
2026-07-18 17:33:29 +03:00
Evgeny 93665c50f0 etcp: remove dead set_*_cbk functions, use add/remove only 2026-07-18 16:52:43 +03:00
Evgeny ef1922327a rename ready_cbks → init_cbks (connection init, not fully ready)
ready_cbks/etcp_conn_set_ready_cbk/add_ready/remove_ready → init_cbks/..._init_cbk
ca_ready_cb → ca_init_cb
cm_direct_ready_cb → cm_direct_init_cb
cm_reverse_ready_cb → cm_reverse_init_cb
ntp_node_on_conn_ready → ntp_node_on_conn_init
connect_ready_cb → connect_init_cb

init accurately describes when conn->initialized=1, before keepalives establish UP.
Also fire etcp_fire_conn_status INIT at this point instead of create time.
2026-07-18 16:38:24 +03:00
Evgeny 4188afb3c5 db_sync: lazy-register channel instance on INIT_SYNC when not pre-registered
When INIT_SYNC arrives for an unknown hash, scan all msg_* tables in
sqlite_master, compute SHA256 of channel_id, match the hash, and
auto-register via db_sync_instance_add. Fixes invite flow where
channel was created but db_sync wasn't notified.
2026-07-18 16:08:47 +03:00
Evgeny a510cb0018 auto_connect: add DB dump diagnostics (node_addresses state, node existence, db path+status)
- chat_core.c: dump node_addresses total, per-node counts, nodes lookup before addr query
- topo_group.c: log db path, rc, and db pointer after sqlite3_open
2026-07-18 15:47:05 +03:00
Evgeny 3aadaa3396 fix: sqlite3_open wrong path in topo_group.c (db_path vs db_path/chats.db), keep separate connections for perf
- topo_group.c: open {db_path}/chats.db instead of {db_path} — was opening a directory
- chat_core.c: remove inst->topo_sqlite_db = g_cc.db — keep topo_sqlite_db as separate
  uTun-only connection without FULLMUTEX, avoiding contention with GUI reads
2026-07-18 15:41:02 +03:00
Evgeny aa1ed006d5 etcp: instance-level conn status callbacks (NEW/UP/DOWN/DELETE), fix member online in chatgui
- Add etcp_add_conn_status_cbk/remove/set — unified callback for all connections
- 4 statuses: NEW, UP, DOWN, DELETE — one subscription covers all connections
- Fire points in etcp.c: create→NEW, on_up→UP, on_down→DOWN, close→DELETE
- chat_sync.c: replace cs_on_new_conn+per-conn subs with cs_on_conn_status
- db_sync.c: same migration to instance-level callback
- cs_is_peer_online: use conn->links_up instead of link->link_status (was 0 during INIT handshake)
2026-07-18 14:51:01 +03:00
Evgeny e71cae16c1 db_sync: unified msg table — one DB, one table per channel
- db_sync: rename column author→node_id, open chats.db instead of sync file,
  db_sync_instance_add takes table_name directly
- chat_core: remove CREATE TABLE msg_ch_*, remove on_db_sync_insert (74 lines),
  remove fallback INSERT in submit_message, add minimal on_msg_inserted
  for GUI notification only
- db_manager: update getMessages for new schema (parse JSON from data column),
  fix ORDER BY to use author_signature instead of datahash,
  update syncCursorNext to parse JSON
- si_register simplified: db_count from same table, no more msg vs sync gap
2026-07-18 13:50:35 +03:00
Evgeny 7c2798ec0e debug: add diagnostic logging for ETCP recv/INIT/dispatch paths
- etcp_connections.c: raw_recv log, INIT conn lookup details, insert_link DUP details
- etcp_api.c: dispatch id/cb logs with DROPPED warning when no handler
- chat_sync.c: etcp_send rc log, fix ac_result_cb to always clear ca_state
2026-07-18 13:04:10 +03:00
Evgeny 93871c592b db_sync: timeout for stuck sync_state=1, handle ERROR properly
- Add sync_start_tb to SI_PEER, set in db_sync_initiate_sync
- peer_check_cb: after 15s timeout reset sync_state=0 with WARN
- db_handle_error: accept si, reset sync_state=0, DEBUG_ERROR with details
- DB_SYNC_SYNC_TIMEOUT=15 added to db_sync.h
2026-07-18 10:52:54 +03:00
Evgeny de4099a26b logging: cleanup connection establishment logs
- Fix wrong categories: BGP/DEBUG -> ETCP for connection lifecycle events
- Fix level: WARN -> DEBUG for normal out-of-order pkts before init
- Fix level: INFO -> DEBUG for per-packet TX DATA spam
- Remove duplicate socket init logs (GENERAL duplicates of ETCP)
- Add missing INFO: INIT_RESPONSE received (client), INIT_RESPONSE sent (server)
- Unify all connection establishment messages under ETCP category
- Remove redundant DEBUG address prints in INIT send (info already in INFO msg)
- PING/PONG operations: BGP -> CONNECTION
- DIRECT IP detection: BGP -> NAT
2026-07-18 10:35:31 +03:00
Evgeny 4b7c7738de debug: add DEBUG_CATEGORY_DB_SYNC (26) + TRACE logs for db_sync, merkle_sync, chat_sync, member_sync 2026-07-18 03:19:46 +03:00
Evgeny 9e34e9f838 fix chatgui: self in member list, online status on reconnect, lb spam
- cs_handle_welcome(): add self to local peers_<ch_id> so
  joiner sees own account in channel member list
- cs_on_conn_up(): post CHANNEL_PEERS_ONLINE after reconnect
  so GUI shows connected status (was only posted on conn_down)
- etcp_loadbalancer: DEBUG_WARN -> DEBUG_DEBUG for no suitable
  link: tmr:wait — normal shaper behavior, not a warning
2026-07-18 02:55:10 +03:00
Evgeny 3e0b9483b7 fix chatgui: channel persistence & peer list after restart
- chat_core_sync_my_addresses(): remove private IP filter (192.168, 10.x, etc.)
  so LAN addresses are stored for direct connections
- topo_node_sqlite_channel_peers_all(): include peers without addresses
  in WELCOME message (was skipped entirely, causing empty peer list)
- chat_core_create_channel(): pass real local addresses to member_sync_put()
  instead of NULL,0
- chat_core_init(): log number of channels loaded from DB on startup
2026-07-18 02:46:53 +03:00
Evgeny c594375023 fix: derive node_id from pubkey instead of privkey
sc_derive_node_id() hashed the private key, while invite links
(joindialog.cpp) hashed the public key — giving two different
node_ids for the same keypair.

Removed sc_derive_node_id(), kept only sc_derive_node_id_from_pubkey().
All callers updated to use pubkey-based derivation.
2026-07-18 00:41:38 +03:00
Evgeny 6f4025e420 refactor: move sqlite3 db from topo_group to UTUN_INSTANCE, remove conn_mgr/topo_group deps from chatgui
- topo_sqlite_db moved from TOPO_GROUPS to UTUN_INSTANCE (direct access)
- topo_groups_set_sqlite_db() removed
- chatgui: chat_sync_connect_node → chat_core_connect_auto (direct ETCP, no conn_mgr)
- chatgui: chat_core_connect_from_invite saves to SQLite → auto connect (no BGP)
- chatgui: removed _on_node_updated BGP callback from member_sync
- chatgui: removed conn_mgr_get_status from status dump
- chatgui: removed all conn_mgr.h and topo_group.h includes
- chatgui: CONN_MGR_ERR_* constants → CC_* in chat_core.h
- removed unused cc_parallel_* code, conn_state_str/conn_type_str helpers
2026-07-18 00:10:06 +03:00
Evgeny 52587c4f2b move ETCP_ID_NAT_DETECTION to etcp_api.h alongside other ETCP_IDs 2026-07-17 21:32:48 +03:00
Evgeny b238089678 refactor(nat_detection): remove BGP dependency, auto trigger on conn_up, clean API
- find_third_node: use nd->inst->connections instead of group->senders_list
- on_conn_up callback auto-calls trigger_checks (no BGP handshake needed)
- Remove group param from link_ready, request_check_all — nat_detection self-sufficient
- Remove trigger_checks + route_ping_cancel_for_conn from public API (internal only)
- topo_group.c: removed last nat_detection_trigger_checks call — completely decoupled
- Header: 4-line summary, no duplicate function list, 7 public functions
2026-07-17 21:23:40 +03:00
Evgeny 90363650e3 refactor(nat_detection): self-sufficient module with own ETCP_ID (0x02) and docs
- Assign ETCP_ID_NAT_DETECTION = 0x02, etcp_bind in create, etcp_unbind in destroy
- nat_detection_receive_cbk: own packet dispatcher (PING_REQ/RESP, NAT_INFO/CHECK_REQ)
- 4 handle_* become static internal, send_nat_check_req stays static (dead code)
- Rename TOPO_* -> NATDET_*: all subcommands, structs, message types
- Remove all NAT/PING constants and structs from topo_group.h
- Remove dispatch lines and subcmd_name cases from topo_group_receive_cbk
- API: clear sections (lifecycle/internal service/control/test helpers)
- trigger_checks: marked with IMPORTANT note about call order (must be after add_to_senders)
2026-07-17 20:45:49 +03:00
Evgeny 910cac8dce refactor: separate NAT detection from topology into standalone nat_detection module
- Extract NAT detection logic (~250 lines) from topo_group.c into new nat_detection.c
- NAT_DETECTION struct owns ping_pending, next_ping_req_id, allow_nat_check_local (removed from TOPO_GROUP)
- route_ping.* signatures changed: TOPO_GROUP* -> NAT_DETECTION* (+ TOPO_GROUP* where needed)
- topo_group.c now only dispatches BGP subcommands to nat_detection_* handlers
- UTUN_INSTANCE gets struct NAT_DETECTION* nat_det field (init/destroy in topo_groups_init/destroy)
- External callers (etcp_connections, etcp, control_server) call nat_detection_* directly
- nat_detection_cancel_for_conn / nat_detection_send_nat_info / nat_detection_link_ready / nat_detection_request_check_all
- route_ping_destroy_pending properly implemented (was dead code)
- TOPO_SUBCMD_PING_REQ/RESP moved from route_ping.h to topo_group.h (all BGP subcommands in one place)
- Tests updated: test_nat_detection, test_route_ping - both PASSED
- Build: auto(CORE_SOURCES+libutun) and cmake(chatgui/libutun)
2026-07-17 19:49:54 +03:00
Evgeny 0381f4df89 Убраны TCP_TIMESTAMPS и TCP_KEEPALIVE из lwip_tcp_opts.h (не используются, конфликтовали с ws2ipdef.h на Windows) 2026-07-17 17:49:15 +03:00