134 Commits
Author SHA1 Message Date
evgeny 6dc017337d chat: fix UAF в member list trampoline (req->inst после u_free) 2026-09-15 21:52:46 +03:00
evgeny 73fe99b33f dm: прямой обмен сообщениями (DM) + soundtouch + архитектура звонков
- src/dm/: dm_core/dm_crypto/dm_mailbox — прямой обмен сообщениями (DM)
- lib/soundtouch-master/: vendored SoundTouch (изменение темпа/высоты аудио)
- doc/dm_arch.md, doc/chat_call_arch.md: архитектура DM и звонков
- tests: test_dm.c, test_dm_e2e.c
- chat: рефакторинг chat_* модулей, member_sync, invite_build
- chatgui/chatgui-android: синхронизация с новым API чата
- lib/u_async, debug_config, utun_instance: сопутствующие изменения
2026-09-15 21:09:17 +03:00
evgeny 761f64cf76 chatgui-android: отображение REALITY-линков (TCP+REALITY) в статусе узлов и сокетов 2026-09-15 12:18:22 +03:00
evgeny 27b591daee reality: использовать NTP-время вместо time(NULL) в анти-реплей проверке timestamp
Часы colo2 отстают на ~52с (ntpd выключен), что ломало reality auth
(diff < -30с = time_window). Теперь клиент и сервер передают
ntp_time_get_seconds() в reality_client_hello_build_at /
reality_server_hello_build_at вместо сырого системного времени.
2026-09-15 05:39:28 +03:00
evgeny 481872d5db chat: invite-ссылки с несколькими адресами + ncd-адреса узлов (join) 2026-09-15 05:16:04 +03:00
evgeny d88dccfa19 etcp/stcp: обрыв etcp_link->tcp_link при закрытии stcp_link (fix UAF) 2026-09-15 00:25:42 +03:00
evgeny af382403ec tun_freebsd: reuse existing TUN interface on SIOCIFCREATE EEXIST 2026-09-14 23:17:59 +03:00
evgeny 952504d8eb etcp: убрать ложный reinit по gop (got_initial_pkt)
reinit должен происходить только при смене reset_id пира
(etcp_conn_apply_peer_reset_id). gop-проверка 'dirty vs clean' на догоняющем
TCP-линке давала ложный reinit и рвала соединение (JOIN_INFO_RESP терялся).
Удалён peer_got_initial_pkt из stcp_link (стал не нужен).
2026-09-14 22:46:41 +03:00
evgeny d58d4d39c4 chat join: запуск JOIN_INFO_REQ при уже поднятом соединении + добавление джойнера в группу
- cm_invite_trampoline: если уже подключены к connection-узлу — cs_on_conn_up
  повторно не сработает, запускаем join-хендшейк напрямую (cs_start_channel_join)
- chat_join_process_request: после верификации ключа сразу добавляем джойнера
  в CHAT-группу (topo_group_new_conn), чтобы BGP подключился
2026-09-14 22:46:41 +03:00
evgeny 910cfeac20 chatgui: настройка звука уведомления на канал (sound_on_message) 2026-09-14 21:12:53 +03:00
evgeny fc2262912a chat_join: верификация join-ключа на инвайтере (REQUEST_FWD)
Инвайтер добавлял мембера без проверки join-ключа — любой известный узел
мог самопригласиться в канал через REQUEST_FWD напрямую владельцу.
register_key теперь всегда хранит выданный ключ локально,
process_request отклоняет запрос с неизвестным/чужим ключом.
Тесты: unknown join_key rejected + регистрация ключа в t_process_request.
2026-09-14 21:12:53 +03:00
evgeny 75e39d2fee chat_join: модуль join-протокола, invite-ссылки v2, синхронизация мемберов 2026-09-14 19:57:27 +03:00
evgeny a01a4ddba4 fix: UAF in _on_member_sync_done (channel_cache freed by refresh)
cs_flush_sync passed &cs->channels[i] as member_sync cb_arg, but the 30s
cs_refresh_channels frees/reallocs cs->channels[], leaving the arg
dangling when _session_done later fires the callback. Re-find the channel
by the (stable) ns id instead of trusting arg.
2026-09-14 15:19:07 +03:00
evgeny 9ee61dce6c fix: ncd handle UAF in topo_recovery and topo_group_invite callbacks
topo_recovery_callback: detach cb on UP handoff and close handle on
DOWN/TIMEOUT so no handle with cb_arg==ctx outlives ctx (ASAN
heap-buffer-overflow in topo_recovery_callback).

tgi_to_channel_cb: force_close handle on NCD_EVENT_TIMEOUT so a late UP
cannot fire into the freed tgi_conn_ctx.

Also: android chatgui reverseLayout scroll + msg order/stable pagination.
2026-09-14 15:10:04 +03:00
evgeny fc3295ce12 nat_detection: гасить NAT-чек при закрытии линка через link-status коллбэк (UAF na->link) 2026-09-14 14:37:34 +03:00
evgeny e0f5afbff8 stcp: гасить on_close_cb/close_arg в stcp_link_close; keepalive: снапшот+ревалидация link до/после каскада (UAF) 2026-09-14 14:27:38 +03:00
evgeny da329d0479 merkle_sync: ms_outq_drain_cb — сохранять e->len до etcp_send (UAF); документировать ownership etcp_send 2026-09-14 13:49:12 +03:00
evgeny 1f8ccabdbf etcp: tcp_link_close_cb — сохранять link поля до каскада (UAF local_link_id) 2026-09-14 13:39:07 +03:00
evgeny 187bcd80b9 uasync: ре-валидация node перед последующими коллбэками (счётчик cb_called) вместо снапшота 2026-09-14 12:56:52 +03:00
evgeny 836c715299 uasync: снапшот полей socket_node в poll()/select() — защита от realloc во время коллбэков (UAF) 2026-09-14 12:47:35 +03:00
evgeny 369b253bdb chatgui-android: автоскролл чата вниз при открытии клавиатуры и своих сообщениях 2026-09-14 12:14:44 +03:00
evgeny d78d652bf5 shutdown: закрывать shared sqlite (topo_sqlite_db) в самом конце utun_instance_destroy (UAF при teardown) 2026-09-14 11:47:45 +03:00
evgeny 884f36c2d4 ncd: cancel connect/fin_wait таймеры в ncd_deferred_close (UAF fin_wait_timeout_cb) 2026-09-14 11:34:36 +03:00
evgeny f2f4a37344 routing: fix use-after-free в route_ping_cancel_for_conn (чтение request_id после u_free) + формат %08x 2026-09-14 11:29:31 +03:00
evgeny 11991e688c build: ASAN-сборка включает lib/libopus (зависимость src) 2026-09-14 11:20:12 +03:00
evgeny d730b06436 build: доработка build_bsd_full.sh (ASAN/hardened через clang, gmake fallback, -jN, FFmpeg) 2026-09-14 11:18:05 +03:00
evgeny d0dfa97a47 transport: SOCKS5-прокси для ETCP-линков (CONNECT + UDP ASSOCIATE + пинги) 2026-09-14 00:24:11 +03:00
evgeny 7c8d65db78 chat: локальное удаление канала (Android) + db_sync инстансы в куче 2026-09-13 22:08:33 +03:00
evgeny ddfffad1ed conn: сопоставление сокетов по ролям (direct/nat/local) + hairpin; NTP-детект оффлайна
- sock_match: единая классификация сокетов (DIRECT/NAT/LOCAL) и адресов (PUB/PRIV/UNREACH)
- eim приравнен к direct; на приватный адрес — только с приватного интерфейса;
  оба EIM за одним NAT (одинаковый публичный IP) — только через LAN
- topo_node: LOCAL-сокеты рекламируются (INTERFACE), private скрыт,
  NAT-адрес не рекламируется для strict и при оффлайне
- ntp: флаг reachable по циклам + пере-анонс nodeinfo при смене доступности
- cm_direct_add_links / ncd_create_links / conn_match_candidate_sockets на sock_match
- test_sock_match: unit-тесты классификации и сопоставления
2026-09-13 16:48:00 +03:00
evgeny ff53c735dd chat: превью медиасообщений в списке каналов (Android + desktop) 2026-09-13 15:55:33 +03:00
evgeny 07ef7d9a01 chat: живой счётчик медиакеша + сверка + DESC-проход докачки/вычистки 2026-09-13 14:49:57 +03:00
evgeny 1eb0566cb5 chat: enforce storage_total_size + media orphan cleanup; async_dns: fix use-after-free
- media cache: удаление сирот на старте (сверка ФС с сообщениями) + лимит
  storage_total_size (удаление самых старых файлов, только старше backfill_days)
- ma_dir_walk(): кроссплатформенный рекурсивный обход каталога (POSIX/Windows)
- async_dns: первый драйв slow-пути отложен через call_soon (исключён
  double-free при синхронном завершении запроса); cancel унифицирован
- chat: видимый WARN 'media too large' + честный счётчик started/skipped
2026-09-13 03:11:10 +03:00
evgeny 7bba39ddcb nodeinfo: 2 адреса на сокет (до/после NAT) + типы адресов local/public/nat/nat_eim/nat_strict
- topo_node: сокет рекламирует INTERFACE (до NAT) + NAT (после NAT, EIM);
  strict/direct отдельным адресом не передаются
- topo_node_sqlite: addr_type_classify различает NAT_VERIFIED_*; добавлен
  ADDR_TYPE_NAT (тип неизвестен), fallback больше не даёт DIRECT по IP
- chat_member: бейдж NAT-типа по приоритету (public>nat_eim>nat>nat_strict>local)
  вместо MIN(addr_type)
- GUI (desktop+android): адреса мембера с socket_id и ролью (local/public/nat/nat_eim/nat_strict)
2026-09-13 01:05:25 +03:00
evgeny a823898379 chat/media: докачка недостающего медиа после стартовой синхронизации + HAVE_BLOCK анонс локальных блоков 2026-09-12 21:28:08 +03:00
evgeny e636c3a420 docs: доступ на узел colo2 2026-09-12 21:24:04 +03:00
evgeny 5ee6bf53d8 conn_mgr: ping активных кандидатов с noop-колбэком вместо NULL (bad args) 2026-09-12 21:07:24 +03:00
evgeny c6118280bd conn_mgr: не удалять как stale кандидатов с активным соединением 2026-09-12 20:00:48 +03:00
evgeny 265008961a conn_mgr: кандидаты-посредники из topo_node-списка (активные соединения); гейт md_super_connect по is_supernode; откат ping→best_candidates 2026-09-12 19:38:48 +03:00
evgeny 2045d4a169 media: восстановить conn_mgr_open (DIR/REV/IND) + популяция best_candidates из RTT + watchdog не трогает CONN-блоки 2026-09-12 18:47:01 +03:00
evgeny 905fde5a27 media: BLOCK_REQ транзитом — убрать требование прямого соединения (NAT-узлы качают через роутер) 2026-09-12 18:27:23 +03:00
evgeny 0055c2c6fe media: фикс детекта суперузла (json_flat_get), backfill осиротевших медиа, fallback на автора при пустом QUERY_RESP 2026-09-12 18:19:47 +03:00
evgeny 5a83aaa786 opencode: подключить aiterm-mcp (интерактивный терминал для отладки) 2026-09-12 16:19:09 +03:00
evgeny ce12f058d9 tcp_io: fix error_cb busy-spin on graceful EPOLLHUP; socks_proxy: close local socket on both FINs
- error_cb (tcp_io): при graceful HUP (SO_ERROR==0) после уже прочитанного FIN
  сокет оставался в epoll и level-triggered EPOLLHUP крутил цикл на 100% CPU.
  Теперь сокет закрывается через tcp_conn_push_close.
- socks_proxy on_fin_cb: при обоих обменянных FIN локальный сокет закрывается
  (send_close + push_close), симметрично exit-стороне (tcp_proxy_server).
2026-09-12 16:08:14 +03:00
evgeny 56d0928632 etcp: wire-MTU кап 2048→1500 (согласовано с TUN_MAX_PACKET_SIZE), enc_buf привязан к MTU; тест multilink: per-link проверка трафика с ретраем через keepalive-тики 2026-09-12 13:53:55 +03:00
evgeny 13e2697f80 test_etcp_link_stress: переписан — 5 вариантов добавления линков (V1/V2, коллизии V3a/b/c), kill/failover и инвариант «нет реинита при живом линке»; фикс stack buffer overflow в etcp_encrypt_send (enc_buf 1600→2048) 2026-09-12 13:31:37 +03:00
evgeny 42a94aa916 BGP: фикс владения адресами в реестре (мульти-группа), проброс group_id в сервис (conn_mgr per-group), membership-фильтр create_group; тест media через chat-группу 2026-09-12 02:13:34 +03:00
evgeny 6c9a47b201 media_delivery: диагностика маршрутизации md_send/md_dl_send/router_deliver (conn, subcmd, no-route) + тест test_media_delivery_chat 2026-09-11 23:48:31 +03:00
evgeny d7bb58c6fb chat: автоподключение к узлам группы (вкл/выкл) + ручной connect к узлу (desktop+android); фикс reinit при QR-join 2026-09-11 19:46:02 +03:00
evgeny 9ef501a2fb etcp_connections: собрать forward-объявления в блок и добавить пояснения к функциям 2026-09-11 18:13:48 +03:00
evgeny d95048160d etcp: закрыть UAF в клиентской ветке tcp_link_close_cb
etcp_on_link_down может каскадно закрыть conn и освободить link;
использовать сохранённый etcp и проверять state!=2 перед reconnect
2026-09-11 17:46:38 +03:00
evgeny be1f08b43e etcp: защита от UAF в start_keepalive_timer/enter_ready_tcp + диагностика указателей
- start_keepalive_timer: guard !link->etcp->instance->ua (SIGSEGV на 80 строке)
- etcp_link_enter_ready_tcp: return при etcp->state==2 после reinit/apply_reset/conn_ready
- tcp_server_on_link/enter_ready_tcp: DEBUG_CATEGORY_DEBUG с адресами link/etcp/inst/ua
- topo_node: убрать лишние DEBUG_CATEGORY_DEBUG (chain_rtt)
2026-09-11 17:13:35 +03:00
evgeny 7de636be6e topo: убрать ALTER-миграции, backfill deleted из adm_tags 2026-09-11 16:48:15 +03:00
evgeny b7da6547c1 connection: логировать только connect/reconnect/disconnect; фиксы конфига чатгуи
- категория connection теперь только жизненный цикл соединения; INIT/PING/keepalive
  переклассифицированы в ETCP/SOCKET/KEEPALIVE/REALITY/ROUTING, удалены дубли
- settingsdialog: устранён дубль my_node_name в конфиге (двухпроходная запись + только при изменении)
- storagesettingspage: канонические ключи storage_total_size/storage_unit_size с суффиксом M/G
- audiodevicesettingspage: compressor_max_gain_db min 0→1
- etcp_connections: %s-баг в логе TCP link (addr → sockaddr_storage_to_str)
- docs/sample: [chatserver], актуализированы ключи хранилища
2026-09-11 15:24:56 +03:00
evgeny b7669eea36 Fix use-after-free of ETCP_LINK in keepalive_timer_cb 2026-09-11 13:21:35 +03:00
evgeny 70a18c8be8 etcp: чистка — убрать мёртвый peer_id/guard reset_id, поправить комментарий STCP_HS 2026-09-11 11:59:48 +03:00
evgeny f36ad8903a etcp: убрать session_id из INIT/STCP handshake, симметричная эпоха reset_id 2026-09-11 11:57:29 +03:00
evgeny ab6773a313 etcp: перепривязка/своп remote_link_id при INIT со сменённым адресом (NAT) 2026-09-10 21:20:26 +03:00
evgeny 311cee0ee1 Fix group node scan stall and UAF on server link teardown
- node_conn_direct_close: immediate conn cleanup when links are down
  (graceful fin_wait on a down conn left a lingering entry, so re-open
  in Phase3 stuck in 'pending' with no timer and scan never continued)
- tcp_link_close_cb: skip etcp_link_close if conn was already closed
  synchronously during etcp_on_link_down cascade (fixes use-after-free)
2026-09-10 19:01:21 +03:00
evgeny 5633903be0 stcp: закрывать принятые коннекты в stcp_server_destroy (UAF server_tlink->conn после tcp_socket_remove) 2026-09-10 17:53:14 +03:00
evgeny 6a674b3422 etcp: etcp_inflight_nullify_link читал entry->data вместо entry — last_link не обнулялся (UAF в bbr/retrans) 2026-09-10 17:53:14 +03:00
evgeny fdad786744 etcp: сбрасывать last_rr_link при закрытии линка в обеих ветках (use-after-free в loadbalancer) 2026-09-10 17:20:19 +03:00
evgeny ee8b2a4a45 chat: deleted-флаг выводить из adm_tags — member_block_put вместо деструктивного member_put, удалить member_put, фильтр deleted=0 в списках 2026-09-10 17:20:18 +03:00
evgeny 94136564ee platform_compat: выбирать глобальный IPv6 (2000::/3) вместо ULA при выборе адреса интерфейса 2026-09-10 17:20:17 +03:00
evgeny 327aac1c12 ncd: убрать временные DEBUG_CATEGORY_DEBUG логи (scan_conns, open/open_node dedup) 2026-09-10 16:01:57 +03:00
evgeny e75e3750aa ncd: синхронный close timed_out-conn чтобы не было коллизии линков при cycle_restart 2026-09-10 15:57:07 +03:00
evgeny d886a2ff84 Диагностика коллизии линков + детектор дубликата node_id + reality-конфиг autogen 2026-09-10 14:40:45 +03:00
evgeny c2882c533e reality: конфиг [client] для REALITY-клиентов (server_name/short_id/public_key/version/fingerprint) + TCP-линки через NCD 2026-09-10 12:33:16 +03:00
evgeny 898a0a4a66 reality: реклама reality-параметров сокетов через BGP и применение к исходящим TCP-линкам 2026-09-10 11:24:17 +03:00
evgeny edb3bb36e1 reality: TLS-камуфляж для STCP (ClientHello/ServerHello + релей на реальный сайт) 2026-09-10 01:11:26 +03:00
evgeny 0f308fdbd7 chatgui/chatgui-android: уведомления со звуком, флаги и прочие улучшения UI 2026-09-10 00:23:14 +03:00
evgeny 6c857ceab8 chat: обрабатывать CHANNEL_INVITE/INFO_REQ для ещё не созданного канала + GUI-результат invite-by-link
- chat_sync: не дропать CHANNEL_INVITE/CHANNEL_INFO_REQ при отсутствии CHAT-группы (group_id == channel_id, выводим ch_id напрямую) — приглашение по ссылке на сервер теперь доходит до autojoin
- inviteby: регистрация connect_result_cb, показ результата, автозакрытие при успехе
- joindialog: исправлен маппинг кодов результата (2=timeout, -1=not found, -7=not init)
2026-09-09 22:13:52 +03:00
evgeny 22da6bd832 chatgui: открывать SQLite-БД до auto_socket_init (переиспользование портов) 2026-09-09 21:29:27 +03:00
evgeny 738ac8928b tests: fix mkdtemp cross-platform in test_media_delivery_download 2026-09-09 20:32:48 +03:00
evgeny e3168d1e55 Удалить обмен метриками ETCP (ETCP_SECTION_METRICS)
Полностью убрана фича периодического обмена CSV-метриками с Ed25519-подписью:
- ETCP_SECTION_METRICS и парсинг/верификация секции
- struct etcp_metrics, etcp_metrics_add_*/start/stop и снапшот-таймер
- мёртвый API on_metrics_rcvd/metrics_user_ptr (никто не подписан)
- link->remote_ed25519_pubkey (заполнялся только на клиенте → верификация на сервере всегда падала)
- stats_dir в UTUN_INSTANCE
Общий sc_stream_sign_* (деривация Ed25519 из X25519) и INIT-передача обоих ключей сохранены.
2026-09-09 19:59:29 +03:00
evgeny 5b31a765c2 chatgui-android: auto_sockets=android — сокеты из конфига, лёгкое обновление без полного рестарта
- auto_sockets три-режимный: off/system/android; android берёт сокеты из [server] конфига
- auto_socket_reload_config: reconcile сокетов (remove→create) по имени+addr:port, сохранение портов
- etcp_create_server_socket вынесен из init_sockets; stcp_server_list_find_by_sock
- instance_lite_update_sockets + JNI nativeUpdateSockets + NativeLib.updateSockets
- NetworkChangeMonitor: updateSockets вместо restart; порты сохраняются при смене сети
2026-09-09 17:34:46 +03:00
evgeny c474c37284 chatgui/chatgui-android: просмотр изображений с зумом + инфо о медиа
- chatgui: просмотрщик изображений (ImageViewerWindow) с зумом/поворотом
- chatgui: отправка картинок как image с определением разрешения и типа
- chatgui: у видео-пузыря строка info (res·size·bitrate) и бейдж только длительность
- chatgui: контекстное меню медиа (сохранить в файл/скачать)
- chatgui-android: большие emoji-only сообщения, битрейт/разрешение у видео
- chatgui-android: сохранение неактивных интерфейсов как закомментированные секции
- chatgui-android: отправка картинок по расширению (jpg/png/gif/bmp/webp)
- photo_sender: content-type и расширение по типу файла
2026-09-09 16:18:26 +03:00
evgeny ca16b46404 chatgui-android: съёмка фото — JPEG, выбор разрешения, отправка аттачем, просмотр с зумом 2026-09-08 20:02:40 +03:00
evgeny c94acd892e chatgui: автозакрытие видеоплеера по завершении воспроизведения 2026-09-08 20:02:37 +03:00
evgeny 73e81032bc chatgui-android: видеосообщения — запись, выбор камеры, качество, подсветка
- C-слой: video_sender (content_type video/mp4, media_video=0 без FFmpeg)
- chat_msg_submit: duration/width/height для метаданных видео в теле сообщения
- JNI: nativeVideoSend + NativeLib/Repository/ViewModel
- VideoRecordScreen: CameraX VideoCapture, 4 пресета качества (разрешение+битрейт),
  выбор камеры (селфи/широкая/обычная/макро/теле по фокусу), тумблер подсветки,
  запись/пауза/отправить, лимит 60с
- InputBar: единая кнопка — короткий тап переключает voice/video, удержание — запись
- режим камеры/микрофона сохраняется в ViewModel
2026-09-08 18:38:18 +03:00
evgeny fe1726ebea chatgui-android: видеоплеер — зум, поворот, полноэкранный режим 2026-09-08 16:39:59 +03:00
evgeny aa166a2030 topo_group: RESYNC — пассивная сторона перезапускает BGP-обмен после флэпа
Сервер (пассивная сторона) на UP шлёт общий RESYNC, клиент по нему
заново анонсирует группы (JOIN_GROUP), которые ему положены по конфигу.
Чинит потерю маршрутов при одностороннем флэпе, когда клиент не замечает
разрыв и не перешлёт JOIN_GROUP повторно.
2026-09-08 16:39:55 +03:00
evgeny f28d3e91f7 media: рефакторинг загрузки блоков — конечный автомат, фейловер, watchdog простоя
- media_download: состояние блока (IDLE/CONN/REQ/RECV/WAIT/DONE/FAILED),
  кандидаты-держатели (MD_MAX_HOLDERS_PER_BLOCK), фейловер + ретраи (MEDIA_DL_MAX_ATTEMPTS),
  watchdog простоя (MEDIA_DL_STALL_TIMEOUT_TB) вместо фейкового времени
- media_delivery: настраиваемые dl_stall_timeout_tb / dl_max_attempts
- chat_msg: пометка ошибки скачивания в local_attrs ("st":"er") для UI
- tests: новый test_media_download_timeout (watchdog/фейловер), рефакторинг
  test_media_delivery_download/full
- chatgui-android: состояние 'error' у вложений/видео (иконка + tap to retry)
2026-09-08 06:53:48 +03:00
evgeny 089bbe5189 chatgui: запуск на XWayland (xcb) + ручной move/resize с пропорцией
- main.cpp: на Linux принудительно QT_QPA_PLATFORM=xcb (если не задан) — на Wayland
  композитор владеет позицией окна, на XWayland ручной move/resize работает полностью
- videoplayer_window: вернуть ручной move + ручной resize с пропорцией (один путь,
  без платформозависимого нативного startSystemMove/Resize)
2026-09-08 06:48:23 +03:00
evgeny 0dfb39cfcb chatgui: гибридный resize — нативный на Wayland (край тянется), ручной с пропорцией на X11/Windows 2026-09-08 02:47:59 +03:00
evgeny 3adf9871fb chatgui: вернуть ручной resize с пропорцией (двигает окно, край под мышью) + фикс перемотки
- resize: ручной setGeometry с живой пропорцией (якорь = противоположный край);
  drag — нативный startSystemMove
- перемотка: QProxyStyle SH_Slider_AbsoluteSetButtons=LeftButton вместо сломанного
  SeekSlider (sliderReleased не эмитился — seekMs не вызывался)
2026-09-08 02:39:06 +03:00
evgeny 645591bb64 chatgui: нативный move/resize (Wayland) + фикс замирания видео при перемотке
- окно: startSystemMove/startSystemResize вместо ручного move/setGeometry
  (Wayland игнорирует позицию окна) — драг и resize за левый/верхний край работают
- пропорции: снэп к videoAspect после нативного resize (дебаунс 150мс)
- перемотка: backpressure по m_buffered (декод не убегает вперёд и воркер живёт
  для seek), seek через av_seek_frame(-1, us), пропуск кадров с AV_NOPTS_VALUE
- SeekSlider: клик по шкале = прыжок к позиции
- крестик закрытия справа вверху; курсор контролов — стрелка
- test_video_engine: проверка seek
2026-09-08 02:04:18 +03:00
evgeny f2b34459c3 chatgui: баббл видео — размер + время m:ss (напр. 1.5MB · 0:45) 2026-09-08 01:47:32 +03:00
evgeny d09a6af52a chatgui: плеер — фикс drag, центр/fullscreen при открытии, resize за края с сохранением пропорций
- m_render WA_TransparentForMouseEvents — события мыши напрямую в окно (drag работает)
- открытие: видео влезает — натуральный размер по центру; больше экрана — fullscreen
- resize мышью за 4 края и 4 угла, пропорции сохраняются (якорь = противоположный край)
- нижний край доступен через eventFilter на панели контролов (пустые поля пропагируют)
- сброс fullscreen при повторном playFile
2026-09-08 01:31:12 +03:00
evgeny 76182d018f chatgui: QPointer-защита при генерации превью в фоне 2026-09-08 01:01:28 +03:00
evgeny 74b17685f7 chatgui: фикс синхронизации кадров (часы не убегают вперёд при быстром декоде) + smoke-тест движка
- presentTick: кадры всегда по мастер-часам, завершение только когда очередь пуста
- clockMs: по окончании аудио часы в конец (хвостовые кадры не теряются)
- fallback на системные часы, если аудио-устройство недоступно
- tests/test_video_engine: smoke-тест FFmpeg-декодера (opened/frames/seek)
2026-09-08 01:00:38 +03:00
evgeny 6542618927 chatgui: воспроизведение видео — FFmpeg-плеер + frameless окно
- videoplayer_engine: декод в worker-потоке (FFmpeg), A/V sync по аудио-курсору,
  seek с epoch-защитой от устаревших кадров
- videoplayer_window: окно без рамки, drag-перенос, dblclick/Alt+Enter fullscreen,
  Esc закрытие, timeline + play/pause/close, авто-скрытие контролов
- видео-баббл в списке: превью-кадр (jpg), play-оверлей, длительность, прогресс
- превью извлекается после скачивания (video_extract_thumbnail) в фоне
- mainwindow: синглтон-плеер, openVideoRequested → openVideo
- CMake: HAVE_FFMPEG для vibechat
2026-09-08 00:55:06 +03:00
evgeny e264b8aaa7 video: video_extract_thumbnail — превью первого кадра в JPEG (MJPEG) 2026-09-08 00:44:01 +03:00
evgeny 7f3b4f61f4 chat: dur/w/h в тело видео-сообщения (после транскода) 2026-09-08 00:41:43 +03:00
evgeny 1855c61275 video: отправка видео-сообщений (FFmpeg пробинг+транскод H.264/AAC MP4)
- src/video/: video_probe/video_get_info/video_transcode_start (FFmpeg, HAVE_FFMPEG)
- chat_core/chat_msg: media_video флаг, chat_core_submit_media_message, транскод перед index
- headless: команда attach с --video
- chatgui: диалог 'Как видео' + прогресс транскода + sendVideoAttachment
- FFmpeg в сборке (configure.ac, Makefile.am, chatgui CMake, build_nix.sh)
- debug: категория DEBUG_CATEGORY_VIDEO
2026-09-08 00:41:04 +03:00
evgeny c8bd5975ae chatgui-android: invite-ссылки с выбором узла входа в GUI + desktop dialog layout 2026-09-07 22:36:27 +03:00
evgeny ba557e2b9d chat: фикс 0:00 у длинных голосовых — b64_decode терял последний байт без padding + formatTime в мс 2026-09-07 22:36:23 +03:00
evgeny 1345f44e10 chat: invite-ссылки с выбором достижимого узла + tcp_io фикс half-close FIN
- Общий модуль invite_build: сборка utun://-ссылок, авто-выбор лучшего
  достижимого узла, список кандидатов (вынесено из android jni_bridge)
- chatgui: диалог invite с выбором узла (комбобокс), события INVITE_CANDIDATES /
  INVITE_LINK_READY через gui_bridge
- headless/chatcli: invite <ch_id> [node_id], команда invite_nodes
- tcp_io: error_cb отличает грациозный EPOLLHUP (SO_ERROR==0) от реальной ошибки —
  при приостановленном чтении FIN не роняет недослейные данные (фикс flaky half-close)
- run_test.sh: half-close echo на выделенном порту 19190 + отдельный DNAT (был недостижим)
2026-09-07 13:59:15 +03:00
evgeny a17a85a383 chat: deleted-аккаунты — колонка deleted, скрытие/зачёркивание, не подключаемся 2026-09-07 01:40:53 +03:00
evgeny 631a176944 chat: права админа + member tags (verified/deleted) + ncd revive
- chat_admin: передача канальных privkey другому узлу (encrypted+signed,
  ETCP_RT_ID_ADMIN_KEY), событие ADMIN_KEY_RECEIVED
- member_sync/bridge: tags verified/deleted, is_channel_owner, transfer_admin
  (android JNI + chatgui memberpropsdialog)
- node_conn_direct: оживление timed_out entry — обновление линков в существующем
  conn (убрать старые, добавить новые) + немедленная очистка не поднявшегося
  conn без fin_wait — фикс invite/reconnect
2026-09-07 00:53:41 +03:00
evgeny 6163f34ce1 etcp_router: эпоха reset_id на wire + master/slave — фикс потери пакетов при рестарте пира 2026-09-06 22:20:08 +03:00
evgeny 61c572cae4 etcp: reset_id эпоха ресета — фикс потери NODEINFO при флапе
Добавлен 64-битный reset_id (эпоха ресета) в INIT-фреймы и STCP-handshake:
- генерится один раз при создании conn, меняется только при фатальном ресете
  (normalizer desync / bad fragment size);
- при приёме чужого id slave принимает эпоху пира и ресетится, master держит свою;
- первый id принимается без ресета (свежие при init);
- «tcp peer clean» реинит гейтится по смене id — нет повторного сброса seq.

Это устраняет коллизию seq=1 (RX dup) из-за двойного реинита нескольких TCP-линков,
из-за которой NODEINFO пира терялся и member не возвращался online.

Также: chat member online теперь из topo_group (без БД/merkle), тест test_etcp_seq_collision.
2026-09-06 21:50:06 +03:00
evgeny f7c08f8f5e nat_detection: применять NAT-тип через topo_node_update_my_addresses с переподписью 2026-09-06 01:29:07 +03:00
evgeny 53bce51201 etcp_router: корректный handshake/restart без таймаут-петли
- START на первом пакете сеанса (seq=0), гейт отправки до первого ACK
- дедуп рестарта по payload init-пакета (ретрансмит START не триггерит рестарт)
- RST переинициализирует только отправку (без CLOSE_ALL), дедуп повторных RST
2026-09-06 00:33:45 +03:00
evgeny 232ad2de6b etcp_router: разделить служебные функции и API в хедере + чистка мёртвого кода
- etcp_router.h: 4 секции (протокол / внутренние структуры / API / служебные)
- переименование router_set_max_inflight -> etcp_router_set_max_inflight
- удалены неиспользуемые: input_q_count, waiter_register/cancel, conn_close_async
  (+ поля close_callback/close_callback_arg)
- docs: актуализировать backpressure (on_send_ready на send_q) вместо удалённого waiter API
2026-09-05 22:34:00 +03:00
evgeny 4b9b0009d8 chat: гейт авто-синхронизации db_sync до member_sync/pubkeys + member tags (verified/deleted) в chatgui 2026-09-05 20:38:22 +03:00
evgeny 01ea6913ae etcp_router: sess_id как локальный счётчик переустановок (не в протоколе) 2026-09-05 20:38:17 +03:00
evgeny 2d04bf941b etcp_router: вынести подпись/шифрование в route_crypto, одна функция отправки с флагами
- route_crypto_encode/decode: автономный пред/пост-обработчик пакета
  (encrypt-then-sign, стандартный Ed25519 вместо sc_stream_sign)
- одна функция отправки etcp_route_send/etcp_router_conn_send с bitmask mode
- формат доставки: [svc_id][src][dst][rx_flags][payload] (статус подписи/шифрования)
- ROUTER_INFLIGHT хранит финальный wire-пакет (чинит потерю подписи при ретрансмите)
- дедуп инициализации: router_conn_reset/free_queues
- учёт inflight через inflight_q count (seq присваивается на enqueue)
2026-09-05 17:40:43 +03:00
evgeny 7c95ddb09b chatgui: значок владельца канала (🔑) в шапке блока мемберов при наличии private key 2026-09-05 15:23:45 +03:00
evgeny 7817bb96aa etcp_router: START/RST handshake для переживания рестарта пира
Раньше START ставился один раз при первой отправке и терялся в ретрансмите,
а принимающая сторона молча авто-синкала rx_seq на любой первый пакет —
после рестарта пира/среднего узла seq рассинхронизировались (тихий data loss
или no-ACK close). Теперь:

- Клиент: флаг start_sent «новый» до первого ACK; пока он стоит, первый
  seq (0) шлётся со START, включая ретрансмит. Первый принятый ACK
  (tx_acked>0) сбрасывает флаг. При рестарте conn флаг снова «новый».
- Сервер: после рестарта peer_sync_done=0 («ждёт START»). Не-START пакет →
  отправка RST (через recv_conn). START → синк rx_seq + ACK → обычный режим.
- RST на принимающей стороне теперь сбрасывает локальное состояние
  (etcp_router_conn_restart), а не закрывает conn.

Тесты unit обновлены: первый data-пакет сессии инжектится со START.
2026-09-05 15:03:44 +03:00
evgeny 84015176b3 etcp_router: досылать потерянный ACK при дубле (рестарт среднего узла)
При рестарте транзитного узла ACK приёмника терялся и не пересылался:
router_schedule_ack выходит рано при rx_seq==last_sent_ack_seq, поэтому
пир ретранслил 17 циклов (~5с), роутер-conn закрывался и цикл повторялся
бесконечно — лечилось только рестартом клиента. Теперь при дубле
(ретрансмите пира) ACK досылается с троттлингом ROUTER_ACK_INTERVAL_TB.
2026-09-05 01:17:37 +03:00
evgeny 32725657dc chatgui-android: обработка скачивания голосовых (download state) + config: секция [chat] алиас для chatserver 2026-09-04 22:46:07 +03:00
evgeny 23fbdf4584 chat: запуск BGP-обмена при подключении узла к группе (invite + auto-connect) 2026-09-04 21:04:18 +03:00
evgeny 5b8c725c01 etcp_router: round-robin send + контроль заполненности send_q через waiter на send_input_q 2026-09-04 19:30:08 +03:00
evgeny 1a88ea88a8 topo_group: узлы меняются только по BGP (NODEINFO/WITHDRAW), удаление по link down через BGP-коллбэк 2026-09-04 17:57:37 +03:00
evgeny b0f149092f BGP: log node add/remove per group at info level 2026-09-04 16:45:33 +03:00
evgeny c049788c08 etcp_router: при no route выводить список узлов группы (<10) 2026-09-04 16:39:23 +03:00
evgeny ed2df9fe88 etcp_router: добавить кол-во узлов группы в лог no route 2026-09-04 16:25:21 +03:00
evgeny d092f649ac chatgui: пути db и лога относительные в конфиге, резолв к бинарнику в рантайме 2026-09-04 14:34:39 +03:00
evgeny 654202b0cb chatgui: debug-категории в секции [debug] построчно (как в utun) 2026-09-04 13:42:59 +03:00
evgeny 20eef8ad16 chatgui: лог по умолчанию в файл chatgui.log (debug_file) 2026-09-04 13:25:06 +03:00
evgeny 656da48e35 topo_group: асимметричный старт BGP (JOIN_GROUP) + вынос group_id/flags из TOPO_NODE
- UTUN: BGP инициирует только сторона-клиент ([client]) и явные etcp_connect;
  сервер добавляет пира по JOIN_GROUP (фикс обратного маршрута: сервер не
  создавал узел клиента из-за дедупа senders_list)
- topo_group_new_conn: conn_add_path до дедупа; handle_request_table без add_to_senders
- TOPO_NODE: убраны group_id/flags (shared-реестр), передаются в topo_node_serialize
- routing/control_server: явный TOPO_GROUP_UTUN
- CHAT: гейт по peers_* (get_member_channels) + пассивное add/remove в member_sync
- tests: тесты адаптированы (route_ping ждёт registry+v4_addrs)
- chatgui-android: voice_recorder_init в utun_bridge_voice_start (idempotent)
2026-09-04 13:21:24 +03:00
evgeny 7b73d99b5b chat: передавать group_id в бинарном виде в протоколе синхронизации
db_sync/merkle_sync/chat_sync шлют group_id (uint64 BE) вместо текстового
channel_id/namespace. Приёмная сторона находит группу/таблицу через
topo_groups_find(group_id) и требует существующую CHAT-группу. db_sync:
instance key заменён с SHA256(ch_id) на strtoull(ch_id)=group_id.
2026-09-04 09:02:46 +03:00
evgeny 326dfd7e2c media_delivery: маршрутизация всегда через CHAT-группу
media_download/медиа-ответы и суперузловая репликация теперь берут группу
из контекста (dl->group_id, group_id из пакета, peer->group_id) вместо
хардкода TOPO_GROUP_UTUN. node_props_changed получает channel_id.
SUPER_HELLO несёт group_id.
2026-09-03 23:24:50 +03:00
evgeny 55da8fd146 media_delivery: логирование отправки QUERY_RESP и провалов md_send 2026-09-03 21:59:57 +03:00
evgeny 01b61a457a etcp_router: fix ACK no-route — слать ACK через recv_conn
При асимметричной топологии topo_group_find_conn_for_node возвращает NULL,
ACK не уходит → ретрансмит-шторм → no-ACK close → inflight сбрасывается в 0.
Теперь ACK уходит через тот же коннект, с которого пришли данные (recv_conn),
с фолбэком на топологию. Довершает фикс из e08956e (deliver), который
аналогично не трогал router_send_ack.

Заодно удалён неиспользуемый ETCP_ID_ROUTE_ENTRY.
2026-09-03 20:05:59 +03:00
evgeny fbea9c3869 socks_proxy: fix client stall + non-blocking DNS via async_dns
- fix HTTP_STATE_RELAY == SOCKS_STATE_CONNECTING enum collision (relay data dropped)
- fix wrong waiter cancel (etcp_router_waiter_cancel -> etcp_router_cancel_send_ready)
- add backpressure retry timer, restore lost queue_resume_callback, defer FIN on pending tx_buf
- async_dns: /etc/hosts fast-path + hosts loading (avoid 4s DNS timeout for localhost)
2026-09-03 18:13:00 +03:00
evgeny f47ad7d87f tcp_proxy: fix throughput (tx_queue backpressure) + connect timeout + integration test
- Router: unified service delivery format [svc_id][src][dst][payload] (end-to-end src/dst from SVC_ROUTE header)
- tcp_proxy_client: replace single tx_buf drop-on-backpressure with bounded tx_queue + send_q waiter + retry timer (no data loss, bulk window reopen — removes 62.5ms delayed-ACK stall)
- tcp_io: configurable connect timeout (tcp_conn_set_connect_timeout)
- tcp_proxy_server: connect_timeout config option (ms, default 2000, 0=off)
- check.sh: run tcp_proxy_full integration test with sudo (skip if passwordless sudo unavailable)
- test_tcp_io: connect timeout unit test; docs (lwip_nuances.md)
2026-09-03 13:49:09 +03:00
evgeny acfaae2801 dns: заменить K&R-объявления указателей на функции на типизированные
gcc 15 (C23) трактует () как 'без аргументов', из-за чего lib/dns.c не собирался
(65 ошибок). struct dns_rr_i.sort и таблица dns_rrtypes[] переведены на явные
прототипы с typedef'ами и кастами указателей.
2026-09-02 23:59:46 +03:00
evgeny e08956e5dc etcp_router: fix NULL conn в deliver при отсутствии пути к узлу
router_handle_data_packet выбрасывал физический conn, а router_incoming_q_cb
восстанавливал его через topo_group_find_conn_for_node, который возвращает NULL,
когда у узла-источника нет пути в топологии. В итоге callback сервиса (routing,
conn_mgr) получал conn=0x0. Теперь физический conn сохраняется в rconn->recv_conn
и прокидывается в deliver.
2026-09-02 23:59:41 +03:00
evgeny 1db4fe1f01 build_linux_full.sh: автоустановка зависимостей и корректная обработка ошибок 2026-09-02 21:29:48 +03:00
evgeny 5d39083a6d auto_socket: пропускать интерфейсы без default-маршрута (опция auto_socket_skip_no_default_route, default=on) 2026-08-17 23:43:15 +03:00
evgeny f841a964aa chat: debug-логи на джойнер-стороне (channel_put, JOIN sent, WELCOME self-add) 2026-08-17 22:53:03 +03:00
jeka 10306f5613 chat: fix CHANNEL_INVITE invalid ch_sig — передавать владельца канала из БД, а не node_id отправителя 2026-08-17 21:16:44 +03:00
evgeny 7dee49ce63 android: мониторинг смены сети — NetworkChangeMonitor, перезапуск ядра при смене интерфейса 2026-08-17 21:03:05 +03:00
341 changed files with 38031 additions and 7133 deletions

No files matched your search

+66 -8
View File
@@ -25,6 +25,13 @@ This file contains essential information for AI coding agents working in the uTu
При поиске багов всегда проверяй достаточно ли отладочной информации. Если не достаточно - фокусируйся на том чтобы добавить нужную информацию в вывод.
Помни что отладка управляется фильтрами по категормяи, которые нужно проверить и при необходимости правильно настроить (в конфиге или в коде).
Для поиска выстараивай информативные логи. По которым максимально чётко понятно что происходит. Если какие-то места которые могут иметь отношение к ошибке в логах пропущены - это повод доработать логи и прогнать еще раз.
Всегда отладку выстраивай через доработку логов, пока точно и однозначно не будет видно в каком точно месте проблема.
По хорошей диагностике сразу должно быть понятно что происходит, в том числе точную причину проблемы. Если приходится догадываться или плохо понятно - это значит что надо улучшать диагностику.
Доствп на узел colo2: ssh -p44322 logs@colo2
## Quick Reference
**Repository:** uTun - Secure VPN tunnel with ETCP protocol
@@ -216,7 +223,7 @@ sc_obfuscate_pubkey(salt, peer_pubkey_bin, my_pubkey_bin, obfuscated_output);
### Debug Levels
`none < error < warn < info < debug < trace`
trace - логируем заходы в функции и важные ветвления внутри функций
debug - всё что нужно для понимание сути происходящего (состояний, важных переменных) алгоритма. тоесть по уровню debug мы должны видеть все важные внутреннии нюансы работы функции включая состояния и переменные.
debug - отладка: всё что нужно для понимание сути происходящего (состояний, важных переменных) алгоритма. тоесть по уровню debug мы должны видеть все важные внутреннии нюансы работы функции включая состояния и переменные.
info - сообщения для пользователя о работе. мы должны видеть в читаемом виде понятные и осмысленные сообщения о каких-либо значимых действиях с точки зрения логики работы модуля или приложения
warn / error - ошибки и предупреждения (аномалии). должны быть во всех ошибочных ветках. Молча вываливаться с ошибкой нельзя, все ошибки и предупреждения должны логироваться.
@@ -283,6 +290,9 @@ MEMBER_SYNC=27
- `nat_transport.c/h` - NAT transport layer (packet relay)
- `ntp_time.c/h` - NTP time synchronization
- `ntp_node_time.c/h` - Inter-node time synchronization
- `broadcast.c/h` - Широковещательная рассылка данных по topo-группе (UUID-дедупликация, TTL)
- `video/video.c/h` - Пробинг и транскодирование видео (FFmpeg, опционально; H.264/MP4 ≤720p)
- `dnsmasq/` - Архив dnsmasq-2.93.tar.xz для встроенной сборки
**Transport Layer (src/transport_layer/)**
- `etcp.c/h` - ETCP protocol implementation (inflight queues, retrans, ACK, RTT/jitter)
@@ -293,6 +303,8 @@ MEMBER_SYNC=27
- `etcp_dump.c/h` - ETCP дамп/декодирование пакетов
- `etcp_bbr.c/h` - BBR congestion control for ETCP
- `etcp_connect.c/h` - ETCP outbound connections
- `etcp_keepalive.c/h` - Keepalive-пакеты, адаптивный период, смена режима standby/normal (мобильные узлы)
- `auto_socket.c/h` - Автосоздание UDP+TCP сокетов на всех интерфейсах и линков к ETCP-соединениям (`auto_sockets=yes`)
- `pkt_normalizer.c/h` - Packet fragmentation/reassembly (packer/unpacker)
- `packet_dump.c/h` - Packet hex dump utility
- `dummynet.c/h` - Network emulator integrated into utun (for testing)
@@ -304,6 +316,9 @@ MEMBER_SYNC=27
- `stcp_link.c/h` - STCP линк-уровень
- `node_conn_direct.c/h` - Direct config-based connections
- `socket_monitor.c/h` - Socket state monitoring
- `reality.c/h` - REALITY-style TLS ClientHello/ServerHello камуфляж для STCP
- `reality_fingerprint.c/h` - Статические отпечатки (cipher suites, groups, ALPN) для REALITY-камуфляжа
- `reality_relay.c/h` - Релей неавторизованных клиентов на реальный HTTPS-сайт
**BBR (src/transport_layer/BBR/)**
- `bbr_v3.c` - BBR congestion control algorithm v3
@@ -323,6 +338,7 @@ MEMBER_SYNC=27
- `conn_mgr.h` + `conn_mgr_core.c`/`conn_mgr_indirect.c`/`conn_mgr_monitor.c` + `conn_mgr_priv.h` — Connection Manager (handle-based API, 3-phase DIR/REV/IND)
- `etcp_router.c/h` - ETCP router/multiplexer (мультиплексирование каналов, transit forwarding)
- `nat_detection.c/h` - NAT type detection
- `route_crypto.c/h` - Пред/пост-обработка SVC_ROUTE пакетов (encrypt-then-sign)
**Chat (src/chat/)** — децентрализованный P2P чат: сообщения и участники автоматически синхронизируются между всеми узлами канала без центрального сервера.
- `chat_core.c/h` - Главный модуль чата. Единственная точка входа из GUI: принять сообщение, создать канал, изменить настройку. Управляет БД и жизненным циклом всех chat-модулей
@@ -337,6 +353,12 @@ MEMBER_SYNC=27
- `chat_channel.c` - Создание и настройка каналов: готовит таблицы в БД, генерирует криптоключи, запускает подключение ко всем участникам (часть chat_core)
- `chat_msg.c` - Отправка и приём сообщений: запись в локальную БД, автоматическая рассылка всем участникам канала (часть chat_core)
- `chat_status.c` - Сбор диагностики: текущее время, активные соединения, типы NAT. Отправляется в GUI (часть chat_core)
- `chat_member.c/h` - Единая структура мембера для отображения в GUI (десктоп/headless/Android), фиксированная сериализация на проводе
- `chat_admin.c/h` - Передача канального приватного ключа (прав админа) другому узлу
- `chat_whisper.c/h` - Whisper speech-to-text транскрипция голосовых сообщений (опционально, асинхронно через media_async)
- `chat_headless_control.c/h` - TCP control-socket для headless chat CLI (JSON line protocol)
- `invite_build.c/h` - Сборка invite-ссылок utun:// (выбор лучшего узла + адреса)
- `invite_link.c/h` - Формат invite-ссылок utun:// для каналов (base64: version|password|channel_id|pubkey|addrs)
**Прокси (src/proxy/)**
- `socks_proxy.c/h` - SOCKS5 прокси
@@ -378,6 +400,13 @@ MEMBER_SYNC=27
- `opus_codec.c/h` - Opus audio codec wrapper
- `audio_compressor.c/h` - Audio dynamic range compressor
- `json_flat.c/h` - Flat JSON parser
- `strbuf.c/h` - Безопасный растущий printf-буфер (замена snprintf-цепочек)
- `async_dns.c/h` - Асинхронный (неблокирующий) DNS-резолвер A-записей поверх uasync, адаптер над libdns
- `dns.c/h` - Рекурсивный reentrant DNS-резолвер (libdns, MIT)
- `miniaudio.h` - Воспроизведение/захват аудио (single-header)
- `dr_mp3.h` - MP3-декодер (single-header, на базе minimp3)
- `liblmdb/` - Встраиваемый key-value store LMDB
- `libopus/` - Встраиваемый кодек Opus
- `wintun.h` - Wintun API header for Windows TUN driver
**Memory Pools in UTUN_INSTANCE:**
@@ -422,12 +451,12 @@ MEMBER_SYNC=27
## Config Rules
- В серверном конфиге только собственные ключи и нет секций `[client]`
- В клиентском конфиге есть собственные ключи и pubkey каждого сервера в секции `[client]`
- Chat-настройки хранятся в секции `[chat]`. Парсер вызывает `chat_setting_set(key, value)` для каждого ключа — неизвестные ключи вызывают ошибку. Пример:
- Chat-настройки хранятся в секции `[chatserver]` (принимается также `[chat]`). Парсер вызывает `chat_setting_set(key, value)` для каждого ключа — неизвестные ключи вызывают ошибку. Пример:
```ini
[chat]
[chatserver]
storage_autoload=1
storage_autoload_maxsize_mb=10
storage_maxsize_gb=1
storage_unit_size=10M
storage_total_size=1G
opus_codec_preset=1
compressor_enabled=0
compressor_max_gain_db=25
@@ -558,19 +587,40 @@ cmake .. && make -j4
| `lottieicon.h/cpp` | Загрузка TGS (gzip→zlib), рендер кадров через `lottie_animation_render` |
| `animtimer.h/cpp` | Синглтон-таймер 30fps, refcount activate/deactivate, сигнал ticked |
| `channellist.h/cpp` `channeldelegate.h/cpp` | Список каналов с аватарами и превью |
| `accountlist.h/cpp` | Список аккаунтов/пользователей |
| `accountlist.h/cpp` `accountdelegate.h/cpp` | Список аккаунтов/пользователей и его делегат отрисовки |
| `memberlistmodel.h/cpp` | Модель списка участников канала |
| `memberpropsdialog.h/cpp` | Диалог свойств участника |
| `flagpainter.h/cpp` | Отрисовка иконок мембера (admin/mod/supernode/storage) |
| `joindialog.h/cpp` | Диалог присоединения к группе/каналу |
| `invitedialog.h/cpp` | Диалог приглашения пользователя |
| `invite_link.h/cpp` | Логика invite-ссылок |
| `inviteby.h/cpp` | Диалог приглашения по invite-ссылке (отправка через узел) |
| `settingsdialog.h/cpp` | Окно настроек |
| `creategroupdialog.h/cpp` | Диалог создания группы |
| `channelsettingsdialog.h/cpp` | Диалог настроек канала |
| `renamedialog.h/cpp` | Диалог переименования |
| `networksettingspage.h/cpp` | Страница сетевых настроек |
| `nodespage.h/cpp` | Таблица узлов + дамп в настройках |
| `statuspage.h/cpp` | Страница статуса (NTP + соединения + живые часы) |
| `databasesettingspage.h/cpp` | Страница обслуживания БД в настройках |
| `storagesettingspage.h/cpp` | Страница настроек хранилища |
| `soundsettingspage.h/cpp` | Страница настроек звука |
| `audiodevicesettingspage.h/cpp` | Страница настроек аудиоустройств |
| `sound_manager.h/cpp` | Управление звуками/уведомлениями |
| `audiorecorder.h/cpp` | Запись голосовых сообщений (захват аудио) |
| `voicemessageencoder.h/cpp` | Кодирование голосовых сообщений (Opus) |
| `voiceplayback.h/cpp` | Воспроизведение голосовых сообщений |
| `media_blocks.h/cpp` | Чтение/сборка фрагментированных медиафайлов |
| `imageviewer_window.h/cpp` | Окно просмотра изображений |
| `videoplayer_engine.h/cpp` | Движок декодирования видео |
| `videoplayer_window.h/cpp` | Окно просмотра видео |
| `connmonitorwindow.h/cpp` | Окно мониторинга ETCP-соединений (standalone) |
| `qrcode_utils.h/cpp` | Утилиты для QR-кодов |
| `debug_ui.h` | Отладочный UI |
### Другие поддиректории
- `db/` — SQLite3 amalgamation (`sqlite3.c/h`), `db_manager.h/cpp` (схема: nodes, node_addresses, channels, msg_<channel>, peers_<channel>, local_identity, accounts, ui_state)
- `transport/` — интеграция с uTun: `gui_bridge.h/gui_bridge_impl.cpp` (Qt signal-based обмен), `chat_core.h/c` (управление), `chat_sync.h/c` (синхронизация), `db_sync_stub.c` (заглушка БД), `node_config.h/cpp`, `config_updater.h/cpp`, `utun_node.h/cpp`, `topo_node_sqlite.h/c`
- `transport/` — интеграция с uTun: `gui_bridge.h/gui_bridge_impl.cpp` (Qt signal-based обмен), `node_config.h/cpp` (конфиг узла), `config_updater.h/cpp` (правка конфига), `utun_node.h/cpp` (обёртка узла), `miniaudio_impl.c` (single-header miniaudio в отдельной единице трансляции). Chat-логика (`chat_core`, `chat_sync` и т.д.) компилируется напрямую из `src/chat/`, роутинг — из `src/routing_layer/` (см. `libutun/CMakeLists.txt`)
- `resources/` — `bg.jpg`, `chatgui.qrc` (встраивает 5 TGS в бинарник), `animations/*.tgs`
### Система анимированных эмодзи
@@ -671,6 +721,14 @@ void lottie_animation_destroy(Lottie_Animation *anim);
- `/doc/etcp_router_arch.md` - ETCP роутер архитектура
- `/doc/route_p2pconn.txt` - Route peer-to-peer соединения
- `/src/route_bgp.txt` - BGP обмен маршрутами (дизайн-документ)
- `/doc/chat_join_test.md` - ТЗ теста join-протокола (chat_join + дерево приглашений)
- `/doc/chat_join_test_tasks.md` - Список задач по тесту join (текущий статус, пошагово)
## Список задач по проекту
Список задач — `/doc/tasks.md` (текущая работа, берём по одной задаче сверху).
При нахождении попутных багов/fail/флаков тестов добавляем задачу (либо фиксим сразу если баг простой).
Как сделали - помечаем [+] выполнено.
## chatgui-android (Android P2P Chat)
@@ -679,7 +737,7 @@ void lottie_animation_destroy(Lottie_Animation *anim);
Android-версия чатгуи — P2P чат на STCP (TCP), UI на Jetpack Compose (Kotlin).
C-ядро: `libutun_lite` (выборочная компиляция нужных .c из `lib/` и `src/`).
Сборка: CMake (headless, Linux) + Gradle/NDK (Android APK).
- 'fw' - собрать и обновить chatgui-android на телефоне (clean + сборка + install)
- 'fw' - собрать и обновить chatgui-android на всех подключенных телефонах (clean + сборка + install)
**Подробная инструкция:** `tools/chatgui-android/AGENTS.md`
**Chat-модули:** все файлы из `src/chat/` (описаны выше в секции «Chat») компилируются в `libutun_lite`.
+89
View File
@@ -0,0 +1,89 @@
# Задача: BGP NODEINFO теряется при флапающем реконнекте (chatgui member online не возвращается)
## Кратко
После рестарта Android-пира и реконнекта локальный chatgui показывает мембера **offline** и не возвращает **online**.
Корневая причина — **не в chat/member коде**, а в ETCP: BGP-пакет NODEINFO (347 байт) расшифровывается, но **не доходит до BGP-обработчика** `topo_group_receive_cbk()`.
Нужно: **точно локализовать точку потери NODEINFO в receive-пути ETCP и починить её.**
## Что уже сделано (исправлено и работает)
1. `tools/chatgui/src/mainwindow.cpp` — **исправлен** `onMemberUpdatedCallback`/`onMemberRemovedCallback`: проверка `len < 1+64+1+79` отбрасывала все события с channel_id короче 64 симв. Теперь `len < 1` + точная проверка. (Это была причина «не обновляется вообще».)
2. `src/chat/chat_member.c/h` — добавлен `chat_core_member_online(ch_id,node_id)` = присутствие в `topo_group` (`topo_node_find_by_id`). Используется в `chat_core_get_member_list`/`get_single_member`. Убран `nodes.online` из SQL и блок `connected→2`.
3. `src/chat/chat_core.c`, `src/chat/chat_sync.c`, `src/chat/member_sync.c/h` — удалён онлайн-статус из БД + merkle-online gossip (`member_sync_set_online`, `_ms_apply_update`, `push_update`).
4. `src/chat/chat_status.c` + `tools/chatgui/src/accountlist.cpp` — флаг «bgp present» (0x08) для detail-панели.
5. `tools/chatgui/src/nodespage.cpp` — убрана колонка Online из диагностики.
Эти правки корректны; статус работает на чистом connect/disconnect/стабильном реконнекте.
## Найденная причина (подтверждено логами)
- Android **шлёт** свой NODEINFO: `send_nodeinfo: node 1657b3ea8281c0a8 ver=2` (bgp-debug на Android).
- Локально пакет **расшифровывается**: `decrypt: code=01 dlen=347 plen=350` (347-байтный BGP-пакет, `code=0x01` = `ETCP_ID_TOPO_ENTRY`).
- Но `BGP recv NODEINFO nid=1657` (строка 193 в `topo_group.c`) **не появляется** — до обработчика пакет не доходит.
- При этом маленький `TABLE_COMPLETE` (10 байт) **доходит** (виден в логе как «mislabeled» `nid=aaaaaaaaaadeadbe` — это строка 193 логирует не-NODEINFO пакет как NODEINFO, garbage-поля).
- Признак: в стабильных рестартах (18:18/18:20) узел **добавляется** каждый раз, в флапающем (18:07) — нет. Т.е. потеря привязана к флапу (TCP-линки + ETCP reinit).
## Receive-путь ETCP (где искать потерю)
```
decrypt (etcp_connections.c:2262, лог "decrypt: code=.. dlen=..")
→ if (link_state==CONNECTED) etcp_conn_input(pkt)
else memory_pool_free(pkt) ← добавлен WARN "decrypted pkt DROPPED"
→ etcp_conn_input (etcp.c:1506, "RX pkt dlen=%d")
→ нормализатор (reassembly) → int_queue
→ etcp_int_recv (etcp_api.c) ← добавлен WARN "int_recv BGP id=0x01 len=.."
→ dispatch по route id (0x01) → topo_group_receive_cbk (topo_group.c:193 "BGP recv NODEINFO")
```
## Уже добавленный дебаг (для локализации)
1. `src/transport_layer/etcp_connections.c` (после дешифровки):
```c
} else {
DEBUG_WARN(DEBUG_CATEGORY_ETCP, "[%s] decrypted pkt DROPPED: link_state=%d code=0x%02x dlen=%u",
link->etcp->log_name, link->link_state, pkt_code, pkt->data_len);
memory_pool_free(e_sock->instance->pkt_pool, pkt);
}
```
2. `src/transport_layer/etcp_api.c` (`etcp_int_recv`, после чтения `id`):
```c
uint8_t id = e->dgram[0];
if (id == 0x01) DEBUG_WARN(DEBUG_CATEGORY_ETCP, "int_recv BGP id=0x01 conn=%s len=%zu", conn->log_name, e->len);
```
3. Android: `tools/chatgui-android/libutun_lite/instance_lite.c` добавлено
`debug_set_category_level(DEBUG_CATEGORY_BGP, DEBUG_LEVEL_DEBUG);` (пересобрать APK: `cd tools/chatgui-android && ./build.sh`).
4. chatgui конфиг `tools/chatgui/build/vibechat.cfg` — включены `bgp=debug`, `chat_sync=debug`, `member_sync=debug`, `debug=debug`.
Как читать (по таймстампу флапа):
- есть `decrypt code=01 dlen=..` + `DROPPED` → потеря в проверке `link_state==CONNECTED`;
- есть `decrypt` + нет `DROPPED` + нет `int_recv BGP` → потеря в `etcp_conn_input`/нормализаторе;
- есть `int_recv BGP` + нет `BGP recv NODEINFO` → потеря в BGP-обработчике.
## Задача для агента
1. Пересобрать chatgui (`cd tools/chatgui && cmake --build build -j4`) и Android APK (`cd tools/chatgui-android && ./build.sh`).
2. Запустить chatgui (DISPLAY=:0, `./vibechat`), воспроизвести флап реконнекта:
```bash
adb shell am force-stop com.utun.chat && adb shell am start -n com.utun.chat/.MainActivity
```
(повторить несколько раз; флап ловится не каждый раз).
3. В логах (`tools/chatgui/build/chatgui.log`) найти потерю NODEINFO и **точно определить точку** по таблице выше.
4. Починить первопричину (варианты):
- буферизовать расшифрованный пакет до `LINK_STATE_CONNECTED` вместо `memory_pool_free`;
- либо гарантированный BGP-retry (после conn UP, если узел пира не в `group->nodes` через N мс — повторно `topo_group_send_table_request`).
5. Убрать весь диагностический мусор:
- WARN "DROPPED" (etcp_connections.c), WARN "int_recv BGP" (etcp_api.c);
- DEBUG-логи в `chat_member.c` (`on_member_props_changed`), `chat_sync.c` (`cs_on_peer_status_changed`), `memberlistmodel.cpp`;
- `instance_lite.c` BGP-debug;
- `vibechat.cfg` — вернуть категории в закомментированное состояние.
6. Прогнать: connect→online, disconnect→offline, реконнект (флап)→online. `./check.sh` (или `cd src && make -j4`).
## Воспроизведение / ключевые артефакты
- Пир: Android SM-A525F (`com.utun.chat`), node_id `0x1657b3ea8281c0a8`.
- Локальный: chatgui (`tools/chatgui/build/vibechat`), node_id `0x24cd036a6e659b9a`.
- Канал: `ch_id=7206723622466219923`, `group_id=0x64036cf3a783ef93`.
- Лог chatgui: `tools/chatgui/build/chatgui.log` (debug_file из конфига).
- Лог Android: `adb logcat -d | grep -iE "send_nodeinfo|NOT in registry|NODEINFO|DROPPED|int_recv"`.
+147
View File
@@ -0,0 +1,147 @@
# План: эпоха (reset_id) для ETCP Router
Аналог фикса `reset_id` из ETCP (см. `61c572c`), применённый к сервисному слою
маршрутизации `etcp_router` (src/routing_layer/etcp_router.c).
## 1. Контекст
`etcp_router` — «упрощённый TCP поверх ETCP»: своя seq-нумерация (`tx_seq`/`rx_seq`),
reorder (`recv_q`), inflight-контроль (`inflight_q`), ретрансмиты, START/RST handshake.
Текущий механизм детекции рестарта пира:
- `SVC_ROUTE_HDR.flags & ROUTER_FLAG_START` — флаг на первом data-пакете сеанса
(`seq==0 && !start_sent && pl_len>0`, `etcp_router.c:207`).
- `router_check_peer_restart` (`etcp_router.c:1055`) — если пришёл START на уже
синхронизированное соединение (`peer_sync_done==1`), то это рестарт пира;
**дедуп ретрансмитов** — побайтовое сравнение с `last_start_dgram`
(`etcp_router.c:1063`). Совпало → это ретрансмит (не рестарт). Не совпало →
`etcp_router_conn_restart` (полный сброс + `sess_id++`).
- `ROUTER_FLAG_RST` + `router_restart_send` (`etcp_router.c:285`) — сброс только
TX-стороны; повторные RST гасятся через `send_restart_pending`.
- `sess_id` (`etcp_router.h:121`) — **локальный счётчик переустановок, на wire
не пишется**. Используется только в `DEBUG_INFO`.
## 2. Проблема (аналог ETCP seq-коллизии)
Дедуп рестарта строится на **побайтовой идентичности** `last_start_dgram`. Это тот
же класс хрупкости, что убрали в ETCP:
1. **Байт-идентичный START** может быть и ретрансмитом, и (теоретически) новым
сеансом — если у нового START совпали `seq==0`, `svc_id`, `timestamp` (16-битные
часы 0.1ms заворачиваются каждые ~6.5 сек) и payload. Тогда рестарт **не
детектится**.
2. Если рестарт не распознан: у пира `tx_seq` ушёл в 0, а у нас `rx_seq` остался
«высоким». В `router_handle_data_packet` (`etcp_router.c:1104`) ветка
`(rx_seq - seq) > 0` / `queue_find_data_by_index` считает пакеты новой сессии
**дубликатами** → молчаливый `c_dup_dropped` + пере-ACK (как `RX dup` в ETCP) →
потеря сервисных пакетов.
3. `sess_id` не на wire → пир вообще не может отличить «мы перезапустились» от
«продолжаем сеанс» иначе как по байтам START.
То есть нужен надёжный маркер **эпохи сеанса** на wire — аналог `reset_id`.
## 3. Решение (аналог reset_id)
Семантика та же, что в ETCP:
- эпоха генерится **один раз** при создании `ETCP_ROUTER_CONN`;
- меняется **только** при (фатальном) рестарте соединения;
- при приёме START/RST с **другой** эпохой → рестарт (adopt);
- **первая** увиденная эпоха принимается без рестарта (свежие при создании);
- дедуп ретрансмитов START — по эпохе, а не по байтам.
Два варианта источника эпохи:
- **(A) Перенести `sess_id` на wire** — он уже есть как `uint32` счётчик
(0 при создании, `++` при рестарте). Минимально инвазивно.
- **(B) Новый `reset_id` (uint64, случайный)** — симметрично ETCP; случайный
надёжнее счётчика (нет wrap-around и предсказуемости), но чуть больше поле.
Рекомендация — **(A)** как минимальный шаг (поле уже есть), при желании апгрейд
до (B).
### Разрыв цикла (master/slave)
Как и в ETCP, симметричное «увидел другую эпоху → рестарт» зацикливается при
одновременном рестарте обеих сторон (A++ → B++ → A видит B → A++ → …).
Ти-брейкер по `node_id` (как в ETCP):
- на приёме START/RST с чужой эпохой:
- **master** (меньший `node_id`) — держит свою эпоху, шлёт RST пиру;
- **slave** (больший `node_id`) — принимает эпоху пира и рестартится.
Существующие предохранители (`send_restart_pending`, `router_check_peer_restart`
на стороне master) сохраняются.
## 4. Wire-формат
`SVC_ROUTE_HDR` (`etcp_router.h:26`) сейчас 33 байта:
```
cmd(1) group_id(8) dst(8) src(8) seq(4) svc_id(1) flags(1) timestamp(2)
```
Добавить поле эпохи (только START/RST, в обычных data/ACK — 0):
- вариант (A): `uint32_t sess_id` → заголовок 33→37 байт (упакован);
- вариант (B): `uint8_t reset_id[8]` → 33→41 байт.
Заполняется в `router_build_packet` (START) и `router_send_rst`; читается в
`router_check_peer_restart` / обработчике RST.
## 5. Шаги реализации
Файлы: `src/routing_layer/etcp_router.h`, `src/routing_layer/etcp_router.c`,
(опц.) `tests/`.
1. **Поле в заголовке**: добавить `sess_id`/`reset_id` в `struct SVC_ROUTE_HDR`;
обновить `SVC_ROUTE_HDR_SIZE`.
2. **Локальное состояние** (`struct ETCP_ROUTER_CONN`):
- `uint32_t sess_id` (уже есть) — оставить как «моя эпоха»;
- добавить `uint32_t peer_sess_id` — последняя увиденная эпоха пира.
- `router_conn_reset` (`etcp_router.c:380`) НЕ сбрасывает `sess_id`
(комментарий на :379 уже это гарантирует).
3. **Запись эпохи**:
- `router_build_packet` (`:193`) — при `ROUTER_FLAG_START` писать `sess_id`;
- `router_send_rst` (`:251`) — писать `sess_id`.
4. **Детект рестарта по эпохе** — переписать `router_check_peer_restart` (`:1055`):
- если `peer_sess_id==0` → первая эпоха, принять без рестарта (сохранить);
- если `hdr->sess_id == peer_sess_id` → ретрансмит (не рестарт);
- если отличается → master/slave: slave → `etcp_router_conn_restart` + adopt;
master → держать свою, при необходимости RST.
- `last_start_dgram`-байт-дедуп заменить на это сравнение (поле можно удалить).
5. **RST-обработчик** (`:1151`) — при RST также сверять `sess_id` (дедуп RST по
эпохе вместо/в дополнение к `send_restart_pending`).
6. **Инициализация** — `peer_sess_id = 0` в `router_conn_reset`/`etcp_router_conn_get`.
## 6. Тест
`tests/test_etcp_router_*` (существующие `test_etcp_router`, `test_etcp_router_reconnect`,
`test_etcp_router_unit`) — прогнать после правки заголовка (37/41 байт).
Добавить (по возможности) регрессию, аналогичную `test_etcp_seq_collision`:
- два инстанса, роутер-коннект, отправка N пакетов;
- принудительный рестарт одной стороны (смена эпохи);
- assert: пакеты новой эпохи доставлены (не `c_dup_dropped`), обе стороны сходятся
к одной эпохе.
## 7. Решения (зафиксировано)
1. **Вариант (B)** — `uint64_t reset_id`, случайный (`random_bytes`). Причина: вариант (A)
`sess_id` стартует с 0 и конфликтует с сентинелом «эпоха не задана». На wire — native
order (как остальные uint64-поля заголовка, SVC_ROUTE не использует htobe*).
2. **master/slave по node_id** — нужен: `etcp_router_conn_restart` делает полный
двунаправленный сброс, без ти-брейкера симметричный рестарт зациклится. master
(меньший node_id) держит эпоху и шлёт RST; slave принимает и ресетится.
3. **route_crypto** — совместим: заголовок не шифруется (только payload), подписывается
только в SIGN-режиме; `SVC_ROUTE_HDR_SIZE = sizeof(...)` авто-адаптируется. START
идёт через `route_crypto_encode`, RST/ACK/CLOSE — plain header-only.
## 8. Статус
Реализовано и проверено: `etcp_router.h/.c` + `tests/test_etcp_router_unit.c`.
Сборка `make clean && make` без ошибок; `./check.sh` → 74 passed / 0 failed / 1 skipped;
`test_etcp_router_unit` 34/34, `test_etcp_router` и `test_etcp_router_reconnect` — PASS.
+14 -3
View File
@@ -137,16 +137,27 @@ else
fi
fi
# Auto-detect FFmpeg (optional video transcoding). If libav libraries are
# available, build with --with-ffmpeg; configure gracefully skips it otherwise.
FFMPEG_OPT=""
if command -v pkg-config >/dev/null 2>&1 && \
pkg-config --exists libavformat libavcodec libavutil libswscale libswresample 2>/dev/null; then
FFMPEG_OPT="--with-ffmpeg"
echo "FFmpeg detected — building with video transcoding"
else
echo "FFmpeg not found — building without video transcoding"
fi
# Generate Makefiles if needed
if [ ! -f Makefile ]; then
echo ""
echo "Running ./configure..."
if [ "$USE_ASAN" -eq 1 ]; then
./configure CFLAGS="-g -Og -fsanitize=address -fno-omit-frame-pointer -fno-pie" LDFLAGS="-no-pie"
./configure CFLAGS="-g -Og -fsanitize=address -fno-omit-frame-pointer -fno-pie" LDFLAGS="-no-pie" $FFMPEG_OPT
elif [ "$USE_HARDENED" -eq 1 ]; then
./configure --enable-hardening
./configure --enable-hardening $FFMPEG_OPT
else
./configure
./configure $FFMPEG_OPT
fi
fi
+92 -8
View File
@@ -1,7 +1,58 @@
#!/bin/sh
# Build script for uTun on FreeBSD (full rebuild)
# Usage: ./build_bsd_full.sh [options]
# Options:
# -h, --help Show this help
# --asan Build with AddressSanitizer (clang)
# --hardened Build with compiler security hardening flags (clang)
# -j[N] Build with N parallel jobs (default: sysctl hw.ncpu)
set -e
# Parse arguments
USE_ASAN=0
USE_HARDENED=0
JOBS=""
MAKE_ARGS=""
while [ $# -gt 0 ]; do
case "$1" in
-h|--help)
echo "Usage: ./build_bsd_full.sh [options]"
echo "Options:"
echo " -h, --help Show this help"
echo " --asan Build with AddressSanitizer"
echo " --hardened Build with compiler security hardening flags"
echo " -j[N] Build with N parallel jobs (default: sysctl hw.ncpu)"
exit 0
;;
--asan)
USE_ASAN=1
shift
;;
--hardened)
USE_HARDENED=1
shift
;;
-j*)
JOBS="$1"
shift
;;
*)
MAKE_ARGS="$MAKE_ARGS $1"
shift
;;
esac
done
if [ -z "$JOBS" ]; then
if command -v nproc >/dev/null 2>&1; then
JOBS="-j$(nproc)"
else
JOBS="-j$(sysctl -n hw.ncpu)"
fi
fi
# Проверка и установка зависимостей
need_install=""
@@ -13,18 +64,24 @@ for cmd in autoreconf aclocal automake libtoolize; do
done
if [ -n "$need_install" ]; then
echo "Устанавливаются необходимые пакеты..."
echo "Устанавливаются необходимые пакеты (нужен root)..."
pkg update
pkg install -y autoconf automake libtool pkgconf openssl
fi
# Предпочитаем GNU make (gmake), но fallback на bmake — он собирает
# autotools-генерённые Makefile данного проекта. gmake ставится отдельно:
# pkg install gmake (нужен root)
GMAKE="$(command -v gmake || command -v make)"
echo "Используем make: $GMAKE"
mkdir -p m4
echo "=== Полная пересборка uTun ==="
# 1. Очистка
echo "[1/4] Очистка..."
make distclean 2>/dev/null || true
$GMAKE distclean 2>/dev/null || true
rm -f config.h config.status config.log
rm -f configure.log autoreconf.log build.log
@@ -42,19 +99,46 @@ chmod +x configure
# 3. Конфигурация
echo "[3/4] Конфигурация..."
./configure --prefix=/usr/local 2>&1 | tee configure.log
# Определяем количество ядер
if command -v nproc >/dev/null 2>&1; then
JOBS=$(nproc)
# FFmpeg (опционально) — автоопределение через pkg-config
FFMPEG_OPT=""
if command -v pkg-config >/dev/null 2>&1 && \
pkg-config --exists libavformat libavcodec libavutil libswscale libswresample 2>/dev/null; then
FFMPEG_OPT="--with-ffmpeg"
echo "FFmpeg detected — building with video transcoding"
else
JOBS=$(sysctl -n hw.ncpu)
echo "FFmpeg not found — building without video transcoding"
fi
if [ "$USE_ASAN" -eq 1 ]; then
echo "(AddressSanitizer enabled — clang, lib/ + src/ only)"
./configure --prefix=/usr/local CC=clang \
CFLAGS="-g -O1 -fsanitize=address -fno-omit-frame-pointer" \
LDFLAGS="-fsanitize=address" $FFMPEG_OPT 2>&1 | tee configure.log
elif [ "$USE_HARDENED" -eq 1 ]; then
echo "(Hardening enabled — clang-safe flags)"
./configure --prefix=/usr/local CC=clang \
CFLAGS="-D_FORTIFY_SOURCE=2 -fstack-protector-strong -fPIE" \
LDFLAGS="-Wl,-z,relro -Wl,-z,now -pie" $FFMPEG_OPT 2>&1 | tee configure.log
else
./configure --prefix=/usr/local $FFMPEG_OPT 2>&1 | tee configure.log
fi
# 4. Сборка
echo "[4/4] Сборка..."
make -j"$JOBS" 2>&1 | tee build.log
if [ "$USE_ASAN" -eq 1 ]; then
(cd lib && $GMAKE $JOBS $MAKE_ARGS 2>&1 | tee ../build.log)
(cd lib/libopus && $GMAKE $JOBS $MAKE_ARGS 2>&1 | tee ../../build.log)
(cd src && $GMAKE $JOBS $MAKE_ARGS 2>&1 | tee ../build.log)
else
$GMAKE $JOBS $MAKE_ARGS 2>&1 | tee build.log
fi
echo
echo "=== Готово ==="
echo "Бинарный файл: src/utun"
if [ "$USE_ASAN" -eq 1 ]; then
echo
echo "Запуск с ASAN (detect_leaks=0 обязателен на FreeBSD):"
echo " ASAN_OPTIONS=detect_leaks=0:halt_on_error=0 ./src/utun -f -p /tmp/utun.pid -l /tmp/utun.log"
fi
+91 -2
View File
@@ -1,7 +1,79 @@
#!/bin/sh
mkdir -p m4
set -e
# --- определение пакетного менеджера ---
detect_pm() {
if command -v apt-get >/dev/null 2>&1; then
PM=apt; PM_UPDATE="apt-get update"; PM_INSTALL="apt-get install -y"
P_AUTOCONF=autoconf; P_AUTOMAKE=automake; P_BUILD=build-essential; P_SSL=libssl-dev
elif command -v dnf >/dev/null 2>&1; then
PM=dnf; PM_UPDATE="true"; PM_INSTALL="dnf install -y"
P_AUTOCONF=autoconf; P_AUTOMAKE=automake; P_BUILD="gcc make"; P_SSL=openssl-devel
elif command -v yum >/dev/null 2>&1; then
PM=yum; PM_UPDATE="true"; PM_INSTALL="yum install -y"
P_AUTOCONF=autoconf; P_AUTOMAKE=automake; P_BUILD="gcc make"; P_SSL=openssl-devel
elif command -v pacman >/dev/null 2>&1; then
PM=pacman; PM_UPDATE="pacman -Sy --noconfirm"; PM_INSTALL="pacman -S --noconfirm"
P_AUTOCONF=autoconf; P_AUTOMAKE=automake; P_BUILD=base-devel; P_SSL=openssl
elif command -v apk >/dev/null 2>&1; then
PM=apk; PM_UPDATE="apk update"; PM_INSTALL="apk add"
P_AUTOCONF=autoconf; P_AUTOMAKE=automake; P_BUILD=build-base; P_SSL=openssl-dev
else
PM=none
fi
}
# --- чего не хватает (логические маркеры) ---
detect_missing() {
missing=""
command -v autoreconf >/dev/null 2>&1 || missing="$missing autoconf"
command -v automake >/dev/null 2>&1 || missing="$missing automake"
if ! command -v gcc >/dev/null 2>&1 || ! command -v make >/dev/null 2>&1; then
missing="$missing buildtools"
fi
[ -f /usr/include/openssl/sha.h ] || missing="$missing openssl"
}
# --- перевести маркеры в пакеты выбранного PM ---
pkg_names() {
pkgs=""
case "$missing" in *autoconf*) pkgs="$pkgs $P_AUTOCONF";; esac
case "$missing" in *automake*) pkgs="$pkgs $P_AUTOMAKE";; esac
case "$missing" in *buildtools*) pkgs="$pkgs $P_BUILD";; esac
case "$missing" in *openssl*) pkgs="$pkgs $P_SSL";; esac
}
# --- установка, если что-то отсутствует ---
detect_pm
detect_missing
if [ -n "$missing" ]; then
if [ "$PM" = "none" ]; then
echo "Ошибка: не найден пакетный менеджер. Установите вручную:$missing"
exit 1
fi
if [ "$(id -u)" -eq 0 ]; then
SUDO=""
elif command -v sudo >/dev/null 2>&1; then
SUDO="sudo"
else
echo "Ошибка: нужны права root (sudo не найден). Установите вручную:$missing"
exit 1
fi
pkg_names
echo "Установка недостающих пакетов ($PM):$pkgs"
$SUDO $PM_UPDATE && $SUDO $PM_INSTALL $pkgs || {
echo "Ошибка: не удалось установить пакеты:$pkgs"
exit 1
}
detect_missing
if [ -n "$missing" ]; then
echo "Ошибка: после установки всё ещё не хватает:$missing"
exit 1
fi
fi
mkdir -p m4
echo "=== Полная пересборка uTun ==="
# 1. Очистка
@@ -9,19 +81,36 @@ echo "[1/4] Очистка..."
make distclean 2>/dev/null || true
rm -f config.h config.status config.log
rm -f configure.log autoreconf.log build.log
rm -f src/utun
# 2. Автогенерация (autoreconf)
echo "[2/4] Автогенерация (autoreconf -fi)..."
autoreconf -fi 2>&1 | tee autoreconf.log
if [ ! -f configure ]; then
echo "Ошибка: autoreconf не создал configure. См. autoreconf.log"
exit 1
fi
# 3. Конфигурация
echo "[3/4] Конфигурация..."
./configure --prefix=/usr/local 2>&1 | tee configure.log
if [ ! -f Makefile ]; then
echo "Ошибка: configure не создал Makefile. См. configure.log"
echo "Возможно не установлен libssl-dev (OpenSSL)."
exit 1
fi
# 4. Сборка
echo "[4/4] Сборка..."
make -j$(nproc) 2>&1 | tee build.log
if [ ! -f src/utun ]; then
echo "Ошибка: сборка не создала src/utun. См. build.log"
exit 1
fi
echo ""
echo "=== Готово ==="
echo "Бинарный файл: src/utun"
echo "Бинарный файл: src/utun (скопирован в ./utun)"
+11
View File
@@ -16,6 +16,17 @@ timeout $TMO make check -j4 > "$TMP" 2>&1 || {
grep -E '\[PASS|\[FAIL|\[SKIP|Test Results|Failed tests|Log directory|^====' "$TMP"
rm -f "$TMP"
# Интеграционный тест tcp_proxy_full (требует sudo; без passwordless sudo — пропускаем)
echo ""
if [ "$(id -u)" -eq 0 ]; then
make -C tests check-proxy
elif command -v sudo >/dev/null 2>&1 && sudo -n true 2>/dev/null; then
echo "=== tcp_proxy_full integration test (sudo) ==="
sudo -n make -C tests check-proxy
else
echo "[SKIP] tcp_proxy_full integration test (requires passwordless sudo)"
fi
ELAPSED=$(( $(date +%s) - START ))
echo "Time: ${ELAPSED}s"
echo "$ELAPSED" > .check_time
+38
View File
@@ -127,6 +127,44 @@ fi
AM_CONDITIONAL([HAVE_WHISPER], [test "x$with_whisper" = "xyes"])
AC_SUBST([WHISPER_LIBS])
# ==================== FFMPEG (OPTIONAL) ====================
AC_ARG_WITH([ffmpeg],
AS_HELP_STRING([--with-ffmpeg], [Enable video transcoding (requires FFmpeg libav libraries)]),
[with_ffmpeg=$withval],
[with_ffmpeg=no])
if test "x$with_ffmpeg" = "xyes"; then
save_LIBS="$LIBS"
FFMPEG_LIBS="-lavformat -lavcodec -lavutil -lswscale -lswresample"
LIBS="$FFMPEG_LIBS $LIBS"
AC_CHECK_HEADER([libavformat/avformat.h], [
AC_CHECK_HEADER([libavcodec/avcodec.h], [
AC_CHECK_HEADER([libavutil/imgutils.h], [
AC_CHECK_HEADER([libswscale/swscale.h], [
AC_CHECK_HEADER([libswresample/swresample.h], [
AC_CHECK_LIB([avformat], [avformat_open_input], [
AC_DEFINE([HAVE_FFMPEG], [1], [Define to enable FFmpeg video transcoding])
AC_MSG_NOTICE([FFmpeg enabled])
], [
AC_MSG_WARN([FFmpeg libs not found, disabling ffmpeg])
with_ffmpeg=no
])
], [AC_MSG_WARN([swresample.h not found, disabling ffmpeg]); with_ffmpeg=no])
], [AC_MSG_WARN([swscale.h not found, disabling ffmpeg]); with_ffmpeg=no])
], [AC_MSG_WARN([imgutils.h not found, disabling ffmpeg]); with_ffmpeg=no])
], [AC_MSG_WARN([avcodec.h not found, disabling ffmpeg]); with_ffmpeg=no])
], [AC_MSG_WARN([avformat.h not found, disabling ffmpeg]); with_ffmpeg=no])
LIBS="$save_LIBS"
fi
if test "x$with_ffmpeg" != "xyes"; then
FFMPEG_LIBS=""
FFMPEG_CFLAGS=""
fi
AM_CONDITIONAL([HAVE_FFMPEG], [test "x$with_ffmpeg" = "xyes"])
AC_SUBST([FFMPEG_LIBS])
AC_SUBST([FFMPEG_CFLAGS])
# Checks for header files, typedefs, functions...
AC_CHECK_HEADERS([arpa/inet.h fcntl.h limits.h netinet/in.h stdint.h stdlib.h string.h sys/ioctl.h sys/socket.h unistd.h])
AC_TYPE_SIZE_T
+1 -1
View File
@@ -203,7 +203,7 @@ SRC_SOURCES=(
transport_layer/pkt_normalizer.c transport_layer/packet_dump.c transport_layer/etcp_api.c
control_server.c
firewall.c eim_nat.c nat_transport.c transport_layer/dummynet.c
etcp_router.c
etcp_router.c route_crypto.c
proxy/udp_proxy.c proxy/socks_proxy.c
lwip_tcp/lwip_pbuf.c
)
-35
View File
@@ -1,35 +0,0 @@
План доработок
[ ] продумать архитектуру репликации таблицы узлов группы
[ ] добавить узлы хелперы (для проксирования трафика)
Категории узлов:
- узлы без нат []
- узлы с nat [restricted]
Узлы за nat сами подключаются к прямым
Узлы за eim nat сами подключаются и сообщают обновление адреса.
Прямые узлы должны сами должны объединиться в сеть следующим образом:
Инит: база содержит RTT по узлам.
выбираем n=16 узлов с минимальным rtt. запускаем подключение к ним.
синхронизируем роутинг таблицу с ними (обмен маршрутами должен инициироваться сразу после подключения).
при потере связи узлы через nat должны найти альтернативы.
в таблице обновляется
как обмен завершен
Нужно собирать и обновлять информацию о том
- к каким публичным нодам подключены nat узлы.
- метрика доступности узла - как давно не менялся ip:port - собираем с клиентов. вместе с рейтигном (каждый клиент по узлу фолрмирует свой рейтинг кандидата в суперузлы и отправляет подписанный рейтинг узлу)
рейтинг включает:
- число разных часов фигурирует в замерах
- число разных дат фигурирует в замерах
- гистограмма rtt: 0-2ms, 2-4ms, 4-6ms, 6-10ms, 10-13ms, 14-18ms, 18-25ms, 25-35ms, 35-50ms, 50-75ms, 75-100ms, 100-130ms, 130-160ms, 160+ms
- гистограмма потерь: 0-1% 1-2% 2-4% 4-8% >8% (обновляется каждый час)
Сделать подключение к группе
+199
View File
@@ -0,0 +1,199 @@
# P2P аудио-звонок пользователю — архитектура
Статус: спроектировано, ждёт реализации.
## 1. Область
Голос 1:1. Звонок можно инициировать **из CHAT-группы или из p2p-чата (DM)** — модуль
не привязан к типу группы: он оперирует `group_id` + `peer_node_id` (для DM `group_id` —
DM-группа, маршрутизация через `etcp_router` идентична). Протокол расширяем под видео
(track-модель), реализуется только аудио. Работает desktop↔desktop и desktop↔android
(обе стороны компилируют общий стек `etcp_router` / `topo_group` (BGP) / `conn_mgr`).
Полный цикл: проверка онлайна (BGP) → INVITE → звонок → ACCEPT/DECLINE →
установка соединения (conn_mgr) → аудиопоток → HANGUP.
## 2. Разбиение файлов (отдельный каталог `src/call/`)
| Файл | Язык | Сборки | Назначение |
|---|---|---|---|
| `src/call/call_proto.h` | C | везде | wire-формат, subcmd, svc_id `ETCP_RT_ID_CALL 0x34` |
| `src/call/call.h` / `call.c` | C | везде (autotools + GUI) | сигналинг, реестр сессий, таймеры, watchdog (20с по приёму), релей медиа с backpressure, счётчики |
| `src/call/call_audio.h/c` | C | только GUI-сборки | Opus encode/decode, фрейминг, pull-обвязка с тонами; регистрация через `call_audio_ops` |
| `src/call/call_tones.h/c` | C | только GUI-сборки | генератор тонов (glitch/ended), синус |
| `src/call/audio_jitter.h/cpp` | C++ | только GUI-сборки | SoundTouch джиттер-буфер, C API |
Autotools-сборка демона остаётся C-only (`.cpp` не компилируется); сигналинг `call.c`
присутствует и в демоне, аудиопроцессинг — только там, где есть GUI.
Новая категория логирования: `DEBUG_CATEGORY_CALL = 31` (добавить в `debug_config.h`,
`DEBUG_CATEGORY_COUNT` → 32).
## 3. Протокол (svc_id 0x34, первый байт payload = subcmd)
Формат приёма (как media_delivery, одноуровневый): `entry->dgram[0]=svc_id`,
`entry->dgram[1]=subcmd`, `entry->dgram[2..]=payload`.
```
CALL_INVITE 0x01 {call_id:8, group_id:8, tracks:[{type:AUDIO, codec:OPUS, sr:4, ch:1, bitrate:4}]}
CALL_RINGING 0x02 {call_id:8}
CALL_ACCEPT 0x03 {call_id:8, agreed_tracks}
CALL_DECLINE 0x04 {call_id:8, reason:1}
CALL_BUSY 0x05 {call_id:8}
CALL_CANCEL 0x06 {call_id:8}
CALL_HANGUP 0x07 {call_id:8, reason:1}
CALL_MEDIA 0x10 {call_id:8, track_id:1, seq:2, ts_ms:4, payload}
```
- `call_id` — случайный uint64, генерит вызывающий; идентифицирует сессию на обеих сторонах.
- `group_id` — CHAT-группа или DM-группа (общая маршрутизация).
- Сигналинг: `etcp_route_send(..., ROUTE_CRYPTO_SIGN | ROUTE_CRYPTO_ENCRYPT)`.
- Медиа: без подписи (уже внутри шифрованного канала etcp_router).
- `CALL_MEDIA` — общий контейнер; `track_id` оставляет место видео (`type=VIDEO`).
- Аудио: Opus 48kHz mono, 20ms (960 сэмплов), ~24–32 kbps. ~50 кадров/с.
## 4. Машина состояний
```
Caller: IDLE → OUTGOING(INVITE) → RINGING(RINGING) → CONNECTING(ACCEPT, conn_mgr_open) → ACTIVE → ENDED
Callee: IDLE → INCOMING(INVITE) → RINGING(sent RINGING) → ACCEPTING(accept, conn_mgr_open) → ACTIVE → ENDED
```
`ENDED` reasons: `LOCAL_HANGUP / REMOTE_HANGUP / DECLINE / BUSY / CANCEL /
NO_TRAFFIC / RING_TIMEOUT / CONNECT_FAIL`.
Таймауты:
- ring-timeout 45с (нет ответа → CANCEL);
- connect: `conn_mgr` `CONN_EVENT_TIMEOUT` → fail;
- **NO_TRAFFIC 20с** — нет входящих медиа-кадров → авто HANGUP.
Онлайн-проверка: `topo_node_find_by_id(group, peer) == NULL` → «не в сети» (event).
Соединение: на ACCEPT обе стороны `conn_mgr_open(inst, group_id, peer, cb, ...)`;
`CONN_EVENT_UP` → ACTIVE. Медиа идёт `etcp_route_send` (после подъёма прямого линка
роутер идёт напрямую — RTT минимален).
## 5. Отправка медиа — backpressure через локальный буфер
Требование: не дропать аудио при переполнении роутера — держим в локальном буфере
отправки, дренируем при появлении места.
Механика (`etcp_router` уже предоставляет):
- `etcp_route_send(force=0)` возвращает `-1`, когда `send_q` роутера полон
(`count >= ROUTER_MAX_SEND_Q_PACKETS`).
- `etcp_router_send_q_has_room(...)` — есть ли место (порог `ROUTER_MAX_SEND_Q_PACKETS-1`).
- `etcp_router_on_send_ready(..., &waiter, drain_cb, arg)` — одноразовый waiter:
колбэк вызывается, когда `send_q->count <= threshold` (освободилось место).
В сессии:
- `tx_q` — локальная FIFO очередь закодированных кадров (ll_entry с dgram), ёмкостью
~1с (≈50 кадров). При переполнении — дроп старейшего + счётчик `c_tx_dropped`.
- Алгоритм (`call_tx_flush`): пока `tx_q` не пуст:
- если `send_q_has_room` → `queue_data_get` + `etcp_route_send(force=0)`;
- иначе — если waiter не зарегистрирован → `etcp_router_on_send_ready(... drain_cb)` и выход.
- `drain_cb(q, arg)` → снова `call_tx_flush(session)` (re-arm при необходимости).
- Весь код — в uasync-потоке (single-threaded, реентерабельность как в
`media_delivery.c:stream_send_chunk_cb`).
## 6. Thread-модель (single-writer)
| Объект | Владелец |
|---|---|
| `call_session` (состояние, encoder, decoder, tx_q, таймеры) | uasync-поток |
| Opus encoder (send) | аудио-поток (encode) |
| Opus decoder (recv) | uasync-поток (`decode → ajb_push`) |
| `ajb` push | uasync-поток |
| `ajb` pull + SoundTouch | аудио-поток |
| `call_tones` | аудио-поток |
Поток данных:
- TX: аудио-поток кодирует → `call_send_encoded(call_id, opus, len)` → `uasync_post` →
uasync: `tx_q` + `call_tx_flush` (backpressure).
- RX: uasync принимает `CALL_MEDIA` → decode → `ajb_push`; аудио-поток `call_pull_pcm` → `ajb_pull` (+ тоны).
`ajb` = mutex-кольцо PCM (гранулярность 20ms/50 ops в сек — мьютекс бесплатен);
SoundTouch инстанс заперт на pull-стороне.
**Контракт жизненного цикла:** GUI стартует аудио-устройство по `CALL_ACCEPTED`,
останавливает по `CALL_ENDED` и только после остановки (join аудио-потока) вызывает
`call_release(call_id)` (post в uasync) → uasync освобождает session/encoder/decoder/jb.
Пока устройство работает — `call_pull_pcm`/`call_send_encoded` не гоняются с освобождением.
## 7. Адаптивный джиттер-буфер (SoundTouch)
Буфер ≤ **1000 мс**, target playout ~60 мс; при росте буфера плавно tempo → **1.6x**.
SoundTouch: `USE_QUICKSEEK=1, SEQUENCE_MS=20, SEEKWINDOW_MS=10, OVERLAP_MS=4`.
API: `ajb_create / ajb_push / ajb_pull / ajb_stats / ajb_destroy`.
## 8. Тоны и диагностика
- **Сталл >500мс** (нет входящих медиа) → в `call_pull_pcm` микшируется
`CALL_TONE_GLITCH`: `пи-пи (2×150мс, gap 100мс) … тишина до 1.0с … пи(200мс)
пауза(200мс) пи(200мс)`, 1000 Гц, цикл пока длится сталл.
- **Завершение звонка → всегда** `CALL_TONE_ENDED`: 3 нисходящих 700/500/400 Гц по 150 мс.
- Счётчики: tx/rx кадры, `c_tx_dropped`/`c_rx_dropped`, дропы буфера, сталлы,
tempo-события, причины завершения. Логи `DEBUG_CATEGORY_CALL` (+стата раз в 1с).
## 9. GUI-события (extend `chat_event.h` + `gui_bridge.h` + android)
```
CHAT_EVT_CALL_INCOMING 32 [ch_id_len][ch_id][caller_node_id:8][call_id:8]
CHAT_EVT_CALL_RINGING 33 [call_id:8]
CHAT_EVT_CALL_ACCEPTED 34 [call_id:8]
CHAT_EVT_CALL_ENDED 35 [call_id:8][reason:1]
CHAT_EVT_CALL_DECLINED 36 [call_id:8][reason:1]
CHAT_EVT_CALL_STATS 37 [call_id:8][rtt_ms:2][buffer_ms:2][tempo_x100:2]
CHAT_EVT_CALL_ERROR 38 [call_id:8][err:1][text:var]
CHAT_EVT_CALL_AUDIO_ROUTE 39 (Android: текущий маршрут/наличие гарнитуры)
```
## 10. Публичный API (call.h)
```
int call_start(ch_id, peer_node_id, out_call_id); // caller (post в uasync)
void call_accept(call_id); // callee
void call_decline(call_id);
void call_hangup(call_id);
int call_is_online(ch_id, peer_node_id); // BGP-проверка для UI
// аудио-поток (общий процессинг в C-ядре):
int call_encode_pcm(call_id, pcm, n, opus_out, cap); // single-writer: audio thread
int call_send_encoded(call_id, opus, len); // post → uasync → tx_q + flush
int call_pull_pcm(call_id, pcm_out, max); // single-writer: audio thread (ajb+tones)
void call_release(call_id); // post → uasync: освободить аудио-объекты
```
`call_init / call_destroy` — вызываются из `chat_core_init / chat_core_destroy`.
## 11. Платформенный I/O
- **Desktop:** `tools/chatgui/src/call_audio_engine.cpp` — miniaudio full-duplex
(capture → `call_encode_pcm`, `call_pull_pcm` → playback) + окно звонка (UI)
+ события в `gui_bridge_impl.cpp`.
- **Android:** JNI `nativeCall*` + Kotlin `AudioRecord`/`AudioTrack`.
**Переключение источника:** `AudioManager` (API31+ `setCommunicationDevice` +
`availableCommunicationDevices`; fallback `setSpeakerphoneOn` + `startBluetoothSco`),
детект гарнитуры (`ACTION_HEADSET_PLUG`, BT-профиль, `ACTION_SCO_AUDIO_STATE_UPDATED`),
кнопка-цикл трубка/громкая/гарнитура, авто-выбор гарнитуры. `nativeCallSetAudioRoute(route)`.
## 12. Сборка
- `src/Makefile.am` + `tests/Makefile.am`: `call.c` (C, без `.cpp`). `src/call/` в include path.
- `tools/chatgui/libutun/CMakeLists.txt`: +`call_audio.c`, `call_tones.c`,
`audio_jitter.cpp`, soundtouch sources (AAFilter, FIFOSampleBuffer, FIRFilter,
InterpolateCubic/Linear/Shannon, RateTransposer, SoundTouch, TDStretch, cpu_detect_x86,
mmx_optimized, sse_optimized) как C++. `call.c` попадёт через GLOB `src/*.c`.
- `tools/chatgui-android/libutun_lite/CMakeLists.txt` + `utun_sources.cmake`:
`LANGUAGES C CXX`, те же файлы.
- `src/transport_layer/etcp_api.h`: `#define ETCP_RT_ID_CALL 0x34`.
- `lib/debug_config.h`: `DEBUG_CATEGORY_CALL 31`, `DEBUG_CATEGORY_COUNT` → 32.
## 13. Порядок задач
1. `call_proto.h` + svc_id + `DEBUG_CATEGORY_CALL`.
2. `call.c` (машина состояний, backpressure-media, watchdog, события, API) + `chat_core` хуки.
3. `audio_jitter.cpp` (SoundTouch) + `test_audio_jitter.c`.
4. `call_tones.c` + `call_audio.c` (Opus + фрейминг + тон-обвязка).
5. Desktop: `call_audio_engine.cpp` + UI + события.
6. Android: JNI + Kotlin AudioRecord/Track + переключение источника + UI.
7. Сборка (Makefile.am / CMake обе).
8. `test_call.c` (два инстанса), интеграция desktop↔desktop и desktop↔android.
-292
View File
@@ -1,292 +0,0 @@
# DB Sync Protocol v2 — Техническое задание
## 1. Проблема
chain_hash = SHA256(prev_chain_hash || id || ts || datahash) — хеш-цепочка, где хеш каждой записи зависит от предыдущей.
`db_record_insert` (db_sync.c:341-463) при вставке записи в середину сортированного порядка делает каскадный пересчёт chain_hash всех последующих записей — O(N). При двустороннем обмене PUSH'ами получается O(N²).
## 2. Решение
**Разделить два концерна:**
- **Сравнение при синхронизации** — перейти на `datahash` (первые 8 байт SHA256(data), позиционно-независимый). Не требует каскада.
- **Целостность цепочки** — `chain_hash` остаётся в схеме БД, но используется только для однократной стартап-проверки и диагностики.
**Синхронизация:** per-peer позиционная (`synced_pos`). При PUSH каскад не делается. При SEND_DATA батч-вставка + один каскад после батча.
## 3. Модель данных
### 3.1. SI_PEER
```c
struct SI_PEER {
uint64_t node_id;
uint32_t synced_pos; // последняя подтверждённо общая позиция (0-based index)
uint8_t sync_state; // 0=not_synced, 1=syncing, 2=synced
};
```
- `synced_pos` = индекс последней записи в глобальном порядке `ORDER BY timestamp, datahash`, для которой chain_hash (а после v2 — datahash) подтверждённо совпадает у обоих пиров.
- При PUSH-вставке записи на позицию P, для всех peer'ов у которых `synced_pos >= P`: `synced_pos = P - 1`.
- При SEND_DATA: `synced_pos = from + received_count - 1`.
### 3.2. Схема БД (без изменений)
```sql
CREATE TABLE "db_sync_<name>_<id>" (
timestamp INTEGER NOT NULL,
datahash INTEGER NOT NULL,
id INTEGER NOT NULL,
chain_hash BLOB NOT NULL,
author INTEGER NOT NULL,
flags INTEGER NOT NULL DEFAULT 0,
data BLOB,
author_signature BLOB,
delivered_peers INTEGER NOT NULL DEFAULT 0,
delivery_chain TEXT NOT NULL DEFAULT '',
PRIMARY KEY (timestamp, datahash)
);
```
## 4. Wire-формат
### 4.1. Сообщения, которые меняются
#### INIT_SYNC
```
Было: [type:1][count:4][last_chain_hash:32] = 37 байт
Стало: [type:1][count:4] = 5 байт
```
#### INIT_RESP
```
Было: [type:1][tp:4][chain_at_tp:32][sc:1][(pos:4,chain_hash:32)*sc]
Стало: [type:1][tp:4][dh_at_tp:8] [sc:1][(pos:4,dh:8)*sc]
dh_at_tp — datahash на позиции tp (8 байт вместо 32)
sparse — datahash вместо chain_hash (8 байт на элемент вместо 32)
При sc=0 (exact match): длина 1+4+8+1 = 14 байт (было 37)
При sc=16 sparse: длина 14 + 16*12 = 206 байт (было 37 + 16*36 = 613)
```
#### SYNC_DONE
```
Было: [type:1][count:4][last_chain_hash:32] = 37 байт
Стало: [type:1][count:4][last_dh:8] = 13 байт
```
### 4.2. Сообщения, которые НЕ меняются
- **PUSH**: `[type:1][id:8][ts:8][dh:8][dlen:4][data][sig_len:1][sig]` — без изменений. `prev_ch` НЕ добавляется.
- **ACK_PUSH**: `[type:1][dh:8][ts:8]` — без изменений.
- **REFINE**: `[type:1][from:4][to:4][hc:1][(pos:4,dh:8)*hc]` — уже использует datahash, без изменений.
- **SEND_DATA**: `[type:1][from:4][count:2][(id:8,ts:8,dh:8,dlen:4,data,sig_len:1,sig)*count]` — без изменений.
- **ERROR**: `[type:1][code:1]` — без изменений.
## 5. Новые/изменённые функции
### 5.1. `db_cascade_from`
```c
static void db_cascade_from(struct DB_SYNC_INSTANCE* si, uint32_t from_pos);
```
Выполняет каскадный пересчёт `chain_hash` для записей с позиции `from_pos` до конца таблицы.
Алгоритм:
1. BEGIN IMMEDIATE
2. SELECT chain_hash записи на позиции `from_pos - 1` (или zero если from_pos = 0) как prev_ch
3. SELECT id, timestamp, datahash от позиции from_pos до конца
4. Для каждой: chain_hash = SHA256(prev_ch || id || ts || dh), UPDATE в БД, prev_ch = новый chain_hash
5. COMMIT
Выделяется из текущего кода `db_record_insert` (строки 412-451) в отдельную функцию.
### 5.2. `db_record_insert` — новый параметр
```c
static int db_record_insert(struct DB_SYNC_INSTANCE* si,
uint64_t id, uint64_t ts, uint64_t dh,
const char* json, size_t jlen,
const uint8_t* sig, size_t sig_len,
int do_cascade);
```
- `do_cascade=1` — после INSERT выполняется cascade (строки 412-451). Используется для локальных вставок.
- `do_cascade=0` — без cascade. Используется для PUSH и SEND_DATA (каскад делается отдельно).
### 5.3. `db_verify_chain` (стартап-проверка)
```c
static void db_verify_chain(struct DB_SYNC_INSTANCE* si);
```
Вызывается один раз после `db_sync_instance_add` (перед инициацией sync с пирами).
Алгоритм:
1. Проход записей 0..N-1 в порядке `ORDER BY timestamp, datahash`
2. Для каждой: chain_hash = SHA256(prev_ch || id || ts || dh)
3. Сравнить с хранимым в БД
4. При первом расхождении на позиции P: пересчитать цепочку от P до конца через `db_cascade_from(si, P)`. Завершить.
Сложность: O(N) SHA256, однократно при старте.
### 5.4. `si_find_pos` (новая)
```c
static uint32_t si_find_pos(struct DB_SYNC_INSTANCE* si,
uint64_t ts, uint64_t dh);
```
Возвращает позицию (0-based index) записи с ключом `(ts, dh)` в глобальном порядке. Используется после вставки PUSH для корректировки `synced_pos`.
### 5.5. `db_datahash_at` — уже существует (стр. 295)
Возвращает `datahash` на заданной позиции. Используется для сравнения в INIT_SYNC/INIT_RESP/SYNC_DONE.
## 6. Обработчики сообщений
### 6.1. PUSH (`db_handle_push`)
```
Было: db_record_insert(si, ..., /* cascade встроен */)
Стало: db_record_insert(si, ..., /* do_cascade= */ 0)
uint32_t pos = si_find_pos(si, ts, dh);
for each peer: if peer->synced_pos >= pos: peer->synced_pos = pos - 1;
```
### 6.2. INIT_SYNC (`db_handle_init_sync`)
```
Изменения:
- Принимает [count:4] вместо [count:4][chain_hash:32] (проверка len >= 5 вместо 36)
- Сравнение: datahash вместо chain_hash (8 байт вместо 32)
- sparse: datahash вместо chain_hash (8 байт на элемент вместо 32)
- Размер sparse-элемента: 4(pos) + 8(dh) = 12 байт (было 4+32=36)
- resp буфер: 4096 → достаточно (14 + 16*12 = 206 байт при max sparse)
```
### 6.3. INIT_RESP (`db_handle_init_resp`)
```
Изменения:
- Принимает [tp:4][dh:8][sc:1][...] вместо [tp:4][ch:32][sc:1][...]
- Проверка len >= 13 вместо 37
- Сравнение: datahash вместо chain_hash
- Пустая БД пира: zero-хеш 8 байт вместо 32
- Sparse элементы: 12 байт вместо 36
- При совпадении: synced_pos = tp (в дополнение к sync_state = 2)
```
### 6.4. SEND_DATA (`db_handle_send_data`)
```
Стало:
struct SI_PEER* sp = si_peer_find(si, src);
uint32_t fix_from = sp ? sp->synced_pos : 0;
for each record:
db_record_insert(si, ..., /* do_cascade= */ 0);
db_cascade_from(si, fix_from);
sp->synced_pos = from + received_count - 1;
sp->sync_state = 2;
```
Каскад от `fix_from` (позиция, которая была synced ДО этого батча) гарантирует, что все chain_hash после этой точки пересчитаны — независимо от того, какие PUSH'и испортили их между синхронизациями.
### 6.5. SYNC_DONE (`db_handle_sync_done`)
```
Изменения:
- Принимает [count:4][last_dh:8] вместо [count:4][last_chain_hash:32] (len >= 13 вместо 36)
- Сравнение последнего datahash вместо chain_hash
```
## 7. Механика многопировой синхронизации
При наличии нескольких пиров с одинаковым `sync_state` (например, syncing=1) выбирается пир с минимальным `synced_pos` — у него самый старый общий префикс. После завершения его синхронизации выбор повторяется.
Это гарантирует: двигаемся от самого старого несинхронизированного участка, не прыгая. Новые записи (в хвосте) синхронизируются последними.
## 8. Полный сценарий: три участника A, B, C
```
Начальное состояние: все пусты.
=== Шаг 1. A создаёт R1 (ts=1000) ===
A: PUSH R1 → B, C. Все вставляют в конец, cascade 0.
synced_pos = 0 у всех.
=== Шаг 2. B создаёт R2 (ts=2000), C создаёт R3 (ts=1500) ===
B: [R1(1000), R2(2000)], PUSH R2 → A, C.
A: R2 в конец. C: R2 в конец.
C: [R1(1000), R2(2000)]? Нет — C создал R3(1500).
C: [R1(1000), R3(1500)] локально, потом PUSH R2 → R2.ts=2000 → в конец.
C: [R1(1000), R3(1500), R2(2000)]
C: PUSH R3 → A, B.
A: [R1(1000), R2(2000)] + R3(1500) → между R1 и R2:
A: [R1, R3, R2] ← R3 в середину, cascade ПРОПУЩЕН
B: [R1, R3, R2] ← аналогично
synced_pos после PUSH R3 на A относительно B:
A вставил R3 на поз.1 → synced_pos_B: было 1 → стало 0
(A считает, что совпадает с B только на позиции 0)
=== Шаг 3. A инициирует sync с B (чемпион: min synced_pos) ===
A→B: INIT_SYNC [count=3]
B→A: INIT_RESP [tp=min(3,3)-1=2, dh_at_tp=H2, sc=2, sparse:{(1,H3),(0,H1)}]
A проверяет: A@2=R2→dh=H2✓, A@1=R3→dh=H3✓, A@0=R1→dh=H1✓
→ sync complete, synced_pos_B=2, sync_state=2
=== Шаг 4. Рестарт A ===
db_verify_chain:
pos 0: R1.ch = SHA256(0||R1) ✓
pos 1: R3.ch = SHA256(R1.ch||R3) ✓ (посчитан при PUSH без cascade — корректен)
pos 2: R2.ch = SHA256(R3.ch||R2) ✗ (STALE: был посчитан как SHA256(R1.ch||R2) до PUSH R3)
→ расхождение на pos=2 → db_cascade_from(2): пересчёт R2.ch = SHA256(R3.ch||R2) ✓
=== Шаг 5. A, B, C активно обмениваются ===
Каждый PUSH: вставка без cascade.
Периодические sync (по min synced_pos): SEND_DATA + один cascade от synced_pos.
synced_pos растёт.
```
## 9. Что НЕ трогать
- `db_sync_insert_signed` — локальная вставка. ts всегда монотонный, запись в конец, `do_cascade=1`, каскад всегда 0 строк.
- TTL cleanup (`db_sync_instance_ttl_cb`) — удаление старых записей. Нужен cascade после удаления (уже существующая логика, не меняется).
- ACK_PUSH, ERROR, REFINE — без изменений.
- Схема БД — без изменений.
- `etcp_bind`/эпилог — без изменений.
## 10. Сложность операций (итого)
| Операция | Каскад | Сложность |
|----------|--------|-----------|
| Локальная вставка | 0 строк (конец) | O(1) |
| PUSH (приём) | нет | O(1) |
| SEND_DATA (батч N записей) | 1 каскад после батча | O(N + tail) |
| Стартап-проверка | 1 раз | O(total) |
| Sync-протокол (INIT_SYNC/INIT_RESP/REFINE) | нет cascade | O(log N) сообщений |
## 11. Порядок реализации
1. **Выделить `db_cascade_from(si, from_pos)`** из тела `db_record_insert`.
2. **Добавить параметр `do_cascade`** в `db_record_insert`. При `0` — только INSERT, cascade вызывается отдельно.
3. **Добавить `synced_pos`** в `struct SI_PEER`. Инициализировать в 0.
4. **Добавить `si_find_pos(si, ts, dh)`** — поиск позиции записи в глобальном порядке.
5. **Изменить wire-формат** INIT_SYNC, INIT_RESP, SYNC_DONE (chain_hash → datahash, 32→8 байт).
6. **Изменить обработчики**:
- `db_handle_init_sync`: новые размеры, datahash вместо chain_hash
- `db_handle_init_resp`: новые размеры, datahash вместо chain_hash, обновлять synced_pos
- `db_handle_send_data`: `do_cascade=0`, `db_cascade_from(fix_from)`, обновить synced_pos
- `db_handle_push`: `do_cascade=0`, скорректировать все synced_pos
- `db_handle_sync_done`: обновить synced_pos
7. **Добавить `db_verify_chain(si)`** — вызвать при старте после `db_sync_instance_add`.
8. **Многопировая синхронизация**: в `db_sync_peer_check_cb` выбирать пира с min synced_pos вместо первого попавшегося.
+93
View File
@@ -0,0 +1,93 @@
# DM — прямой p2p чат между двумя пользователями группы
## 1. Цель
Возможность начать личный чат с любым пользователем доступных групп.
Пока target offline сообщения временно хранятся на storage-узлах.
## 2. Ключевые решения
- Отдельная DM-подсистема (НЕ канал: без TOPO_GROUP/member_sync/merkle).
- Идентичность детерминированная: одна ветка на пару, согласование без handshake.
- E2E-шифрование содержимого сразу.
- Offline-хранение: dm_mailbox на storage-узлах с флагом dm_storage.
- Доставка: fallback — сперва проверка наличия узла в BGP-группе; доступен →
прямой send; нет связи → storage.
## 3. Идентичность и ключи (детерминированные)
conv_id = SHA256("utun_dm_v1" || min(node_a,node_b) || max(node_a,node_b))[..63]
shared = X25519(my_x25519_priv, peer_x25519_pub)
content_key = SHA256(shared || "utun_dm_content")
Обе стороны выводят одинаковые значения из своего privkey и pubkey пира
(известен из общей группы). Смена x25519 у пира = новая ветка.
## 4. Модель данных (SQLite, общий chats.db)
dm_conversations(conv_id TEXT PK, peer_node_id, peer_x25519 BLOB, peer_ed25519 BLOB,
peer_name TEXT, last_out_seq INT, last_in_seq INT, created_at INT, last_ts INT)
dm_messages(conv_id TEXT, dir INT, seq INT, ts INT, author INT, ct TEXT,
data BLOB, sig BLOB, PRIMARY KEY(conv_id,dir,seq))
dm_mail(recipient INT, sender INT, conv_id TEXT, seq INT, ts INT, data BLOB, sig BLOB,
ttl INT, PRIMARY KEY(recipient,sender,seq))
Два независимых направленных потока (A→B, B→A) с монотонным seq.
Без chain-хеша: для 2 пиров достаточно seq + catch-up по диапазону.
## 5. Доставка
1. Проверка доступности: peer присутствует в BGP-группе (общая группа).
2. Доступен → прямой send (сервис ETCP_RT_ID_DM = 0x34), ждём DM_ACK{conv_id,seq}.
Нет ACK за таймаут → fallback в mailbox.
3. Нет связи → MAIL_PUT на 1..N storage-узлов.
4. Catch-up при connect: DM_HELLO{conv_id,last_out_seq,last_in_seq} → пир досылает
(in_seq+1 .. out_seq).
5. Dedup: вставка только при seq > last_in_seq.
## 6. E2E-шифрование
data_enc = nonce(13B, счётчик по seq) || AES-CCM_ct || tag
secure_channel уже умеет AES-CCM. Storage/relay видят только шифротекст.
## 7. Storage-узлы (dm_storage)
- Новый флаг dm_storage=yes в adm_tags (блок B владельца), распространяется
существующим member_sync без нового синка.
- DM читает adm_tags мемберов общих групп (peers_<ch>) и кэширует storage-узлы.
- Протокол mailbox (сервис ETCP_RT_ID_DM_MAILBOX = 0x35):
MAIL_PUT {recipient,sender,conv_id,seq,ts,data,sig}
MAIL_PULL {recipient, per-sender cursor}
MAIL_ACK {up_to_seq}
TTL 7 дней + лимит объёма, dedup по (recipient,sender,seq).
## 8. Старт и anti-spam
- dm_start(peer): создать беседу (вывести conv_id/ключи), обеспечить прямое
соединение (переиспользовать chat_sync_connect_node/node_conn_direct).
- Пир автопринимает на первом DM_HELLO/сообщении, только если отправитель в
общей группе.
## 9. Файлы и интеграция
| Файл | Назначение |
|------|-----------|
| src/dm/dm_core.c/h | беседы, send/recv, таблицы, bind 0x34, conn-события, fallback |
| src/dm/dm_crypto.c/h | derive + AES-CCM |
| src/dm/dm_mailbox.c/h | storage-узел, push/pull/ack, bind 0x35 |
Плюс:
- src/chat/chat_member.c — бит CHAT_MEMBER_FLAG_DM_STORAGE + парсинг dm_storage.
- utun_instance.c — dm_core_init/destroy.
- chat_core — API dm_start/send/list + события CHAT_EVT_DM_*.
- headless CLI — dm list/send/messages.
- src/Makefile.am — новые источники.
## 10. Сервисы ETCP
ETCP_RT_ID_DM = 0x34
ETCP_RT_ID_DM_MAILBOX = 0x35
-10
View File
@@ -1,10 +0,0 @@
В конфиге явно прописываются сокеты с ip и портом.
Через эти сокеты идёт весь трафик, случайные порты клиент не использует - только эти сокеты.
Соответственно, возможен только один линк между двумя пирами по одному каналу.
виртуальные сокеты - плохо т.к. используются ip/port для поиск подключения
./configure --with-openssl - openssl
./configure --without-openssl - tinycrypt
+45 -42
View File
@@ -20,7 +20,8 @@ ETCP_ROUTER_CONN (один на пару remote_node_id + svc_id)
│ │
│ ack_timer: void* периодический 100ms │
│ idle_ack_timer: void* idle таймаут 500ms │
│ send_resume_timer: void* retry send_q 50ms │
│ send_waiter: queue_waiter_handle waiter на send_input_q │
│ watchdog_timer: void* проверка инварианта 500ms │
│ send_blocked: u8 inflight полон │
└─────────────────────────────────────────────────────┘
@@ -50,40 +51,41 @@ SVC_ROUTE_HDR (22 байта)
│
▼
┌──────────────────────────────┐
│ inflight < ROUTER_MAX(256)? │
│ tx_seq - tx_acked < 256 ? │
└──────┬───────────┬───────────┘
│ YES │ NO (inflight полон)
▼ ▼
┌──────────────┐ ┌─────────────────────┐
│router_send_one│ │ router_enqueue_send │ → send_q (FIFO)
│ (прямая) │ │ (в очередь) │ send_blocked = 1
└──────┬────────┘ └─────────┬───────────┘ send_resume_timer (50ms)
│ │
▼ │
┌────────────────────┐ │
│ tx_seq++ │ │
│ hdr.dst = dst │ │
│ hdr.src = node_id │ │
│ hdr.seq = tx_seq │ │
│ hdr.svc_id = svc │ │
│ │ │
│ etcp_send(conn, e) │ │
└────────────────────┘ │
│
┌────────────────────────────┘
│ (при получении ACK от remote)
▼
┌────────────────────┐
│ router_drain_send_q│ вызывается из ACK handler
│ │ или по send_resume_timer
│ while inflight < 256│
│ e = send_q.pop() │
│ router_send_one(e)│
│ │
│ если send_q пуст: │
│ send_blocked = 0 │
└────────────────────┘
│ router_enqueue_send(rconn) │ всегда через send_q (FIFO)
│ if send_q >= 64 && !force: │ → вернуть -1 (backpressure)
│ return -1 │
└──────┬───────────────────────┘
│ копия в send_q + router_send_kick()
▼
┌─────────────────────────────────────────┐
│ router_send_kick(rconn) │
│ closed/no_route → return │
│ send_q пуст → return │
│ inflight >= limit → send_blocked=1, ret │
│ waiter уже ждёт → return │
│ нет conn → set_no_route │
│ queue_waiter_wait(send_input_q, │
│ &send_waiter, drain_cb) │
└──────┬───────────────────────────────────┘
│ send_input_q опустел (round-robin: FIFO waiter-ов)
▼
┌──────────────────────────────┐
│ router_send_drain_cb(rconn) │ шлём ОДИН пакет
│ e = send_q.pop() │
│ router_send_one_flags(e) │
│ → tx_seq++ │
│ → etcp_send(conn, entry) │
│ если send_q ещё непуст: │
│ inflight полон → blocked=1│
│ иначе → queue_waiter_wait │ (в хвост → round-robin)
│ если пуст → send_blocked=0 │
└──────────────────────────────┘
Повторный запуск drain:
• приход ACK → router_handle_ack → kick
• маршрут появился → router_no_route_retry_cb → kick
• рост inflight-лимита → router_update_inflight_limit → kick
• страховка → router_send_watchdog_cb (500ms) → DEBUG_ERROR + kick
```
## Receive Path (приём данных)
@@ -119,7 +121,7 @@ SVC_ROUTE_HDR (22 байта)
│ │ = hdr.seq││ |d| < ROUTER_MAX_INFLIGHT? │
│ │ (только ││ │
│ │ вперёд) ││ Дубликат? │
│ │ drain_q ││ rx_seq - seq > 0 ИЛИ │
│ │ kick ││ rx_seq - seq > 0 ИЛИ │
│ └──────────┘│ seq уже в recv_q? │
│ └──────┬─────────────┬──────────────────┘
│ │ OK │ out-of-bounds / dup
@@ -226,7 +228,8 @@ SVC_ROUTE_HDR (22 байта)
│ DEBUG_WARN "stale ACK" │
│ │
│ if send_blocked: │
│ router_drain_send_q(rconn) │
│ send_blocked = 0 │
│ router_send_kick(rconn) │
└────────────────────────────────────────┘
```
@@ -241,12 +244,12 @@ SVC_ROUTE_HDR (22 байта)
При блокировке:
данные → send_q (FIFO, без дропов)
send_blocked = 1
send_resume_timer = 50ms retry
Разблокировка:
приход ACK от remote → tx_acked обновляется
→ router_drain_send_q() разгребает send_q
или send_resume_timer (50ms) → router_drain_send_q()
→ router_send_kick() → waiter → drain_cb разгребает send_q
или маршрут появился / inflight-лимит вырос → kick
страховка: router_send_watchdog_cb (500ms) → force kick
send_q полностью разобран → send_blocked = 0
```
@@ -304,8 +307,8 @@ SVC_ROUTE_HDR (22 байта)
│ idle_ack_timer ─── 500ms ─────▶ router_idle_ack_cb │
│ (досылка последнего ACK) │
│ │
│ send_resume_timer ─ 50ms ─────▶ router_send_resume │
│ (retry когда send_q не пуст) │
│ watchdog_timer ── 500ms ─────▶ router_send_watchdog│
│ (страховка инварианта drain, force kick) │
│ │
└─────────────────────────────────────────────────────┘
```
+66
View File
@@ -0,0 +1,66 @@
# Важные нюансы работы встроенного lwIP TCP-стека
Краткая памятка по тонким местам `src/lwip_tcp/`, критичным для прокси.
Полное описание модуля — в `src/lwip_tcp/lwip_tcp_doc.md`.
## Ключевые константы (`lwip_tcp_opts.h`)
| Константа | Значение | Смысл |
|-----------|----------|-------|
| `TCP_MSS` | 1460 | максимальный размер сегмента |
| `TCP_WND` | 8×MSS = 11680 | окно приёма lwIP (объявляется пиру) |
| `TCP_SND_BUF` | 16×MSS = 23360 | буфер отправки |
| `TCP_TMR_INTERVAL` | 250 мс | базовый интервал таймера |
| `tmr_interval_ms` | `TCP_TMR_INTERVAL/4` = 62.5 мс | фактический шаг `tcp_fasttmr` |
| `TCP_WND_UPDATE_THRESHOLD` | `TCP_WND/4` = 2920 | порог немедленного ACK окна |
**Важно:** таймер uasync использует timebase 0.1 мс — `uasync_set_timeout(ua, N, …)`
задаёт N×0.1 мс. Например, 5000 = 500 мс.
## Контракт recv_cb — главный источник багов
lwIP вызывает `recv_cb(arg, pcb, pbuf, err)` для данных, **уже извлечённых из pcb**.
Дальше приложение само решает судьбу данных и окна:
| Действие | Результат |
|----------|-----------|
| скопировать + `tcp_recved(pcb, len)` | данные приняты, окно восстановлено (норма) |
| `return != LERR_OK` | lwIP кладёт pbuf в `refused_data` и отдаст позже — **без потери**, окно закрывается |
| `pbuf_free(p)` без `tcp_recved` | данные потеряны, окно не восстановлено → схлопывание окна |
`refused_data` — штатный backpressure lwIP: одноканальный (один pbuf), ре-доставка
через `tcp_fasttmr` (62.5 мс) или при следующем входящем сегменте.
**Ошибка, из-за которой падал `tcp_proxy_full`:** при backpressure прокси делал
`pbuf_free(p)` без `tcp_recved`. Каждый такой сброс уменьшал `rcv_wnd` на MSS; после
~8 пакетов окно падало в 0, OS-TCP переставал слать, и передача умирала.
## Управление окном и «деградация 62.5 мс»
`tcp_recved(pcb, len)` увеличивает `rcv_wnd` и через `tcp_update_rcv_ann_wnd` решает,
слать ли ACK окна немедленно. Немедленный ACK идёт только если прирост окна
`wnd_inflation ≥ TCP_WND_UPDATE_THRESHOLD` (=2920 = 2×MSS); иначе ACK откладывается
до `tcp_fasttmr` (62.5 мс).
Отсюда классическая ловушка: если после backpressure окно восстанавливать **по одному
пакету** (`tcp_recved(1460)` за раз), прирост 1460 < 2920 → ACK уходит раз в 62.5 мс →
пир шлёт 1 пакет за 62.5 мс ≈ 23 КБ/с. Это выглядит как «пропускная способность
застряла», хотя сеть свободна.
**Правильно:** восстанавливать окно **пачкой** (несколько `tcp_recved` подряд либо один
`tcp_recved(TCP_WND_MAX - rcv_wnd)`), тогда прирост ≥ 2920 → ACK немедленный → пир
сразу возобновляет бурст.
## Рекомендуемый паттерн для relay-потребителя (как в `tcp_proxy_client.c`)
1. При получении данных — копировать в ограниченную очередь, **не вызывать**
`tcp_recved` (окно само плавно закрывается; очередь ограничена `TCP_WND`).
2. Дрейн очереди по сигналу освобождения нижележащего канала
(`etcp_router_on_send_ready`) + retry-таймер (force=1) как safety-net.
3. `tcp_recved(len)` вызывать **только после успешной отправки** — окно
восстанавливается пачкой по мере дрена, ACK уходит немедленно.
4. Никогда не сбрасывать pbuf без `tcp_recved`; не держать данные в `refused_data`
дольше одного пакета (single-slot).
См. реализацию: `tcp_proxy_client_tx_queue_drain_cb` / `tcp_proxy_client_fin_flush`
в `src/proxy/tcp_proxy_client.c`.
+137
View File
@@ -0,0 +1,137 @@
# Задачи по проекту
Сюда пишется список задач с короткой аннотацией. Если задача большая и имеет ТЗ - то ТЗ оформляется отдельным файлом, а сюда помещается аннотация и ссылка на ТЗ.
## Текущая задача
[ ] **DM: прямой p2p чат с любым пользователем группы** — отдельная DM-подсистема.
ТЗ и архитектура: `/doc/dm_arch.md`. Статус: реализовано (dm_core/dm_mailbox/dm_crypto),
тесты `test_dm` (крипто) и `test_dm_e2e` (интеграция). Остались задачи ниже.
[ ] **Звонок (P2P аудио)** — голос 1:1, из CHAT-группы или p2p-чата (DM). Отдельный
каталог `src/call/`. Сигналинг + conn_mgr + аудиопоток (Opus) с backpressure
(локальный буфер отправки при переполнении роутера) + адаптивный джиттер-буфер
(SoundTouch, ускорение до 1.6x) + тоны (глитч/завершение) + watchdog 20с.
Android: переключение источника (трубка/громкая/гарнитура).
Архитектура: `/doc/chat_call_arch.md`. Статус: спроектировано, ждёт реализации.
## Рефакторинг: per-instance chat/dm (однопоточный test_dm_e2e)
Сделано (глобальные синглтоны переведены на `UTUN_INSTANCE`):
- `chat_setting` → `inst->chat_settings` (`struct chat_setting_state`, state-level API для парсера конфига + instance-level для runtime).
- `chat_event` → `inst->chat_event_handler` (обработчик теперь получает `inst`).
- `chat_core` (`g_cc`) → `inst->chat_core` (`struct chat_core_ctx*`).
- `chat_sync` (`g_cs`) → `inst->chat_sync`.
- `member_sync` callback-списки (`g_props_cbks`/`g_apply_cbks`) → в `chat_core_ctx`.
- `chat_join` (`g_keys`) → в `chat_core_ctx`.
- `chat_msg` media-счётчики/backfill → в `chat_core_ctx`.
- `dm_core` (`g_dm`) → `inst->dm`; `dm_mailbox` (`g_mb`) → `inst->dm_mailbox`.
- `chat_headless_control` (`g_hc`) → `inst->headless`.
- `chat_whisper` — сигнатуры `init/available/trigger` принимают `inst`.
- API протащено через `inst` (chat_core_*/dm_*), обновлены call-сайты в src/ (headless, topo, media_delivery, auto_socket, config_parser).
- `test_dm_e2e.c` переписан на однопоточный: один `uasync`, три `utun_instance` (A,B,C), master state-machine, без fork. Фаза 2 (B offline) — `utun_instance_destroy(B1)` + пересоздание B2.
- Попутно исправлен 1-байтовый overflow в `dm_mailbox.c: mb_route_send` (`u_malloc(1+body_len)` → `1+1+body_len`).
Статус сборки: `make -C src` (libutun.a + бинарник utun) и `make -C tests` — OK.
### Осталось (по порядку)
[+] **test_dm_e2e: зависание в utun_instance_destroy(B1)** (фаза P_B_OFFLINE) — сделано.
Причин было несколько (не одна):
1. `lib/u_async.c`: если ближайший таймер уже истёк на входе в `uasync_poll`,
`get_next_timeout` возвращал {0,0}, `timeout_ms=-1` → `epoll_wait(-1)` блокировался
навсегда, а `process_timeouts` (вызывается только после epoll_wait) не запускался.
Фикс: `if (timeout_ms == -1 && heap не пуст) timeout_ms = 0`.
2. `routing_layer/etcp_router.c`: `router_send_conn` с group_id=0 (глобальная
маршрутизация DM/mailbox) не находил маршрут (`topo_groups_find(0)==NULL`).
Фикс: fallback на `instance_find_conn(inst, remote)` при group==NULL.
3. `dm/dm_core.c`: `dm_on_conn_status` PULL-ил только беседы с подключившимся пиром;
B2 подключается к storage (C), а беседа — с A. Фикс: на подъём любого соединения
PULL для всех бесед, чей пир не подключён напрямую.
4. `dm/dm_mailbox.c`: не было ACK после PULL_RESP → storage не чистил dm_mail.
Фикс: PULL_RESP теперь несёт sender, получатель шлёт MB_SUBCMD_ACK.
5. `utun_instance.c`: `dm_core_init` ставил deliver-cb до `dm_mailbox_init` →
mailbox не находился, deliver-cb не регистрировался. Фикс: mailbox init раньше core.
`test_dm_e2e` проходит (несколько прогонов), `test_dm` тоже.
[+] **chatgui-android: переезд на per-instance API** — сделано.
`instance_lite.c/h` (+`instance_lite_get_instance()`), `jni_bridge.c`, `standby.c`,
`headless_control.c`, `voice_recorder.c`, `attachment_sender.c`, `video_sender.c`,
`photo_sender.c` — все call-сайты переведены на `inst`. `libutun_lite` компилируется,
`jni_bridge.c` — синтакс-чисто. (headless-линковка сломана ПРЕДЫДУЩЕ — instance_lite.c
ссылается на jni_bridge-функции, не включённые в headless-сборку; отдельная задача.)
[+] **test_chat_join: segfault** (регресс рефакторинга) — сделано.
`chat_join` хранил `join_keys` в `chat_core_ctx` (CC(inst)), а тест использует chat_join
без chat_core → NULL-deref. Фикс: `join_keys` вынесен в `UTUN_INSTANCE.join_keys`
(у chat_join собственный init/destroy).
[ ] **BGP-гонка: дропнутый REQUEST_TABLE** (отдельная задача, как договорились).
**Место:** `topo_group_new_conn()` в `src/routing_layer/topo_group.c:571`. Дедуп-ветка
«conn already in senders_list» (строки 579–587) делает `return` БЕЗ повторной отправки
`topo_group_send_table_request()`.
**Зачем дедуп:** тот же `ETCP_CONN` стреляет `ETCP_CONN_STATUS_UP` дважды (UDP-линк,
затем TCP-линк). Повторная обработка задваивает `active_conn_count`, из-за чего
переподключение не стартует. Поэтому `return` в дедупе — правильный.
**В чём гонка:** REQUEST_TABLE — это то, что заставляет пира отдать свою таблицу
(NODEINFO/узлы группы). Сценарий «разнесённого старта»:
1. A↔C соединение поднято. A шлёт C `REQUEST_TABLE` для группы G (`topo_group_new_conn`
→ `topo_group_send_table_request`, `topo_group.c:602`).
2. У C группа G ещё **не создана** (создаётся позже, по мере загрузки каналов/синка).
C получает `REQUEST_TABLE`, но `topo_group_handle_request_table` не находит G →
дропает запрос.
3. C наконец создаёт G (`topo_groups_create_group` → обход connections →
`topo_group_new_conn(G, conn_A)` на стороне C) → C шлёт A свой `REQUEST_TABLE`.
A отвечает своей таблицей → **C узнаёт A**.
4. Но A больше не шлёт C `REQUEST_TABLE` → **A так и не узнаёт C** в группе G.
Итог: при разнесённом старте A не видит C в CHAT-группе (односторонняя видимость).
**Фикс (предполагаемый):** в дедуп-ветке не просто `return`, а предварительно
идемпотентно `topo_group_send_table_request(group, conn)`:
```c
if (((struct TOPO_GROUP_CONN_ITEM*)se->data)->conn == conn) {
topo_group_send_table_request(group, conn); /* повторно, идемпотентно */
DEBUG_INFO(..., "conn already in senders_list, re-request table (%s)", conn->log_name);
return;
}
```
`topo_group_send_table_request` не ведёт состояния — просто шлёт REQUEST_TABLE,
пир отвечает снимком таблицы (идемпотентно). Лишний дубль таблицы безвреден.
**Замечание:** в однопоточном `test_dm_e2e` (и `test_chat_join_e2e`) из-за
детерминированного порядка группа успевает создаться до прихода REQUEST_TABLE —
поэтому тест проходит. Реальная гонка остаётся для асинхронных сетей. Отдельный
воспроизводимый тест для этой гонки пока не написан.
[ ] **«packet undecryptable» на линках C** (вторичное). A, узнав B через BGP, через
`topo_group_connect` пытается поднять прямой A↔B линк (`seskey 4777737b`), но B не
настроен принимать A → crypto-шум каждые ~1с. Разобраться с авто-подключением
(не пытаться соединяться с узлами, для которых нет `[client]`/ncd-конфигурации,
либо обрабатывать неудачу без спама).
[+] **Обновить GUI call-сайты `tools/chatgui` (Qt, desktop)** — сделано.
- В `chat_core.h` вынесены trampoline-структуры из `.c`: `update_my_name_arg`,
`save_ui_state_arg`, `chat_setting_arg` (были в chat_profile.c/chat_core.c).
- `gui_bridge.h/impl`: `gui_bridge_set_inst()`/`gui_bridge_get_inst()` (+`g_inst`).
- `utun_node.cpp`: `gui_bridge_set_inst(m_instance)`, `chat_event_set_handler(m_instance,…)`,
`chat_core_sync_my_addresses(m_instance)`.
- Обновлены ~45 call-сайтов в 16 файлах (`mainwindow`, `messagelist`, `messagedelegate`,
`memberlistmodel`, `memberpropsdialog`, `accountlist`, `invitedialog`, `inviteby`,
`joindialog`, `settingsdialog`, `channelsettingsdialog`, `soundsettingspage`,
`audiodevicesettingspage`, `storagesettingspage`, `statuspage`, `connmonitorwindow`).
- Сборка `cmake --build . --target vibechat` — OK.
Попутно: `chat_sync_connect_from_invite` писал `cs->pending_*` на вызывающем (GUI) потоке —
перенёс в `cm_invite_trampoline` (uasync-поток), убрав гонку; заодно исправил утечку
`inv->addrs_data` и добавил null-check `cs` в трамплине.
[+] **Прогон всех тестов** — сделано. `make clean && make -j4` OK; все тесты проходят
(кроме `test_auto_socket_dynamic` — пропуск «requires root»). `test_dm_e2e` стабилен
в нескольких прогонах. В `test_dm_e2e` при teardown остаётся timer-leak (~68 узлов,
`router_no_route`), не влияет на результат — стоит разобрать отдельно.
## Открытые флаки
[ ] **test_etcp_reconnect** — флаки под параллельной нагрузкой `make check -j4`:
phase 4 (reconnect после server restart) таймаутит (`sent=480 recv=0`), при одиночном
запуске стабильно `TEST PASSED`. Наш код (chat_join/chat_sync) его не трогает.
+118 -1
View File
@@ -20,6 +20,7 @@
#include <stdlib.h>
#include <string.h>
#include <stdio.h>
#include <ctype.h>
#ifdef _WIN32
#include <iphlpapi.h>
@@ -37,6 +38,9 @@ struct adns_query {
void* timer; /* handle таймаута (NULL когда нет) */
int finished;
char* name;
int fast; /* 1 = готовый /etc/hosts-ответ, без libdns */
struct adns_result fast_result;
void* call_soon_id; /* отложенный вызов (fast-путь и первый драйв slow-пути) */
};
static void adns_drive(struct adns_query* q);
@@ -44,6 +48,8 @@ static void adns_finish(struct adns_query* q, int status, struct sockaddr_in* ad
static void adns_read_cb(socket_t sock, void* arg);
static void adns_write_cb(socket_t sock, void* arg);
static void adns_timer_cb(void* arg);
static void adns_fast_ready_cb(void* arg);
static int adns_hosts_lookup(const char* name, struct sockaddr_in* out);
/* ─── system DNS servers discovery ─── */
@@ -124,6 +130,49 @@ int adns_system_servers(struct sockaddr_in* out, int max) {
}
#endif
/* ─── синхронный /etc/hosts-лукап (быстрый путь, без libdns) ─── */
// Ищет name в /etc/hosts. Возвращает 1 если найден IPv4-адрес (out заполнен).
// Блокирующее чтение небольшого файла — только для локальных имён (localhost и т.п.).
static int adns_hosts_lookup(const char* name, struct sockaddr_in* out) {
#ifndef _WIN32
if (!name || !out) return 0;
FILE* f = fopen("/etc/hosts", "r");
if (!f) return 0;
char line[512];
int found = 0;
while (!found && fgets(line, sizeof(line), f)) {
char ip[64]; int n = 0;
const char* p = line;
while (*p && !isspace(*p) && n < 63) ip[n++] = *p++;
ip[n] = '\0';
if (!ip[0] || ip[0] == '#') continue;
struct in_addr a;
if (inet_pton(AF_INET, ip, &a) != 1) continue;
while (*p && isspace(*p)) p++;
while (*p && *p != '\n') {
if (*p == '#') break;
char h[256]; int hn = 0;
while (*p && !isspace(*p) && *p != '#' && hn < 255) h[hn++] = *p++;
h[hn] = '\0';
if (h[0] && strcasecmp(h, name) == 0) {
memset(out, 0, sizeof(*out));
out->sin_family = AF_INET;
out->sin_addr = a;
found = 1;
break;
}
while (*p && isspace(*p)) p++;
}
}
fclose(f);
return found;
#else
(void)name; (void)out;
return 0;
#endif
}
/* ─── resolver setup ─── */
static struct dns_resolver* adns_open_resolver(const struct sockaddr_in* servers, int count,
@@ -155,6 +204,11 @@ static struct dns_resolver* adns_open_resolver(const struct sockaddr_in* servers
dns_resconf_close(rc);
return NULL;
}
#ifndef _WIN32
// Локальные имена из /etc/hosts (localhost и т.п.). Не фатально, если файла нет.
if (dns_hosts_loadpath(hosts, "/etc/hosts") != 0)
DEBUG_DEBUG(ADNS_DEBUG_CAT, "adns: /etc/hosts not loaded");
#endif
struct dns_hints* hints = dns_hints_local(rc, &error);
if (!hints) {
@@ -326,6 +380,34 @@ static void adns_finish(struct adns_query* q, int status, struct sockaddr_in* ad
u_free(name);
}
// Отложенный вызов коллбэка для fast-пути (/etc/hosts): сохраняет async-контракт
// (adns_resolve возвращает handle, коллбэк вызывается в потоке event loop).
static void adns_fast_ready_cb(void* arg) {
struct adns_query* q = (struct adns_query*)arg;
q->call_soon_id = NULL;
if (q->finished) return; // отменён до срабатывания
q->finished = 1;
adns_done_cb cb = q->cb;
void* a = q->arg;
struct adns_result res = q->fast_result;
char* nm = q->name;
q->name = NULL;
u_free(q);
if (cb) cb(&res, a);
u_free(nm);
}
// Отложенный первый драйв slow-пути (libdns): adns_resolve возвращает живой handle,
// а реальный запуск запроса происходит в event loop. Это исключает ситуацию, когда
// dns_res_check завершается синхронно прямо внутри adns_resolve() и освобождает q
// до того, как вызывающий код получил и сохранил handle.
static void adns_drive_trampoline(void* arg) {
struct adns_query* q = (struct adns_query*)arg;
q->call_soon_id = NULL;
if (q->finished) return; // отменён до срабатывания
adns_drive(q);
}
/* ─── public API ─── */
struct adns_query* adns_resolve(struct UASYNC* ua, const char* name,
@@ -334,6 +416,28 @@ struct adns_query* adns_resolve(struct UASYNC* ua, const char* name,
if (!ua || !name || !name[0] || !cb) return NULL;
if (strlen(name) > 255) { DEBUG_ERROR(ADNS_DEBUG_CAT, "adns: name too long"); return NULL; }
// Быстрый путь: имя из /etc/hosts (localhost и т.п.) — без libdns и сети.
// libdns по умолчанию идёт в DNS первым (lookup="bf"), поэтому локальные имена
// ждали бы таймаут DNS-запроса (2×2s); обрабатываем их сразу.
{
struct sockaddr_in ha;
if (adns_hosts_lookup(name, &ha)) {
struct adns_query* q = u_calloc(1, sizeof(*q));
if (!q) { DEBUG_ERROR(ADNS_DEBUG_CAT, "adns: calloc failed"); return NULL; }
q->ua = ua; q->cb = cb; q->arg = arg;
q->name = u_strdup(name);
if (!q->name) { u_free(q); DEBUG_ERROR(ADNS_DEBUG_CAT, "adns: strdup failed"); return NULL; }
q->fast = 1;
q->fast_result.status = ADNS_OK;
q->fast_result.count = 1;
q->fast_result.addrs[0] = ha;
q->call_soon_id = uasync_call_soon(ua, q, adns_fast_ready_cb);
if (!q->call_soon_id) { u_free(q->name); u_free(q); return NULL; }
DEBUG_DEBUG(ADNS_DEBUG_CAT, "adns: '%s' resolved from /etc/hosts (fast)", name);
return q;
}
}
struct sockaddr_in servers[3];
int count;
if (opts && opts->server.sin_family != 0) {
@@ -373,13 +477,26 @@ struct adns_query* adns_resolve(struct UASYNC* ua, const char* name,
DEBUG_DEBUG(ADNS_DEBUG_CAT, "adns: resolving '%s' via %d server(s), timeout=%dms attempts=%d",
name, count, timeout_ms, attempts);
adns_drive(q);
q->call_soon_id = uasync_call_soon(ua, q, adns_drive_trampoline);
if (!q->call_soon_id) {
DEBUG_ERROR(ADNS_DEBUG_CAT, "adns: uasync_call_soon failed for '%s'", name);
dns_res_close(q->res);
u_free(q->name);
u_free(q);
return NULL;
}
return q;
}
void adns_cancel(struct adns_query* q) {
if (!q || q->finished) return;
q->finished = 1;
if (q->call_soon_id) { uasync_call_soon_cancel(q->ua, q->call_soon_id); q->call_soon_id = NULL; }
if (q->fast) {
u_free(q->name);
u_free(q);
return;
}
adns_teardown(q);
u_free(q->name);
u_free(q);
+3
View File
@@ -129,6 +129,9 @@ static const struct {
{"chat_sync", DEBUG_CATEGORY_CHAT_SYNC},
{"member_sync", DEBUG_CATEGORY_MEMBER_SYNC},
{"proxy", DEBUG_CATEGORY_PROXY},
{"video", DEBUG_CATEGORY_VIDEO},
{"reality", DEBUG_CATEGORY_REALITY},
{"dm", DEBUG_CATEGORY_DM},
{"all", DEBUG_CATEGORY_ALL},
{NULL, DEBUG_CATEGORY_NONE}
};
+5 -2
View File
@@ -41,7 +41,7 @@ typedef int debug_category_t;
#define DEBUG_CATEGORY_NONE 0
#define DEBUG_CATEGORY_SYS 1 // System: u_async, ll_queue, memory, timing
// #define DEBUG_CATEGORY_LL_QUEUE 2 (merged into SYS)
#define DEBUG_CATEGORY_CONNECTION 3 // Connection: links, keepalive, status events
#define DEBUG_CATEGORY_CONNECTION 3 // Connection: connect/reconnect/disconnect events only
#define DEBUG_CATEGORY_ETCP 4 // ETCP: protocol, BBR, normalizer
#define DEBUG_CATEGORY_CRYPTO 5 // crypto operations
// #define DEBUG_CATEGORY_MEMORY 6 (merged into SYS)
@@ -67,7 +67,10 @@ typedef int debug_category_t;
#define DEBUG_CATEGORY_CHAT_SYNC 26 // Message DB sync (INIT_SYNC, PUSH, chain hash, recalc)
#define DEBUG_CATEGORY_MEMBER_SYNC 27 // Member + address + merkle sync
#define DEBUG_CATEGORY_PROXY 28 // Proxy modules (SOCKS5, TCP, UDP, ICMP)
#define DEBUG_CATEGORY_COUNT 29 // Total number of categories
#define DEBUG_CATEGORY_VIDEO 29 // Video module (probe/transcode)
#define DEBUG_CATEGORY_REALITY 30 // REALITY TLS camouflage
#define DEBUG_CATEGORY_DM 31 // DM — direct p2p chat between two users
#define DEBUG_CATEGORY_COUNT 32 // Total number of categories
#define DEBUG_CATEGORY_ALL (-1) // special value for all categories
/* Debug configuration structure */
+23 -17
View File
@@ -3403,27 +3403,33 @@ size_t dns_txt_print(void *dst_, size_t lim, struct dns_txt *txt) {
} /* dns_txt_print() */
typedef int (*dns_parse_fn)(union dns_any *, struct dns_rr *, struct dns_packet *);
typedef int (*dns_push_fn)(struct dns_packet *, union dns_any *);
typedef int (*dns_cmp_fn)(const union dns_any *, const union dns_any *);
typedef size_t (*dns_print_fn)(void *, size_t, union dns_any *);
typedef size_t (*dns_cname_fn)(void *, size_t, union dns_any *);
static const struct {
enum dns_type type;
const char *name;
int (*parse)();
int (*push)();
int (*cmp)();
size_t (*print)();
size_t (*cname)();
dns_parse_fn parse;
dns_push_fn push;
dns_cmp_fn cmp;
dns_print_fn print;
dns_cname_fn cname;
} dns_rrtypes[] = {
{ DNS_T_A, "A", &dns_a_parse, &dns_a_push, &dns_a_cmp, &dns_a_print, 0 },
{ DNS_T_AAAA, "AAAA", &dns_aaaa_parse, &dns_aaaa_push, &dns_aaaa_cmp, &dns_aaaa_print, 0 },
{ DNS_T_MX, "MX", &dns_mx_parse, &dns_mx_push, &dns_mx_cmp, &dns_mx_print, &dns_mx_cname },
{ DNS_T_NS, "NS", &dns_ns_parse, &dns_ns_push, &dns_ns_cmp, &dns_ns_print, &dns_ns_cname },
{ DNS_T_CNAME, "CNAME", &dns_cname_parse, &dns_cname_push, &dns_cname_cmp, &dns_cname_print, &dns_cname_cname },
{ DNS_T_SOA, "SOA", &dns_soa_parse, &dns_soa_push, &dns_soa_cmp, &dns_soa_print, 0 },
{ DNS_T_SRV, "SRV", &dns_srv_parse, &dns_srv_push, &dns_srv_cmp, &dns_srv_print, &dns_srv_cname },
{ DNS_T_OPT, "OPT", &dns_opt_parse, &dns_opt_push, &dns_opt_cmp, &dns_opt_print, 0 },
{ DNS_T_PTR, "PTR", &dns_ptr_parse, &dns_ptr_push, &dns_ptr_cmp, &dns_ptr_print, &dns_ptr_cname },
{ DNS_T_TXT, "TXT", &dns_txt_parse, &dns_txt_push, &dns_txt_cmp, &dns_txt_print, 0 },
{ DNS_T_SPF, "SPF", &dns_txt_parse, &dns_txt_push, &dns_txt_cmp, &dns_txt_print, 0 },
{ DNS_T_SSHFP, "SSHFP", &dns_sshfp_parse, &dns_sshfp_push, &dns_sshfp_cmp, &dns_sshfp_print, 0 },
{ DNS_T_A, "A", (dns_parse_fn)&dns_a_parse, (dns_push_fn)&dns_a_push, (dns_cmp_fn)&dns_a_cmp, (dns_print_fn)&dns_a_print, 0 },
{ DNS_T_AAAA, "AAAA", (dns_parse_fn)&dns_aaaa_parse, (dns_push_fn)&dns_aaaa_push, (dns_cmp_fn)&dns_aaaa_cmp, (dns_print_fn)&dns_aaaa_print, 0 },
{ DNS_T_MX, "MX", (dns_parse_fn)&dns_mx_parse, (dns_push_fn)&dns_mx_push, (dns_cmp_fn)&dns_mx_cmp, (dns_print_fn)&dns_mx_print, (dns_cname_fn)&dns_mx_cname },
{ DNS_T_NS, "NS", (dns_parse_fn)&dns_ns_parse, (dns_push_fn)&dns_ns_push, (dns_cmp_fn)&dns_ns_cmp, (dns_print_fn)&dns_ns_print, (dns_cname_fn)&dns_ns_cname },
{ DNS_T_CNAME, "CNAME", (dns_parse_fn)&dns_cname_parse, (dns_push_fn)&dns_cname_push, (dns_cmp_fn)&dns_cname_cmp, (dns_print_fn)&dns_cname_print, (dns_cname_fn)&dns_cname_cname },
{ DNS_T_SOA, "SOA", (dns_parse_fn)&dns_soa_parse, (dns_push_fn)&dns_soa_push, (dns_cmp_fn)&dns_soa_cmp, (dns_print_fn)&dns_soa_print, 0 },
{ DNS_T_SRV, "SRV", (dns_parse_fn)&dns_srv_parse, (dns_push_fn)&dns_srv_push, (dns_cmp_fn)&dns_srv_cmp, (dns_print_fn)&dns_srv_print, (dns_cname_fn)&dns_srv_cname },
{ DNS_T_OPT, "OPT", (dns_parse_fn)&dns_opt_parse, (dns_push_fn)&dns_opt_push, (dns_cmp_fn)&dns_opt_cmp, (dns_print_fn)&dns_opt_print, 0 },
{ DNS_T_PTR, "PTR", (dns_parse_fn)&dns_ptr_parse, (dns_push_fn)&dns_ptr_push, (dns_cmp_fn)&dns_ptr_cmp, (dns_print_fn)&dns_ptr_print, (dns_cname_fn)&dns_ptr_cname },
{ DNS_T_TXT, "TXT", (dns_parse_fn)&dns_txt_parse, (dns_push_fn)&dns_txt_push, (dns_cmp_fn)&dns_txt_cmp, (dns_print_fn)&dns_txt_print, 0 },
{ DNS_T_SPF, "SPF", (dns_parse_fn)&dns_txt_parse, (dns_push_fn)&dns_txt_push, (dns_cmp_fn)&dns_txt_cmp, (dns_print_fn)&dns_txt_print, 0 },
{ DNS_T_SSHFP, "SSHFP", (dns_parse_fn)&dns_sshfp_parse, (dns_push_fn)&dns_sshfp_push, (dns_cmp_fn)&dns_sshfp_cmp, (dns_print_fn)&dns_sshfp_print, 0 },
}; /* dns_rrtypes[] */
+1 -1
View File
@@ -508,7 +508,7 @@ struct dns_rr_i {
int follow;
int (*sort)();
int (*sort)(struct dns_rr *, struct dns_rr *, struct dns_rr_i *, struct dns_packet *);
unsigned args[2];
struct {
+7 -7
View File
@@ -23,24 +23,24 @@ int get_outgoing_local_ip(const struct sockaddr_storage *local_addr,
struct sockaddr_storage *result)
{
if (!local_addr || !remote_addr || !result) {
DEBUG_ERROR(DEBUG_CATEGORY_CONNECTION, "get_outgoing_local_ip: NULL argument");
DEBUG_ERROR(DEBUG_CATEGORY_SOCKET, "get_outgoing_local_ip: NULL argument");
return -1;
}
int family = local_addr->ss_family;
if (family != AF_INET && family != AF_INET6) {
DEBUG_ERROR(DEBUG_CATEGORY_CONNECTION, "get_outgoing_local_ip: unsupported family %d", family);
DEBUG_ERROR(DEBUG_CATEGORY_SOCKET, "get_outgoing_local_ip: unsupported family %d", family);
return -1;
}
if (remote_addr->ss_family != family) {
DEBUG_ERROR(DEBUG_CATEGORY_CONNECTION, "get_outgoing_local_ip: family mismatch");
DEBUG_ERROR(DEBUG_CATEGORY_SOCKET, "get_outgoing_local_ip: family mismatch");
return -1;
}
socket_t sock = socket_create_udp(family);
if (sock == SOCKET_INVALID) {
DEBUG_ERROR(DEBUG_CATEGORY_CONNECTION, "get_outgoing_local_ip: socket_create_udp failed");
DEBUG_ERROR(DEBUG_CATEGORY_SOCKET, "get_outgoing_local_ip: socket_create_udp failed");
return -1;
}
@@ -56,7 +56,7 @@ int get_outgoing_local_ip(const struct sockaddr_storage *local_addr,
socklen_t bind_len = (family == AF_INET) ? sizeof(struct sockaddr_in) : sizeof(struct sockaddr_in6);
if (bind(sock, (struct sockaddr *)&bind_addr, bind_len) != 0) {
int err = socket_get_error();
DEBUG_ERROR(DEBUG_CATEGORY_CONNECTION, "get_outgoing_local_ip: bind failed: %s", socket_strerror(err));
DEBUG_ERROR(DEBUG_CATEGORY_SOCKET, "get_outgoing_local_ip: bind failed: %s", socket_strerror(err));
socket_close_wrapper(sock);
return -1;
}
@@ -65,7 +65,7 @@ int get_outgoing_local_ip(const struct sockaddr_storage *local_addr,
socklen_t remote_len = (family == AF_INET) ? sizeof(struct sockaddr_in) : sizeof(struct sockaddr_in6);
if (connect(sock, (struct sockaddr *)remote_addr, remote_len) != 0) {
int err = socket_get_error();
DEBUG_ERROR(DEBUG_CATEGORY_CONNECTION, "get_outgoing_local_ip: connect failed: %s", socket_strerror(err));
DEBUG_ERROR(DEBUG_CATEGORY_SOCKET, "get_outgoing_local_ip: connect failed: %s", socket_strerror(err));
socket_close_wrapper(sock);
return -1;
}
@@ -75,7 +75,7 @@ int get_outgoing_local_ip(const struct sockaddr_storage *local_addr,
socklen_t local_len = sizeof(local);
if (getsockname(sock, (struct sockaddr *)&local, &local_len) != 0) {
int err = socket_get_error();
DEBUG_ERROR(DEBUG_CATEGORY_CONNECTION, "get_outgoing_local_ip: getsockname failed: %s", socket_strerror(err));
DEBUG_ERROR(DEBUG_CATEGORY_SOCKET, "get_outgoing_local_ip: getsockname failed: %s", socket_strerror(err));
socket_close_wrapper(sock);
return -1;
}
+41 -22
View File
@@ -60,6 +60,12 @@ int random_bytes(uint8_t *buffer, size_t len) {
#endif
}
/* Глобальный unicast IPv6 (2000::/3). При выборе адреса интерфейса предпочитаем его
* ULA (fc00::/7) и link-local (fe80::/10). */
static int ipv6_is_global_unicast(const uint8_t* a) {
return (a[0] & 0xe0) == 0x20;
}
#ifndef _WIN32
uint32_t get_interface_ip_by_index(uint32_t netif_index) {
struct ifaddrs *ifaddr, *ifa;
@@ -89,7 +95,8 @@ int get_interface_ipv6_by_index(uint32_t netif_index, int temporary, uint8_t* ad
struct ifaddrs *ifaddr, *ifa;
if (getifaddrs(&ifaddr) == -1) return -1;
int found = 0;
uint8_t global_match[16], local_match[16];
int has_global = 0, has_local = 0;
for (ifa = ifaddr; ifa != NULL; ifa = ifa->ifa_next) {
if (ifa->ifa_addr == NULL) continue;
if (ifa->ifa_name == NULL) continue;
@@ -100,20 +107,18 @@ int get_interface_ipv6_by_index(uint32_t netif_index, int temporary, uint8_t* ad
struct sockaddr_in6* sin6 = (struct sockaddr_in6*)ifa->ifa_addr;
if ((sin6->sin6_addr.s6_addr[0] & 0xfe) == 0xfe && sin6->sin6_addr.s6_addr[1] == 0x80) continue;
int is_temporary = (ifa->ifa_flags & IFA_F_TEMPORARY) ? 1 : 0;
if (temporary && is_temporary) {
memcpy(addr_out, &sin6->sin6_addr, 16);
found = 1;
break;
}
if (!temporary && !is_temporary) {
memcpy(addr_out, &sin6->sin6_addr, 16);
found = 1;
break;
if ((temporary && !is_temporary) || (!temporary && is_temporary)) continue;
if (ipv6_is_global_unicast(sin6->sin6_addr.s6_addr)) {
if (!has_global) { memcpy(global_match, &sin6->sin6_addr, 16); has_global = 1; }
} else {
if (!has_local) { memcpy(local_match, &sin6->sin6_addr, 16); has_local = 1; }
}
}
freeifaddrs(ifaddr);
return found ? 0 : -1;
if (has_global) { memcpy(addr_out, global_match, 16); return 0; }
if (has_local) { memcpy(addr_out, local_match, 16); return 0; }
return -1;
}
#ifdef __linux__
@@ -151,8 +156,8 @@ int get_interface_ipv6_addr_nl(uint32_t netif_index, int prefer_stable, uint8_t*
if (send(nl_sock, &req, req.nlh.nlmsg_len, 0) < 0) { close(nl_sock); return -1; }
uint8_t fallback[16], stable_addr[16], temp_addr[16];
int has_fallback = 0, has_stable = 0, has_temp = 0;
uint8_t fallback[16], g_stable[16], g_temp[16], l_stable[16], l_temp[16];
int has_fallback = 0, has_g_stable = 0, has_g_temp = 0, has_l_stable = 0, has_l_temp = 0;
char buf[8192];
int done = 0;
@@ -181,19 +186,32 @@ int get_interface_ipv6_addr_nl(uint32_t netif_index, int prefer_stable, uint8_t*
int is_temp = (ifa_flags & IFA_F_TEMPORARY) != 0;
int is_mngtmp = (ifa_flags & IFA_F_MANAGETEMPADDR) != 0;
int is_global = ipv6_is_global_unicast(addr);
if (!has_fallback) { memcpy(fallback, addr, 16); has_fallback = 1; }
if (!has_stable && (!is_temp || is_mngtmp)) { memcpy(stable_addr, addr, 16); has_stable = 1; }
if (!has_temp && is_temp && !is_mngtmp) { memcpy(temp_addr, addr, 16); has_temp = 1; }
if (is_global) {
if (!has_g_stable && (!is_temp || is_mngtmp)) { memcpy(g_stable, addr, 16); has_g_stable = 1; }
if (!has_g_temp && is_temp && !is_mngtmp) { memcpy(g_temp, addr, 16); has_g_temp = 1; }
} else {
if (!has_l_stable && (!is_temp || is_mngtmp)) { memcpy(l_stable, addr, 16); has_l_stable = 1; }
if (!has_l_temp && is_temp && !is_mngtmp) { memcpy(l_temp, addr, 16); has_l_temp = 1; }
}
}
}
close(nl_sock);
if (prefer_stable && has_stable) { memcpy(addr_out, stable_addr, 16); return 0; }
if (!prefer_stable && has_temp) { memcpy(addr_out, temp_addr, 16); return 0; }
if (has_stable) { memcpy(addr_out, stable_addr, 16); return 0; }
if (has_temp) { memcpy(addr_out, temp_addr, 16); return 0; }
if (prefer_stable) {
if (has_g_stable) { memcpy(addr_out, g_stable, 16); return 0; }
if (has_g_temp) { memcpy(addr_out, g_temp, 16); return 0; }
if (has_l_stable) { memcpy(addr_out, l_stable, 16); return 0; }
if (has_l_temp) { memcpy(addr_out, l_temp, 16); return 0; }
} else {
if (has_g_temp) { memcpy(addr_out, g_temp, 16); return 0; }
if (has_g_stable) { memcpy(addr_out, g_stable, 16); return 0; }
if (has_l_temp) { memcpy(addr_out, l_temp, 16); return 0; }
if (has_l_stable) { memcpy(addr_out, l_stable, 16); return 0; }
}
if (has_fallback) { memcpy(addr_out, fallback, 16); return 0; }
return -1;
}
@@ -335,6 +353,7 @@ int get_interface_ipv6_addr_nl(uint32_t netif_index, int prefer_stable, uint8_t*
PIP_ADAPTER_ADDRESSES adapters = plat_win_get_adapters(AF_INET6);
if (!adapters) return -1;
int found = 0;
uint8_t local_match[16]; int has_local = 0;
PIP_ADAPTER_ADDRESSES a = plat_win_find_adapter(adapters, netif_index);
if (a) {
for (PIP_ADAPTER_UNICAST_ADDRESS u = a->FirstUnicastAddress; u; u = u->Next) {
@@ -344,12 +363,12 @@ int get_interface_ipv6_addr_nl(uint32_t netif_index, int prefer_stable, uint8_t*
{ static const uint8_t lb[16] = {0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1};
if (memcmp(b, lb, 16) == 0) continue; } /* loopback */
{ static const uint8_t z[16]; if (memcmp(b, z, 16) == 0) continue; } /* :: */
memcpy(addr_out, b, 16);
found = 1;
break;
if (ipv6_is_global_unicast(b)) { memcpy(addr_out, b, 16); found = 1; break; }
if (!has_local) { memcpy(local_match, b, 16); has_local = 1; }
}
}
free(adapters);
if (!found && has_local) { memcpy(addr_out, local_match, 16); found = 1; }
return found ? 0 : -1;
}
+55
View File
@@ -0,0 +1,55 @@
bin
lib
# Win build files
*.dll
*.exe
*.lib
# GNU build files
*.o
*.so*
*.lo
*.P*
*.la*
*.a
*.pc
*config*
Makefile
Makefile.in
.libs
aclocal.m4
autom4te.cache
stamp-*
libtool
soundstretch
# Files generated by MSVC
*.bsc
*.suo
*.sdf
*.filters
*.user
source/SoundTouch/Win32/
source/SoundTouch/x64/
source/SoundStretch/Win32/
source/SoundStretch/x64/
source/SoundTouchDll/Win32/
source/SoundTouchDll/x64/
source/SoundTouchDll/DllTest/Win32/
source/SoundTouchDll/DllTest/x64/
.vs
# Files generated by Android Studio
source/android-lib/.gradle
source/android-lib/.idea
**/*.iml
source/android-lib/local.properties
source/android-lib/build
source/android-lib/.externalNativeBuild
# CMake build directory
build*
CMakeFiles
CMakeCache.txt
*.cmake
+131
View File
@@ -0,0 +1,131 @@
cmake_minimum_required(VERSION 3.16)
project(SoundTouch VERSION 2.3.3 LANGUAGES CXX)
set(CMAKE_CXX_STANDARD 17)
include(GNUInstallDirs)
set(COMPILE_OPTIONS)
if(MSVC)
list(APPEND COMPILE_OPTIONS /O2 /fp:fast)
else()
list(APPEND COMPILE_OPTIONS -Wall -Wextra -Wzero-as-null-pointer-constant -Wno-unknown-pragmas)
if(EMSCRIPTEN)
list(APPEND COMPILE_OPTIONS -O3)
else()
list(APPEND COMPILE_OPTIONS -Ofast)
endif()
endif()
set(CMAKE_C_VISIBILITY_PRESET hidden)
set(CMAKE_CXX_VISIBILITY_PRESET hidden)
#####################
# SoundTouch library
add_library(SoundTouch STATIC
source/SoundTouch/AAFilter.cpp
source/SoundTouch/BPMDetect.cpp
source/SoundTouch/cpu_detect_x86.cpp
source/SoundTouch/FIFOSampleBuffer.cpp
source/SoundTouch/FIRFilter.cpp
source/SoundTouch/InterpolateCubic.cpp
source/SoundTouch/InterpolateLinear.cpp
source/SoundTouch/InterpolateShannon.cpp
source/SoundTouch/mmx_optimized.cpp
source/SoundTouch/PeakFinder.cpp
source/SoundTouch/RateTransposer.cpp
source/SoundTouch/SoundTouch.cpp
source/SoundTouch/sse_optimized.cpp
source/SoundTouch/TDStretch.cpp
)
target_include_directories(SoundTouch PUBLIC
$<BUILD_INTERFACE:${CMAKE_CURRENT_SOURCE_DIR}/include>
$<INSTALL_INTERFACE:${CMAKE_INSTALL_INCLUDEDIR}>
)
target_include_directories(SoundTouch PRIVATE
${CMAKE_CURRENT_SOURCE_DIR}/include/soundtouch
)
target_compile_definitions(SoundTouch PRIVATE ${COMPILE_DEFINITIONS})
target_compile_options(SoundTouch PRIVATE ${COMPILE_OPTIONS})
set_target_properties(SoundTouch PROPERTIES OUTPUT_NAME "soundtouch-static")
option(INTEGER_SAMPLES "Use integers instead of floats for samples" OFF)
if(INTEGER_SAMPLES)
target_compile_definitions(SoundTouch PRIVATE SOUNDTOUCH_INTEGER_SAMPLES)
set(SOUNDTOUCH_INTEGER_SAMPLES TRUE)
else()
target_compile_definitions(SoundTouch PRIVATE SOUNDTOUCH_FLOAT_SAMPLES)
set(SOUNDTOUCH_FLOAT_SAMPLES TRUE)
endif()
if(CMAKE_SYSTEM_PROCESSOR MATCHES "^(armv7.*)$")
target_compile_options(SoundTouch PRIVATE -mfpu=neon)
endif()
install(
FILES
include/soundtouch/BPMDetect.h
include/soundtouch/FIFOSampleBuffer.h
include/soundtouch/FIFOSamplePipe.h
include/soundtouch/STTypes.h
include/soundtouch/SoundTouch.h
DESTINATION
"${CMAKE_INSTALL_INCLUDEDIR}/soundtouch"
)
########################
# SoundTouchDll library
option(SOUNDTOUCH_DLL "Build SoundTouchDLL C wrapper library" ON)
if(SOUNDTOUCH_DLL)
add_library(SoundTouchDLL SHARED
source/SoundTouchDLL/SoundTouchDLL.cpp
source/SoundTouchDLL/SoundTouchDLL.rc
)
set_target_properties(SoundTouch PROPERTIES POSITION_INDEPENDENT_CODE TRUE)
target_compile_options(SoundTouchDLL PRIVATE ${COMPILE_OPTIONS})
set_target_properties(SoundTouchDLL PROPERTIES CXX_VISIBILITY_PRESET hidden)
target_compile_definitions(SoundTouchDLL PRIVATE DLL_EXPORTS)
target_include_directories(SoundTouchDLL INTERFACE $<BUILD_INTERFACE:${CMAKE_CURRENT_SOURCE_DIR}/include>)
target_include_directories(SoundTouchDLL PRIVATE "${CMAKE_CURRENT_SOURCE_DIR}/include/soundtouch")
target_link_libraries(SoundTouchDLL PRIVATE SoundTouch)
set_target_properties(SoundTouchDLL PROPERTIES OUTPUT_NAME "soundtouch")
if(NOT ANDROID)
set_target_properties(SoundTouchDLL PROPERTIES VERSION ${CMAKE_PROJECT_VERSION})
if(NOT WIN32)
set_target_properties(SoundTouchDLL PROPERTIES SOVERSION ${PROJECT_VERSION_MAJOR})
endif()
endif()
install(FILES include/soundtouch/SoundTouchDLL.h DESTINATION "${CMAKE_INSTALL_INCLUDEDIR}/soundtouch")
install(TARGETS SoundTouchDLL EXPORT SoundTouchTargets)
endif()
########################
# CMake config
include(CMakePackageConfigHelpers)
set(SOUNDTOUCH_INSTALL_CMAKEDIR "${CMAKE_INSTALL_LIBDIR}/cmake/SoundTouch")
install(
EXPORT SoundTouchTargets
FILE SoundTouchTargets.cmake
NAMESPACE SoundTouch::
DESTINATION "${SOUNDTOUCH_INSTALL_CMAKEDIR}"
)
configure_package_config_file(SoundTouchConfig.cmake.in
"${CMAKE_CURRENT_BINARY_DIR}/SoundTouchConfig.cmake"
INSTALL_DESTINATION "${SOUNDTOUCH_INSTALL_CMAKEDIR}"
)
write_basic_package_version_file(
"${CMAKE_CURRENT_BINARY_DIR}/SoundTouchConfigVersion.cmake"
VERSION "${CMAKE_PROJECT_VERSION}"
COMPATIBILITY SameMajorVersion
)
install(
FILES
"${CMAKE_CURRENT_BINARY_DIR}/SoundTouchConfig.cmake"
"${CMAKE_CURRENT_BINARY_DIR}/SoundTouchConfigVersion.cmake"
DESTINATION "${SOUNDTOUCH_INSTALL_CMAKEDIR}"
)
+458
View File
@@ -0,0 +1,458 @@
GNU LESSER GENERAL PUBLIC LICENSE
Version 2.1, February 1999
Copyright (C) 1991, 1999 Free Software Foundation, Inc.
51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
Everyone is permitted to copy and distribute verbatim copies
of this license document, but changing it is not allowed.
[This is the first released version of the Lesser GPL. It also counts
as the successor of the GNU Library Public License, version 2, hence
the version number 2.1.]
Preamble
The licenses for most software are designed to take away your
freedom to share and change it. By contrast, the GNU General Public
Licenses are intended to guarantee your freedom to share and change
free software--to make sure the software is free for all its users.
This license, the Lesser General Public License, applies to some
specially designated software packages--typically libraries--of the
Free Software Foundation and other authors who decide to use it. You
can use it too, but we suggest you first think carefully about whether
this license or the ordinary General Public License is the better
strategy to use in any particular case, based on the explanations below.
When we speak of free software, we are referring to freedom of use,
not price. Our General Public Licenses are designed to make sure that
you have the freedom to distribute copies of free software (and charge
for this service if you wish); that you receive source code or can get
it if you want it; that you can change the software and use pieces of
it in new free programs; and that you are informed that you can do
these things.
To protect your rights, we need to make restrictions that forbid
distributors to deny you these rights or to ask you to surrender these
rights. These restrictions translate to certain responsibilities for
you if you distribute copies of the library or if you modify it.
For example, if you distribute copies of the library, whether gratis
or for a fee, you must give the recipients all the rights that we gave
you. You must make sure that they, too, receive or can get the source
code. If you link other code with the library, you must provide
complete object files to the recipients, so that they can relink them
with the library after making changes to the library and recompiling
it. And you must show them these terms so they know their rights.
We protect your rights with a two-step method: (1) we copyright the
library, and (2) we offer you this license, which gives you legal
permission to copy, distribute and/or modify the library.
To protect each distributor, we want to make it very clear that
there is no warranty for the free library. Also, if the library is
modified by someone else and passed on, the recipients should know
that what they have is not the original version, so that the original
author's reputation will not be affected by problems that might be
introduced by others.
Finally, software patents pose a constant threat to the existence of
any free program. We wish to make sure that a company cannot
effectively restrict the users of a free program by obtaining a
restrictive license from a patent holder. Therefore, we insist that
any patent license obtained for a version of the library must be
consistent with the full freedom of use specified in this license.
Most GNU software, including some libraries, is covered by the
ordinary GNU General Public License. This license, the GNU Lesser
General Public License, applies to certain designated libraries, and
is quite different from the ordinary General Public License. We use
this license for certain libraries in order to permit linking those
libraries into non-free programs.
When a program is linked with a library, whether statically or using
a shared library, the combination of the two is legally speaking a
combined work, a derivative of the original library. The ordinary
General Public License therefore permits such linking only if the
entire combination fits its criteria of freedom. The Lesser General
Public License permits more lax criteria for linking other code with
the library.
We call this license the "Lesser" General Public License because it
does Less to protect the user's freedom than the ordinary General
Public License. It also provides other free software developers Less
of an advantage over competing non-free programs. These disadvantages
are the reason we use the ordinary General Public License for many
libraries. However, the Lesser license provides advantages in certain
special circumstances.
For example, on rare occasions, there may be a special need to
encourage the widest possible use of a certain library, so that it becomes
a de-facto standard. To achieve this, non-free programs must be
allowed to use the library. A more frequent case is that a free
library does the same job as widely used non-free libraries. In this
case, there is little to gain by limiting the free library to free
software only, so we use the Lesser General Public License.
In other cases, permission to use a particular library in non-free
programs enables a greater number of people to use a large body of
free software. For example, permission to use the GNU C Library in
non-free programs enables many more people to use the whole GNU
operating system, as well as its variant, the GNU/Linux operating
system.
Although the Lesser General Public License is Less protective of the
users' freedom, it does ensure that the user of a program that is
linked with the Library has the freedom and the wherewithal to run
that program using a modified version of the Library.
The precise terms and conditions for copying, distribution and
modification follow. Pay close attention to the difference between a
"work based on the library" and a "work that uses the library". The
former contains code derived from the library, whereas the latter must
be combined with the library in order to run.
GNU LESSER GENERAL PUBLIC LICENSE
TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION
0. This License Agreement applies to any software library or other
program which contains a notice placed by the copyright holder or
other authorized party saying it may be distributed under the terms of
this Lesser General Public License (also called "this License").
Each licensee is addressed as "you".
A "library" means a collection of software functions and/or data
prepared so as to be conveniently linked with application programs
(which use some of those functions and data) to form executables.
The "Library", below, refers to any such software library or work
which has been distributed under these terms. A "work based on the
Library" means either the Library or any derivative work under
copyright law: that is to say, a work containing the Library or a
portion of it, either verbatim or with modifications and/or translated
straightforwardly into another language. (Hereinafter, translation is
included without limitation in the term "modification".)
"Source code" for a work means the preferred form of the work for
making modifications to it. For a library, complete source code means
all the source code for all modules it contains, plus any associated
interface definition files, plus the scripts used to control compilation
and installation of the library.
Activities other than copying, distribution and modification are not
covered by this License; they are outside its scope. The act of
running a program using the Library is not restricted, and output from
such a program is covered only if its contents constitute a work based
on the Library (independent of the use of the Library in a tool for
writing it). Whether that is true depends on what the Library does
and what the program that uses the Library does.
1. You may copy and distribute verbatim copies of the Library's
complete source code as you receive it, in any medium, provided that
you conspicuously and appropriately publish on each copy an
appropriate copyright notice and disclaimer of warranty; keep intact
all the notices that refer to this License and to the absence of any
warranty; and distribute a copy of this License along with the
Library.
You may charge a fee for the physical act of transferring a copy,
and you may at your option offer warranty protection in exchange for a
fee.
2. You may modify your copy or copies of the Library or any portion
of it, thus forming a work based on the Library, and copy and
distribute such modifications or work under the terms of Section 1
above, provided that you also meet all of these conditions:
a) The modified work must itself be a software library.
b) You must cause the files modified to carry prominent notices
stating that you changed the files and the date of any change.
c) You must cause the whole of the work to be licensed at no
charge to all third parties under the terms of this License.
d) If a facility in the modified Library refers to a function or a
table of data to be supplied by an application program that uses
the facility, other than as an argument passed when the facility
is invoked, then you must make a good faith effort to ensure that,
in the event an application does not supply such function or
table, the facility still operates, and performs whatever part of
its purpose remains meaningful.
(For example, a function in a library to compute square roots has
a purpose that is entirely well-defined independent of the
application. Therefore, Subsection 2d requires that any
application-supplied function or table used by this function must
be optional: if the application does not supply it, the square
root function must still compute square roots.)
These requirements apply to the modified work as a whole. If
identifiable sections of that work are not derived from the Library,
and can be reasonably considered independent and separate works in
themselves, then this License, and its terms, do not apply to those
sections when you distribute them as separate works. But when you
distribute the same sections as part of a whole which is a work based
on the Library, the distribution of the whole must be on the terms of
this License, whose permissions for other licensees extend to the
entire whole, and thus to each and every part regardless of who wrote
it.
Thus, it is not the intent of this section to claim rights or contest
your rights to work written entirely by you; rather, the intent is to
exercise the right to control the distribution of derivative or
collective works based on the Library.
In addition, mere aggregation of another work not based on the Library
with the Library (or with a work based on the Library) on a volume of
a storage or distribution medium does not bring the other work under
the scope of this License.
3. You may opt to apply the terms of the ordinary GNU General Public
License instead of this License to a given copy of the Library. To do
this, you must alter all the notices that refer to this License, so
that they refer to the ordinary GNU General Public License, version 2,
instead of to this License. (If a newer version than version 2 of the
ordinary GNU General Public License has appeared, then you can specify
that version instead if you wish.) Do not make any other change in
these notices.
Once this change is made in a given copy, it is irreversible for
that copy, so the ordinary GNU General Public License applies to all
subsequent copies and derivative works made from that copy.
This option is useful when you wish to copy part of the code of
the Library into a program that is not a library.
4. You may copy and distribute the Library (or a portion or
derivative of it, under Section 2) in object code or executable form
under the terms of Sections 1 and 2 above provided that you accompany
it with the complete corresponding machine-readable source code, which
must be distributed under the terms of Sections 1 and 2 above on a
medium customarily used for software interchange.
If distribution of object code is made by offering access to copy
from a designated place, then offering equivalent access to copy the
source code from the same place satisfies the requirement to
distribute the source code, even though third parties are not
compelled to copy the source along with the object code.
5. A program that contains no derivative of any portion of the
Library, but is designed to work with the Library by being compiled or
linked with it, is called a "work that uses the Library". Such a
work, in isolation, is not a derivative work of the Library, and
therefore falls outside the scope of this License.
However, linking a "work that uses the Library" with the Library
creates an executable that is a derivative of the Library (because it
contains portions of the Library), rather than a "work that uses the
library". The executable is therefore covered by this License.
Section 6 states terms for distribution of such executables.
When a "work that uses the Library" uses material from a header file
that is part of the Library, the object code for the work may be a
derivative work of the Library even though the source code is not.
Whether this is true is especially significant if the work can be
linked without the Library, or if the work is itself a library. The
threshold for this to be true is not precisely defined by law.
If such an object file uses only numerical parameters, data
structure layouts and accessors, and small macros and small inline
functions (ten lines or less in length), then the use of the object
file is unrestricted, regardless of whether it is legally a derivative
work. (Executables containing this object code plus portions of the
Library will still fall under Section 6.)
Otherwise, if the work is a derivative of the Library, you may
distribute the object code for the work under the terms of Section 6.
Any executables containing that work also fall under Section 6,
whether or not they are linked directly with the Library itself.
6. As an exception to the Sections above, you may also combine or
link a "work that uses the Library" with the Library to produce a
work containing portions of the Library, and distribute that work
under terms of your choice, provided that the terms permit
modification of the work for the customer's own use and reverse
engineering for debugging such modifications.
You must give prominent notice with each copy of the work that the
Library is used in it and that the Library and its use are covered by
this License. You must supply a copy of this License. If the work
during execution displays copyright notices, you must include the
copyright notice for the Library among them, as well as a reference
directing the user to the copy of this License. Also, you must do one
of these things:
a) Accompany the work with the complete corresponding
machine-readable source code for the Library including whatever
changes were used in the work (which must be distributed under
Sections 1 and 2 above); and, if the work is an executable linked
with the Library, with the complete machine-readable "work that
uses the Library", as object code and/or source code, so that the
user can modify the Library and then relink to produce a modified
executable containing the modified Library. (It is understood
that the user who changes the contents of definitions files in the
Library will not necessarily be able to recompile the application
to use the modified definitions.)
b) Use a suitable shared library mechanism for linking with the
Library. A suitable mechanism is one that (1) uses at run time a
copy of the library already present on the user's computer system,
rather than copying library functions into the executable, and (2)
will operate properly with a modified version of the library, if
the user installs one, as long as the modified version is
interface-compatible with the version that the work was made with.
c) Accompany the work with a written offer, valid for at
least three years, to give the same user the materials
specified in Subsection 6a, above, for a charge no more
than the cost of performing this distribution.
d) If distribution of the work is made by offering access to copy
from a designated place, offer equivalent access to copy the above
specified materials from the same place.
e) Verify that the user has already received a copy of these
materials or that you have already sent this user a copy.
For an executable, the required form of the "work that uses the
Library" must include any data and utility programs needed for
reproducing the executable from it. However, as a special exception,
the materials to be distributed need not include anything that is
normally distributed (in either source or binary form) with the major
components (compiler, kernel, and so on) of the operating system on
which the executable runs, unless that component itself accompanies
the executable.
It may happen that this requirement contradicts the license
restrictions of other proprietary libraries that do not normally
accompany the operating system. Such a contradiction means you cannot
use both them and the Library together in an executable that you
distribute.
7. You may place library facilities that are a work based on the
Library side-by-side in a single library together with other library
facilities not covered by this License, and distribute such a combined
library, provided that the separate distribution of the work based on
the Library and of the other library facilities is otherwise
permitted, and provided that you do these two things:
a) Accompany the combined library with a copy of the same work
based on the Library, uncombined with any other library
facilities. This must be distributed under the terms of the
Sections above.
b) Give prominent notice with the combined library of the fact
that part of it is a work based on the Library, and explaining
where to find the accompanying uncombined form of the same work.
8. You may not copy, modify, sublicense, link with, or distribute
the Library except as expressly provided under this License. Any
attempt otherwise to copy, modify, sublicense, link with, or
distribute the Library is void, and will automatically terminate your
rights under this License. However, parties who have received copies,
or rights, from you under this License will not have their licenses
terminated so long as such parties remain in full compliance.
9. You are not required to accept this License, since you have not
signed it. However, nothing else grants you permission to modify or
distribute the Library or its derivative works. These actions are
prohibited by law if you do not accept this License. Therefore, by
modifying or distributing the Library (or any work based on the
Library), you indicate your acceptance of this License to do so, and
all its terms and conditions for copying, distributing or modifying
the Library or works based on it.
10. Each time you redistribute the Library (or any work based on the
Library), the recipient automatically receives a license from the
original licensor to copy, distribute, link with or modify the Library
subject to these terms and conditions. You may not impose any further
restrictions on the recipients' exercise of the rights granted herein.
You are not responsible for enforcing compliance by third parties with
this License.
11. If, as a consequence of a court judgment or allegation of patent
infringement or for any other reason (not limited to patent issues),
conditions are imposed on you (whether by court order, agreement or
otherwise) that contradict the conditions of this License, they do not
excuse you from the conditions of this License. If you cannot
distribute so as to satisfy simultaneously your obligations under this
License and any other pertinent obligations, then as a consequence you
may not distribute the Library at all. For example, if a patent
license would not permit royalty-free redistribution of the Library by
all those who receive copies directly or indirectly through you, then
the only way you could satisfy both it and this License would be to
refrain entirely from distribution of the Library.
If any portion of this section is held invalid or unenforceable under any
particular circumstance, the balance of the section is intended to apply,
and the section as a whole is intended to apply in other circumstances.
It is not the purpose of this section to induce you to infringe any
patents or other property right claims or to contest validity of any
such claims; this section has the sole purpose of protecting the
integrity of the free software distribution system which is
implemented by public license practices. Many people have made
generous contributions to the wide range of software distributed
through that system in reliance on consistent application of that
system; it is up to the author/donor to decide if he or she is willing
to distribute software through any other system and a licensee cannot
impose that choice.
This section is intended to make thoroughly clear what is believed to
be a consequence of the rest of this License.
12. If the distribution and/or use of the Library is restricted in
certain countries either by patents or by copyrighted interfaces, the
original copyright holder who places the Library under this License may add
an explicit geographical distribution limitation excluding those countries,
so that distribution is permitted only in or among countries not thus
excluded. In such case, this License incorporates the limitation as if
written in the body of this License.
13. The Free Software Foundation may publish revised and/or new
versions of the Lesser General Public License from time to time.
Such new versions will be similar in spirit to the present version,
but may differ in detail to address new problems or concerns.
Each version is given a distinguishing version number. If the Library
specifies a version number of this License which applies to it and
"any later version", you have the option of following the terms and
conditions either of that version or of any later version published by
the Free Software Foundation. If the Library does not specify a
license version number, you may choose any version ever published by
the Free Software Foundation.
14. If you wish to incorporate parts of the Library into other free
programs whose distribution conditions are incompatible with these,
write to the author to ask for permission. For software which is
copyrighted by the Free Software Foundation, write to the Free
Software Foundation; we sometimes make exceptions for this. Our
decision will be guided by the two goals of preserving the free status
of all derivatives of our free software and of promoting the sharing
and reuse of software generally.
NO WARRANTY
15. BECAUSE THE LIBRARY IS LICENSED FREE OF CHARGE, THERE IS NO
WARRANTY FOR THE LIBRARY, TO THE EXTENT PERMITTED BY APPLICABLE LAW.
EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR
OTHER PARTIES PROVIDE THE LIBRARY "AS IS" WITHOUT WARRANTY OF ANY
KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE
LIBRARY IS WITH YOU. SHOULD THE LIBRARY PROVE DEFECTIVE, YOU ASSUME
THE COST OF ALL NECESSARY SERVICING, REPAIR OR CORRECTION.
16. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN
WRITING WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY
AND/OR REDISTRIBUTE THE LIBRARY AS PERMITTED ABOVE, BE LIABLE TO YOU
FOR DAMAGES, INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR
CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OR INABILITY TO USE THE
LIBRARY (INCLUDING BUT NOT LIMITED TO LOSS OF DATA OR DATA BEING
RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD PARTIES OR A
FAILURE OF THE LIBRARY TO OPERATE WITH ANY OTHER SOFTWARE), EVEN IF
SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH
DAMAGES.
END OF TERMS AND CONDITIONS
File diff suppressed because it is too large. Load diff
@@ -0,0 +1,11 @@
@PACKAGE_INIT@
include("${CMAKE_CURRENT_LIST_DIR}/SoundTouchTargets.cmake")
check_required_components(SoundTouch)
if(@SOUNDTOUCH_DLL@)
check_required_components(SoundTouchDLL)
get_target_property(SoundTouchDLL_LOCATION SoundTouch::SoundTouchDLL LOCATION)
message(STATUS "Found SoundTouchDLL: ${SoundTouchDLL_LOCATION}")
endif()
@@ -0,0 +1,205 @@
////////////////////////////////////////////////////////////////////////////////
///
/// Beats-per-minute (BPM) detection routine.
///
/// The beat detection algorithm works as follows:
/// - Use function 'inputSamples' to input a chunks of samples to the class for
/// analysis. It's a good idea to enter a large sound file or stream in smallish
/// chunks of around few kilosamples in order not to extinguish too much RAM memory.
/// - Input sound data is decimated to approx 500 Hz to reduce calculation burden,
/// which is basically ok as low (bass) frequencies mostly determine the beat rate.
/// Simple averaging is used for anti-alias filtering because the resulting signal
/// quality isn't of that high importance.
/// - Decimated sound data is enveloped, i.e. the amplitude shape is detected by
/// taking absolute value that's smoothed by sliding average. Signal levels that
/// are below a couple of times the general RMS amplitude level are cut away to
/// leave only notable peaks there.
/// - Repeating sound patterns (e.g. beats) are detected by calculating short-term
/// autocorrelation function of the enveloped signal.
/// - After whole sound data file has been analyzed as above, the bpm level is
/// detected by function 'getBpm' that finds the highest peak of the autocorrelation
/// function, calculates it's precise location and converts this reading to bpm's.
///
/// Author : Copyright (c) Olli Parviainen
/// Author e-mail : oparviai 'at' iki.fi
/// SoundTouch WWW: http://www.surina.net/soundtouch
///
////////////////////////////////////////////////////////////////////////////////
//
// License :
//
// SoundTouch audio processing library
// Copyright (c) Olli Parviainen
//
// This library is free software; you can redistribute it and/or
// modify it under the terms of the GNU Lesser General Public
// License as published by the Free Software Foundation; either
// version 2.1 of the License, or (at your option) any later version.
//
// This library is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
// Lesser General Public License for more details.
//
// You should have received a copy of the GNU Lesser General Public
// License along with this library; if not, write to the Free Software
// Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
//
////////////////////////////////////////////////////////////////////////////////
#ifndef _BPMDetect_H_
#define _BPMDetect_H_
#include <vector>
#include "STTypes.h"
#include "FIFOSampleBuffer.h"
namespace soundtouch
{
/// Minimum allowed BPM rate. Used to restrict accepted result above a reasonable limit.
#define MIN_BPM 45
/// Maximum allowed BPM rate range. Used for calculating algorithm parametrs
#define MAX_BPM_RANGE 200
/// Maximum allowed BPM rate range. Used to restrict accepted result below a reasonable limit.
#define MAX_BPM_VALID 190
////////////////////////////////////////////////////////////////////////////////
typedef struct
{
float pos;
float strength;
} BEAT;
class IIR2_filter
{
double coeffs[5];
double prev[5];
public:
IIR2_filter(const double *lpf_coeffs);
float update(float x);
};
/// Class for calculating BPM rate for audio data.
class BPMDetect
{
protected:
/// Auto-correlation accumulator bins.
float *xcorr;
/// Sample average counter.
int decimateCount;
/// Sample average accumulator for FIFO-like decimation.
soundtouch::LONG_SAMPLETYPE decimateSum;
/// Decimate sound by this coefficient to reach approx. 500 Hz.
int decimateBy;
/// Auto-correlation window length
int windowLen;
/// Number of channels (1 = mono, 2 = stereo)
int channels;
/// sample rate
int sampleRate;
/// Beginning of auto-correlation window: Autocorrelation isn't being updated for
/// the first these many correlation bins.
int windowStart;
/// window functions for data preconditioning
float *hamw;
float *hamw2;
// beat detection variables
int pos;
int peakPos;
int beatcorr_ringbuffpos;
int init_scaler;
float peakVal;
float *beatcorr_ringbuff;
/// FIFO-buffer for decimated processing samples.
soundtouch::FIFOSampleBuffer *buffer;
/// Collection of detected beat positions
//BeatCollection beats;
std::vector<BEAT> beats;
// 2nd order low-pass-filter
IIR2_filter beat_lpf;
/// Updates auto-correlation function for given number of decimated samples that
/// are read from the internal 'buffer' pipe (samples aren't removed from the pipe
/// though).
void updateXCorr(int process_samples /// How many samples are processed.
);
/// Decimates samples to approx. 500 Hz.
///
/// \return Number of output samples.
int decimate(soundtouch::SAMPLETYPE *dest, ///< Destination buffer
const soundtouch::SAMPLETYPE *src, ///< Source sample buffer
int numsamples ///< Number of source samples.
);
/// Calculates amplitude envelope for the buffer of samples.
/// Result is output to 'samples'.
void calcEnvelope(soundtouch::SAMPLETYPE *samples, ///< Pointer to input/output data buffer
int numsamples ///< Number of samples in buffer
);
/// remove constant bias from xcorr data
void removeBias();
// Detect individual beat positions
void updateBeatPos(int process_samples);
public:
/// Constructor.
BPMDetect(int numChannels, ///< Number of channels in sample data.
int sampleRate ///< Sample rate in Hz.
);
/// Destructor.
virtual ~BPMDetect();
/// Inputs a block of samples for analyzing: Envelopes the samples and then
/// updates the autocorrelation estimation. When whole song data has been input
/// in smaller blocks using this function, read the resulting bpm with 'getBpm'
/// function.
///
/// Notice that data in 'samples' array can be disrupted in processing.
void inputSamples(const soundtouch::SAMPLETYPE *samples, ///< Pointer to input/working data buffer
int numSamples ///< Number of samples in buffer
);
/// Analyzes the results and returns the BPM rate. Use this function to read result
/// after whole song data has been input to the class by consecutive calls of
/// 'inputSamples' function.
///
/// \return Beats-per-minute rate, or zero if detection failed.
float getBpm();
/// Get beat position arrays. Note: The array includes also really low beat detection values
/// in absence of clear strong beats. Consumer may wish to filter low values away.
/// - "pos" receive array of beat positions
/// - "values" receive array of beat detection strengths
/// - max_num indicates max.size of "pos" and "values" array.
///
/// You can query a suitable array sized by calling this with nullptr in "pos" & "values".
///
/// \return number of beats in the arrays.
int getBeats(float *pos, float *strength, int max_num);
};
}
#endif // _BPMDetect_H_
@@ -0,0 +1,180 @@
////////////////////////////////////////////////////////////////////////////////
///
/// A buffer class for temporarily storaging sound samples, operates as a
/// first-in-first-out pipe.
///
/// Samples are added to the end of the sample buffer with the 'putSamples'
/// function, and are received from the beginning of the buffer by calling
/// the 'receiveSamples' function. The class automatically removes the
/// output samples from the buffer as well as grows the storage size
/// whenever necessary.
///
/// Author : Copyright (c) Olli Parviainen
/// Author e-mail : oparviai 'at' iki.fi
/// SoundTouch WWW: http://www.surina.net/soundtouch
///
////////////////////////////////////////////////////////////////////////////////
//
// License :
//
// SoundTouch audio processing library
// Copyright (c) Olli Parviainen
//
// This library is free software; you can redistribute it and/or
// modify it under the terms of the GNU Lesser General Public
// License as published by the Free Software Foundation; either
// version 2.1 of the License, or (at your option) any later version.
//
// This library is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
// Lesser General Public License for more details.
//
// You should have received a copy of the GNU Lesser General Public
// License along with this library; if not, write to the Free Software
// Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
//
////////////////////////////////////////////////////////////////////////////////
#ifndef FIFOSampleBuffer_H
#define FIFOSampleBuffer_H
#include "FIFOSamplePipe.h"
namespace soundtouch
{
/// Sample buffer working in FIFO (first-in-first-out) principle. The class takes
/// care of storage size adjustment and data moving during input/output operations.
///
/// Notice that in case of stereo audio, one sample is considered to consist of
/// both channel data.
class FIFOSampleBuffer : public FIFOSamplePipe
{
private:
/// Sample buffer.
SAMPLETYPE *buffer;
// Raw unaligned buffer memory. 'buffer' is made aligned by pointing it to first
// 16-byte aligned location of this buffer
SAMPLETYPE *bufferUnaligned;
/// Sample buffer size in bytes
uint sizeInBytes;
/// How many samples are currently in buffer.
uint samplesInBuffer;
/// Channels, 1=mono, 2=stereo.
uint channels;
/// Current position pointer to the buffer. This pointer is increased when samples are
/// removed from the pipe so that it's necessary to actually rewind buffer (move data)
/// only new data when is put to the pipe.
uint bufferPos;
/// Rewind the buffer by moving data from position pointed by 'bufferPos' to real
/// beginning of the buffer.
void rewind();
/// Ensures that the buffer has capacity for at least this many samples.
void ensureCapacity(uint capacityRequirement);
/// Returns current capacity.
uint getCapacity() const;
public:
/// Constructor
FIFOSampleBuffer(int numChannels = 2 ///< Number of channels, 1=mono, 2=stereo.
///< Default is stereo.
);
/// destructor
~FIFOSampleBuffer() override;
/// Returns a pointer to the beginning of the output samples.
/// This function is provided for accessing the output samples directly.
/// Please be careful for not to corrupt the book-keeping!
///
/// When using this function to output samples, also remember to 'remove' the
/// output samples from the buffer by calling the
/// 'receiveSamples(numSamples)' function
virtual SAMPLETYPE *ptrBegin() override;
/// Returns a pointer to the end of the used part of the sample buffer (i.e.
/// where the new samples are to be inserted). This function may be used for
/// inserting new samples into the sample buffer directly. Please be careful
/// not corrupt the book-keeping!
///
/// When using this function as means for inserting new samples, also remember
/// to increase the sample count afterwards, by calling the
/// 'putSamples(numSamples)' function.
SAMPLETYPE *ptrEnd(
uint slackCapacity ///< How much free capacity (in samples) there _at least_
///< should be so that the caller can successfully insert the
///< desired samples to the buffer. If necessary, the function
///< grows the buffer size to comply with this requirement.
);
/// Adds 'numSamples' pcs of samples from the 'samples' memory position to
/// the sample buffer.
virtual void putSamples(const SAMPLETYPE *samples, ///< Pointer to samples.
uint numSamples ///< Number of samples to insert.
) override;
/// Adjusts the book-keeping to increase number of samples in the buffer without
/// copying any actual samples.
///
/// This function is used to update the number of samples in the sample buffer
/// when accessing the buffer directly with 'ptrEnd' function. Please be
/// careful though!
virtual void putSamples(uint numSamples ///< Number of samples been inserted.
);
/// Output samples from beginning of the sample buffer. Copies requested samples to
/// output buffer and removes them from the sample buffer. If there are less than
/// 'numsample' samples in the buffer, returns all that available.
///
/// \return Number of samples returned.
virtual uint receiveSamples(SAMPLETYPE *output, ///< Buffer where to copy output samples.
uint maxSamples ///< How many samples to receive at max.
) override;
/// Adjusts book-keeping so that given number of samples are removed from beginning of the
/// sample buffer without copying them anywhere.
///
/// Used to reduce the number of samples in the buffer when accessing the sample buffer directly
/// with 'ptrBegin' function.
virtual uint receiveSamples(uint maxSamples ///< Remove this many samples from the beginning of pipe.
) override;
/// Returns number of samples currently available.
virtual uint numSamples() const override;
/// Sets number of channels, 1 = mono, 2 = stereo.
void setChannels(int numChannels);
/// Get number of channels
int getChannels()
{
return channels;
}
/// Returns nonzero if there aren't any samples available for outputting.
virtual int isEmpty() const override;
/// Clears all the samples.
virtual void clear() override;
/// allow trimming (downwards) amount of samples in pipeline.
/// Returns adjusted amount of samples
uint adjustAmountOfSamples(uint numSamples) override;
/// Add silence to end of buffer
void addSilent(uint nSamples);
};
}
#endif
@@ -0,0 +1,230 @@
////////////////////////////////////////////////////////////////////////////////
///
/// 'FIFOSamplePipe' : An abstract base class for classes that manipulate sound
/// samples by operating like a first-in-first-out pipe: New samples are fed
/// into one end of the pipe with the 'putSamples' function, and the processed
/// samples are received from the other end with the 'receiveSamples' function.
///
/// 'FIFOProcessor' : A base class for classes the do signal processing with
/// the samples while operating like a first-in-first-out pipe. When samples
/// are input with the 'putSamples' function, the class processes them
/// and moves the processed samples to the given 'output' pipe object, which
/// may be either another processing stage, or a fifo sample buffer object.
///
/// Author : Copyright (c) Olli Parviainen
/// Author e-mail : oparviai 'at' iki.fi
/// SoundTouch WWW: http://www.surina.net/soundtouch
///
////////////////////////////////////////////////////////////////////////////////
//
// License :
//
// SoundTouch audio processing library
// Copyright (c) Olli Parviainen
//
// This library is free software; you can redistribute it and/or
// modify it under the terms of the GNU Lesser General Public
// License as published by the Free Software Foundation; either
// version 2.1 of the License, or (at your option) any later version.
//
// This library is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
// Lesser General Public License for more details.
//
// You should have received a copy of the GNU Lesser General Public
// License along with this library; if not, write to the Free Software
// Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
//
////////////////////////////////////////////////////////////////////////////////
#ifndef FIFOSamplePipe_H
#define FIFOSamplePipe_H
#include <assert.h>
#include <stdlib.h>
#include "STTypes.h"
namespace soundtouch
{
/// Abstract base class for FIFO (first-in-first-out) sample processing classes.
class FIFOSamplePipe
{
protected:
bool verifyNumberOfChannels(int nChannels) const
{
if ((nChannels > 0) && (nChannels <= SOUNDTOUCH_MAX_CHANNELS))
{
return true;
}
ST_THROW_RT_ERROR("Error: Illegal number of channels");
return false;
}
public:
// virtual default destructor
virtual ~FIFOSamplePipe() {}
/// Returns a pointer to the beginning of the output samples.
/// This function is provided for accessing the output samples directly.
/// Please be careful for not to corrupt the book-keeping!
///
/// When using this function to output samples, also remember to 'remove' the
/// output samples from the buffer by calling the
/// 'receiveSamples(numSamples)' function
virtual SAMPLETYPE *ptrBegin() = 0;
/// Adds 'numSamples' pcs of samples from the 'samples' memory position to
/// the sample buffer.
virtual void putSamples(const SAMPLETYPE *samples, ///< Pointer to samples.
uint numSamples ///< Number of samples to insert.
) = 0;
// Moves samples from the 'other' pipe instance to this instance.
void moveSamples(FIFOSamplePipe &other ///< Other pipe instance where from the receive the data.
)
{
const uint oNumSamples = other.numSamples();
putSamples(other.ptrBegin(), oNumSamples);
other.receiveSamples(oNumSamples);
}
/// Output samples from beginning of the sample buffer. Copies requested samples to
/// output buffer and removes them from the sample buffer. If there are less than
/// 'numsample' samples in the buffer, returns all that available.
///
/// \return Number of samples returned.
virtual uint receiveSamples(SAMPLETYPE *output, ///< Buffer where to copy output samples.
uint maxSamples ///< How many samples to receive at max.
) = 0;
/// Adjusts book-keeping so that given number of samples are removed from beginning of the
/// sample buffer without copying them anywhere.
///
/// Used to reduce the number of samples in the buffer when accessing the sample buffer directly
/// with 'ptrBegin' function.
virtual uint receiveSamples(uint maxSamples ///< Remove this many samples from the beginning of pipe.
) = 0;
/// Returns number of samples currently available.
virtual uint numSamples() const = 0;
// Returns nonzero if there aren't any samples available for outputting.
virtual int isEmpty() const = 0;
/// Clears all the samples.
virtual void clear() = 0;
/// allow trimming (downwards) amount of samples in pipeline.
/// Returns adjusted amount of samples
virtual uint adjustAmountOfSamples(uint numSamples) = 0;
};
/// Base-class for sound processing routines working in FIFO principle. With this base
/// class it's easy to implement sound processing stages that can be chained together,
/// so that samples that are fed into beginning of the pipe automatically go through
/// all the processing stages.
///
/// When samples are input to this class, they're first processed and then put to
/// the FIFO pipe that's defined as output of this class. This output pipe can be
/// either other processing stage or a FIFO sample buffer.
class FIFOProcessor :public FIFOSamplePipe
{
protected:
/// Internal pipe where processed samples are put.
FIFOSamplePipe *output;
/// Sets output pipe.
void setOutPipe(FIFOSamplePipe *pOutput)
{
assert(output == nullptr);
assert(pOutput != nullptr);
output = pOutput;
}
/// Constructor. Doesn't define output pipe; it has to be set be
/// 'setOutPipe' function.
FIFOProcessor()
{
output = nullptr;
}
/// Constructor. Configures output pipe.
FIFOProcessor(FIFOSamplePipe *pOutput ///< Output pipe.
)
{
output = pOutput;
}
/// Destructor.
virtual ~FIFOProcessor() override
{
}
/// Returns a pointer to the beginning of the output samples.
/// This function is provided for accessing the output samples directly.
/// Please be careful for not to corrupt the book-keeping!
///
/// When using this function to output samples, also remember to 'remove' the
/// output samples from the buffer by calling the
/// 'receiveSamples(numSamples)' function
virtual SAMPLETYPE *ptrBegin() override
{
return output->ptrBegin();
}
public:
/// Output samples from beginning of the sample buffer. Copies requested samples to
/// output buffer and removes them from the sample buffer. If there are less than
/// 'numsample' samples in the buffer, returns all that available.
///
/// \return Number of samples returned.
virtual uint receiveSamples(SAMPLETYPE *outBuffer, ///< Buffer where to copy output samples.
uint maxSamples ///< How many samples to receive at max.
) override
{
return output->receiveSamples(outBuffer, maxSamples);
}
/// Adjusts book-keeping so that given number of samples are removed from beginning of the
/// sample buffer without copying them anywhere.
///
/// Used to reduce the number of samples in the buffer when accessing the sample buffer directly
/// with 'ptrBegin' function.
virtual uint receiveSamples(uint maxSamples ///< Remove this many samples from the beginning of pipe.
) override
{
return output->receiveSamples(maxSamples);
}
/// Returns number of samples currently available.
virtual uint numSamples() const override
{
return output->numSamples();
}
/// Returns nonzero if there aren't any samples available for outputting.
virtual int isEmpty() const override
{
return output->isEmpty();
}
/// allow trimming (downwards) amount of samples in pipeline.
/// Returns adjusted amount of samples
virtual uint adjustAmountOfSamples(uint numSamples) override
{
return output->adjustAmountOfSamples(numSamples);
}
};
}
#endif
@@ -0,0 +1,176 @@
////////////////////////////////////////////////////////////////////////////////
///
/// Common type definitions for SoundTouch audio processing library.
///
/// Author : Copyright (c) Olli Parviainen
/// Author e-mail : oparviai 'at' iki.fi
/// SoundTouch WWW: http://www.surina.net/soundtouch
///
////////////////////////////////////////////////////////////////////////////////
//
// License :
//
// SoundTouch audio processing library
// Copyright (c) Olli Parviainen
//
// This library is free software; you can redistribute it and/or
// modify it under the terms of the GNU Lesser General Public
// License as published by the Free Software Foundation; either
// version 2.1 of the License, or (at your option) any later version.
//
// This library is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
// Lesser General Public License for more details.
//
// You should have received a copy of the GNU Lesser General Public
// License along with this library; if not, write to the Free Software
// Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
//
////////////////////////////////////////////////////////////////////////////////
#ifndef STTypes_H
#define STTypes_H
typedef unsigned int uint;
typedef unsigned long ulong;
// Patch for MinGW: on Win64 long is 32-bit
#ifdef _WIN64
typedef unsigned long long ulongptr;
#else
typedef ulong ulongptr;
#endif
// Helper macro for aligning pointer up to next 16-byte boundary
#define SOUNDTOUCH_ALIGN_POINTER_16(x) ( ( (ulongptr)(x) + 15 ) & ~(ulongptr)15 )
namespace soundtouch
{
/// Max allowed number of channels
#define SOUNDTOUCH_MAX_CHANNELS 16
/// Activate these undef's to overrule the possible sampletype
/// setting inherited from some other header file:
//#undef SOUNDTOUCH_INTEGER_SAMPLES
//#undef SOUNDTOUCH_FLOAT_SAMPLES
/// If following flag is defined, always uses multichannel processing
/// routines also for mono and stero sound. This is for routine testing
/// purposes; output should be same with either routines, yet disabling
/// the dedicated mono/stereo processing routines will result in slower
/// runtime performance so recommendation is to keep this off.
// #define USE_MULTICH_ALWAYS
#if !(SOUNDTOUCH_INTEGER_SAMPLES || SOUNDTOUCH_FLOAT_SAMPLES)
/// Choose either 32bit floating point or 16bit integer sampletype
/// by choosing one of the following defines, unless this selection
/// has already been done in some other file.
////
/// Notes:
/// - In Windows environment, choose the sample format with the
/// following defines.
/// - In GNU environment, the floating point samples are used by
/// default, but integer samples can be chosen by giving the
/// following switch to the configure script:
/// ./configure --enable-integer-samples
/// However, if you still prefer to select the sample format here
/// also in GNU environment, then please #undef the INTEGER_SAMPLE
/// and FLOAT_SAMPLE defines first as in comments above.
//#define SOUNDTOUCH_INTEGER_SAMPLES 1 //< 16bit integer samples
#define SOUNDTOUCH_FLOAT_SAMPLES 1 //< 32bit float samples
#endif
#if (_M_IX86 || __i386__ || __x86_64__ || _M_X64)
/// Define this to allow X86-specific assembler/intrinsic optimizations.
/// Notice that library contains also usual C++ versions of each of these
/// these routines, so if you're having difficulties getting the optimized
/// routines compiled for whatever reason, you may disable these optimizations
/// to make the library compile.
#define SOUNDTOUCH_ALLOW_X86_OPTIMIZATIONS 1
#undef SOUNDTOUCH_ALLOW_NEON_OPTIMIZATIONS
#elif defined(__arm__) || defined(__aarch64__)
#undef SOUNDTOUCH_ALLOW_X86_OPTIMIZATIONS
#define SOUNDTOUCH_ALLOW_NEON_OPTIMIZATIONS 1
#else
/// Always disable optimizations when not using a x86 systems.
#undef SOUNDTOUCH_ALLOW_X86_OPTIMIZATIONS
#undef SOUNDTOUCH_ALLOW_NEON_OPTIMIZATIONS
#endif
// If defined, allows the SIMD-optimized routines to skip unevenly aligned
// memory offsets that can cause performance penalty in some SIMD implementations.
// Causes slight compromise in sound quality.
// #define SOUNDTOUCH_ALLOW_NONEXACT_SIMD_OPTIMIZATION 1
#ifdef SOUNDTOUCH_INTEGER_SAMPLES
// 16bit integer sample type
typedef short SAMPLETYPE;
// data type for sample accumulation: Use 32bit integer to prevent overflows
typedef long LONG_SAMPLETYPE;
#ifdef SOUNDTOUCH_FLOAT_SAMPLES
// check that only one sample type is defined
#error "conflicting sample types defined"
#endif // SOUNDTOUCH_FLOAT_SAMPLES
#ifdef SOUNDTOUCH_ALLOW_X86_OPTIMIZATIONS
// Allow MMX optimizations (not available in X64 mode)
#if (!_M_X64)
#define SOUNDTOUCH_ALLOW_MMX 1
#endif
#endif
#else
// floating point samples
typedef float SAMPLETYPE;
// data type for sample accumulation: Use float also here to enable
// efficient autovectorization
typedef float LONG_SAMPLETYPE;
#ifdef SOUNDTOUCH_ALLOW_X86_OPTIMIZATIONS
// Allow SSE optimizations
#define SOUNDTOUCH_ALLOW_SSE 1
#endif
#ifdef SOUNDTOUCH_ALLOW_NEON_OPTIMIZATIONS
#define SOUNDTOUCH_USE_NEON 1
#endif
#endif // SOUNDTOUCH_INTEGER_SAMPLES
#if ((SOUNDTOUCH_ALLOW_SSE) || (__SSE__) || (SOUNDTOUCH_USE_NEON))
#if SOUNDTOUCH_ALLOW_NONEXACT_SIMD_OPTIMIZATION
#define ST_SIMD_AVOID_UNALIGNED
#endif
#endif
}
// define ST_NO_EXCEPTION_HANDLING switch to disable throwing std exceptions:
#define ST_NO_EXCEPTION_HANDLING 1
#ifdef ST_NO_EXCEPTION_HANDLING
// Exceptions disabled. Throw asserts instead if enabled.
#include <assert.h>
#define ST_THROW_RT_ERROR(x) {assert((const char *)x);}
#else
// use c++ standard exceptions
#include <stdexcept>
#include <string>
#define ST_THROW_RT_ERROR(x) {throw std::runtime_error(x);}
#endif
// When this #define is active, eliminates a clicking sound when the "rate" or "pitch"
// parameter setting crosses from value <1 to >=1 or vice versa during processing.
// Default is off as such crossover is untypical case and involves a slight sound
// quality compromise.
//#define SOUNDTOUCH_PREVENT_CLICK_AT_RATE_CROSSOVER 1
#endif
@@ -0,0 +1,348 @@
//////////////////////////////////////////////////////////////////////////////
///
/// SoundTouch - main class for tempo/pitch/rate adjusting routines.
///
/// Notes:
/// - Initialize the SoundTouch object instance by setting up the sound stream
/// parameters with functions 'setSampleRate' and 'setChannels', then set
/// desired tempo/pitch/rate settings with the corresponding functions.
///
/// - The SoundTouch class behaves like a first-in-first-out pipeline: The
/// samples that are to be processed are fed into one of the pipe by calling
/// function 'putSamples', while the ready processed samples can be read
/// from the other end of the pipeline with function 'receiveSamples'.
///
/// - The SoundTouch processing classes require certain sized 'batches' of
/// samples in order to process the sound. For this reason the classes buffer
/// incoming samples until there are enough of samples available for
/// processing, then they carry out the processing step and consequently
/// make the processed samples available for outputting.
///
/// - For the above reason, the processing routines introduce a certain
/// 'latency' between the input and output, so that the samples input to
/// SoundTouch may not be immediately available in the output, and neither
/// the amount of outputtable samples may not immediately be in direct
/// relationship with the amount of previously input samples.
///
/// - The tempo/pitch/rate control parameters can be altered during processing.
/// Please notice though that they aren't currently protected by semaphores,
/// so in multi-thread application external semaphore protection may be
/// required.
///
/// - This class utilizes classes 'TDStretch' for tempo change (without modifying
/// pitch) and 'RateTransposer' for changing the playback rate (that is, both
/// tempo and pitch in the same ratio) of the sound. The third available control
/// 'pitch' (change pitch but maintain tempo) is produced by a combination of
/// combining the two other controls.
///
/// Author : Copyright (c) Olli Parviainen
/// Author e-mail : oparviai 'at' iki.fi
/// SoundTouch WWW: http://www.surina.net/soundtouch
///
////////////////////////////////////////////////////////////////////////////////
//
// License :
//
// SoundTouch audio processing library
// Copyright (c) Olli Parviainen
//
// This library is free software; you can redistribute it and/or
// modify it under the terms of the GNU Lesser General Public
// License as published by the Free Software Foundation; either
// version 2.1 of the License, or (at your option) any later version.
//
// This library is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
// Lesser General Public License for more details.
//
// You should have received a copy of the GNU Lesser General Public
// License along with this library; if not, write to the Free Software
// Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
//
////////////////////////////////////////////////////////////////////////////////
#ifndef SoundTouch_H
#define SoundTouch_H
#include "FIFOSamplePipe.h"
#include "STTypes.h"
namespace soundtouch
{
/// Soundtouch library version string
#define SOUNDTOUCH_VERSION "2.3.3"
/// SoundTouch library version id
#define SOUNDTOUCH_VERSION_ID (20303)
//
// Available setting IDs for the 'setSetting' & 'get_setting' functions:
/// Enable/disable anti-alias filter in pitch transposer (0 = disable)
#define SETTING_USE_AA_FILTER 0
/// Pitch transposer anti-alias filter length (8 .. 128 taps, default = 32)
#define SETTING_AA_FILTER_LENGTH 1
/// Enable/disable quick seeking algorithm in tempo changer routine
/// (enabling quick seeking lowers CPU utilization but causes a minor sound
/// quality compromising)
#define SETTING_USE_QUICKSEEK 2
/// Time-stretch algorithm single processing sequence length in milliseconds. This determines
/// to how long sequences the original sound is chopped in the time-stretch algorithm.
/// See "STTypes.h" or README for more information.
#define SETTING_SEQUENCE_MS 3
/// Time-stretch algorithm seeking window length in milliseconds for algorithm that finds the
/// best possible overlapping location. This determines from how wide window the algorithm
/// may look for an optimal joining location when mixing the sound sequences back together.
/// See "STTypes.h" or README for more information.
#define SETTING_SEEKWINDOW_MS 4
/// Time-stretch algorithm overlap length in milliseconds. When the chopped sound sequences
/// are mixed back together, to form a continuous sound stream, this parameter defines over
/// how long period the two consecutive sequences are let to overlap each other.
/// See "STTypes.h" or README for more information.
#define SETTING_OVERLAP_MS 5
/// Call "getSetting" with this ID to query processing sequence size in samples.
/// This value gives approximate value of how many input samples you'll need to
/// feed into SoundTouch after initial buffering to get out a new batch of
/// output samples.
///
/// This value does not include initial buffering at beginning of a new processing
/// stream, use SETTING_INITIAL_LATENCY to get the initial buffering size.
///
/// Notices:
/// - This is read-only parameter, i.e. setSetting ignores this parameter
/// - This parameter value is not constant but change depending on
/// tempo/pitch/rate/samplerate settings.
#define SETTING_NOMINAL_INPUT_SEQUENCE 6
/// Call "getSetting" with this ID to query nominal average processing output
/// size in samples. This value tells approcimate value how many output samples
/// SoundTouch outputs once it does DSP processing run for a batch of input samples.
///
/// Notices:
/// - This is read-only parameter, i.e. setSetting ignores this parameter
/// - This parameter value is not constant but change depending on
/// tempo/pitch/rate/samplerate settings.
#define SETTING_NOMINAL_OUTPUT_SEQUENCE 7
/// Call "getSetting" with this ID to query initial processing latency, i.e.
/// approx. how many samples you'll need to enter to SoundTouch pipeline before
/// you can expect to get first batch of ready output samples out.
///
/// After the first output batch, you can then expect to get approx.
/// SETTING_NOMINAL_OUTPUT_SEQUENCE ready samples out for every
/// SETTING_NOMINAL_INPUT_SEQUENCE samples that you enter into SoundTouch.
///
/// Example:
/// processing with parameter -tempo=5
/// => initial latency = 5509 samples
/// input sequence = 4167 samples
/// output sequence = 3969 samples
///
/// Accordingly, you can expect to feed in approx. 5509 samples at beginning of
/// the stream, and then you'll get out the first 3969 samples. After that, for
/// every approx. 4167 samples that you'll put in, you'll receive again approx.
/// 3969 samples out.
///
/// This also means that average latency during stream processing is
/// INITIAL_LATENCY-OUTPUT_SEQUENCE/2, in the above example case 5509-3969/2
/// = 3524 samples
///
/// Notices:
/// - This is read-only parameter, i.e. setSetting ignores this parameter
/// - This parameter value is not constant but change depending on
/// tempo/pitch/rate/samplerate settings.
#define SETTING_INITIAL_LATENCY 8
class SoundTouch final : public FIFOProcessor
{
private:
/// Rate transposer class instance
class RateTransposer *pRateTransposer;
/// Time-stretch class instance
class TDStretch *pTDStretch;
/// Virtual pitch parameter. Effective rate & tempo are calculated from these parameters.
double virtualRate;
/// Virtual pitch parameter. Effective rate & tempo are calculated from these parameters.
double virtualTempo;
/// Virtual pitch parameter. Effective rate & tempo are calculated from these parameters.
double virtualPitch;
/// Flag: Has sample rate been set?
bool bSrateSet;
/// Accumulator for how many samples in total will be expected as output vs. samples put in,
/// considering current processing settings.
double samplesExpectedOut;
/// Accumulator for how many samples in total have been read out from the processing so far
long samplesOutput;
/// Calculates effective rate & tempo valuescfrom 'virtualRate', 'virtualTempo' and
/// 'virtualPitch' parameters.
void calcEffectiveRateAndTempo();
protected :
/// Number of channels
uint channels;
/// Effective 'rate' value calculated from 'virtualRate', 'virtualTempo' and 'virtualPitch'
double rate;
/// Effective 'tempo' value calculated from 'virtualRate', 'virtualTempo' and 'virtualPitch'
double tempo;
public:
SoundTouch();
~SoundTouch() override;
/// Get SoundTouch library version string
static const char *getVersionString();
/// Get SoundTouch library version Id
static uint getVersionId();
/// Sets new rate control value. Normal rate = 1.0, smaller values
/// represent slower rate, larger faster rates.
void setRate(double newRate);
/// Sets new tempo control value. Normal tempo = 1.0, smaller values
/// represent slower tempo, larger faster tempo.
void setTempo(double newTempo);
/// Sets new rate control value as a difference in percents compared
/// to the original rate (-50 .. +100 %)
void setRateChange(double newRate);
/// Sets new tempo control value as a difference in percents compared
/// to the original tempo (-50 .. +100 %)
void setTempoChange(double newTempo);
/// Sets new pitch control value. Original pitch = 1.0, smaller values
/// represent lower pitches, larger values higher pitch.
void setPitch(double newPitch);
/// Sets pitch change in octaves compared to the original pitch
/// (-1.00 .. +1.00)
void setPitchOctaves(double newPitch);
/// Sets pitch change in semi-tones compared to the original pitch
/// (-12 .. +12)
void setPitchSemiTones(int newPitch);
void setPitchSemiTones(double newPitch);
/// Sets the number of channels, 1 = mono, 2 = stereo
void setChannels(uint numChannels);
/// Sets sample rate.
void setSampleRate(uint srate);
/// Get ratio between input and output audio durations, useful for calculating
/// processed output duration: if you'll process a stream of N samples, then
/// you can expect to get out N * getInputOutputSampleRatio() samples.
///
/// This ratio will give accurate target duration ratio for a full audio track,
/// given that the the whole track is processed with same processing parameters.
///
/// If this ratio is applied to calculate intermediate offsets inside a processing
/// stream, then this ratio is approximate and can deviate +- some tens of milliseconds
/// from ideal offset, yet by end of the audio stream the duration ratio will become
/// exact.
///
/// Example: if processing with parameters "-tempo=15 -pitch=-3", the function
/// will return value 0.8695652... Now, if processing an audio stream whose duration
/// is exactly one million audio samples, then you can expect the processed
/// output duration be 0.869565 * 1000000 = 869565 samples.
double getInputOutputSampleRatio();
/// Flushes the last samples from the processing pipeline to the output.
/// Clears also the internal processing buffers.
//
/// Note: This function is meant for extracting the last samples of a sound
/// stream. This function may introduce additional blank samples in the end
/// of the sound stream, and thus it's not recommended to call this function
/// in the middle of a sound stream.
void flush();
/// Adds 'numSamples' pcs of samples from the 'samples' memory position into
/// the input of the object. Notice that sample rate _has_to_ be set before
/// calling this function, otherwise throws a runtime_error exception.
void putSamples(
const SAMPLETYPE *samples, ///< Pointer to sample buffer.
uint numSamples ///< Number of samples in buffer. Notice
///< that in case of stereo-sound a single sample
///< contains data for both channels.
) override;
/// Output samples from beginning of the sample buffer. Copies requested samples to
/// output buffer and removes them from the sample buffer. If there are less than
/// 'numsample' samples in the buffer, returns all that available.
///
/// \return Number of samples returned.
uint receiveSamples(SAMPLETYPE *output, ///< Buffer where to copy output samples.
uint maxSamples ///< How many samples to receive at max.
) override;
/// Adjusts book-keeping so that given number of samples are removed from beginning of the
/// sample buffer without copying them anywhere.
///
/// Used to reduce the number of samples in the buffer when accessing the sample buffer directly
/// with 'ptrBegin' function.
uint receiveSamples(uint maxSamples ///< Remove this many samples from the beginning of pipe.
) override;
/// Clears all the samples in the object's output and internal processing
/// buffers.
void clear() override;
/// Changes a setting controlling the processing system behaviour. See the
/// 'SETTING_...' defines for available setting ID's.
///
/// \return 'true' if the setting was successfully changed
bool setSetting(int settingId, ///< Setting ID number. see SETTING_... defines.
int value ///< New setting value.
);
/// Reads a setting controlling the processing system behaviour. See the
/// 'SETTING_...' defines for available setting ID's.
///
/// \return the setting value.
int getSetting(int settingId ///< Setting ID number, see SETTING_... defines.
) const;
/// Returns number of samples currently unprocessed.
uint numUnprocessedSamples() const;
/// Return number of channels
uint numChannels() const
{
return channels;
}
/// Other handy functions that are implemented in the ancestor classes (see
/// classes 'FIFOProcessor' and 'FIFOSamplePipe')
///
/// - receiveSamples() : Use this function to receive 'ready' processed samples from SoundTouch.
/// - numSamples() : Get number of 'ready' samples that can be received with
/// function 'receiveSamples()'
/// - isEmpty() : Returns nonzero if there aren't any 'ready' samples.
/// - clear() : Clears all samples from ready/processing buffers.
};
}
#endif
@@ -0,0 +1,177 @@
//////////////////////////////////////////////////////////////////////////////
///
/// SoundTouch DLL wrapper - wraps SoundTouch routines into a Dynamic Load
/// Library interface.
///
/// Author : Copyright (c) Olli Parviainen
/// Author e-mail : oparviai 'at' iki.fi
/// SoundTouch WWW: http://www.surina.net/soundtouch
///
////////////////////////////////////////////////////////////////////////////////
//
// License :
//
// SoundTouch audio processing library
// Copyright (c) Olli Parviainen
//
// This library is free software; you can redistribute it and/or
// modify it under the terms of the GNU Lesser General Public
// License as published by the Free Software Foundation; either
// version 2.1 of the License, or (at your option) any later version.
//
// This library is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
// Lesser General Public License for more details.
//
// You should have received a copy of the GNU Lesser General Public
// License along with this library; if not, write to the Free Software
// Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
//
////////////////////////////////////////////////////////////////////////////////
#ifndef _SoundTouchDLL_h_
#define _SoundTouchDLL_h_
#if defined(_WIN32) || defined(WIN32)
// Windows
#ifndef __cplusplus
#error "Expected g++"
#endif
#ifdef DLL_EXPORTS
#define SOUNDTOUCHDLL_API extern "C" __declspec(dllexport)
#else
#define SOUNDTOUCHDLL_API extern "C" __declspec(dllimport)
#endif
#else
// GNU version
#if defined(DLL_EXPORTS) || defined(SoundTouchDLL_EXPORTS)
// GCC declaration for exporting functions
#define SOUNDTOUCHDLL_API extern "C" __attribute__((__visibility__("default")))
#else
// import function
#define SOUNDTOUCHDLL_API extern "C"
#endif
// Linux-replacements for Windows declarations:
#define __cdecl
typedef unsigned int DWORD;
#define FALSE 0
#define TRUE 1
#endif
typedef void * ST_HANDLE;
/// Create a new instance of SoundTouch processor.
SOUNDTOUCHDLL_API ST_HANDLE __cdecl soundtouch_createInstance();
/// Destroys a SoundTouch processor instance.
SOUNDTOUCHDLL_API void __cdecl soundtouch_destroyInstance(ST_HANDLE h);
/// Get SoundTouch library version string
SOUNDTOUCHDLL_API const char *__cdecl soundtouch_getVersionString();
/// Get SoundTouch library version Id
SOUNDTOUCHDLL_API unsigned int __cdecl soundtouch_getVersionId();
/// Sets new rate control value. Normal rate = 1.0, smaller values
/// represent slower rate, larger faster rates.
SOUNDTOUCHDLL_API void __cdecl soundtouch_setRate(ST_HANDLE h, float newRate);
/// Sets new tempo control value. Normal tempo = 1.0, smaller values
/// represent slower tempo, larger faster tempo.
SOUNDTOUCHDLL_API void __cdecl soundtouch_setTempo(ST_HANDLE h, float newTempo);
/// Sets new rate control value as a difference in percents compared
/// to the original rate (-50 .. +100 %);
SOUNDTOUCHDLL_API void __cdecl soundtouch_setRateChange(ST_HANDLE h, float newRate);
/// Sets new tempo control value as a difference in percents compared
/// to the original tempo (-50 .. +100 %);
SOUNDTOUCHDLL_API void __cdecl soundtouch_setTempoChange(ST_HANDLE h, float newTempo);
/// Sets new pitch control value. Original pitch = 1.0, smaller values
/// represent lower pitches, larger values higher pitch.
SOUNDTOUCHDLL_API void __cdecl soundtouch_setPitch(ST_HANDLE h, float newPitch);
/// Sets pitch change in octaves compared to the original pitch
/// (-1.00 .. +1.00);
SOUNDTOUCHDLL_API void __cdecl soundtouch_setPitchOctaves(ST_HANDLE h, float newPitch);
/// Sets pitch change in semi-tones compared to the original pitch
/// (-12 .. +12);
SOUNDTOUCHDLL_API void __cdecl soundtouch_setPitchSemiTones(ST_HANDLE h, float newPitch);
/// Sets the number of channels, 1 = mono, 2 = stereo, n = multichannel
SOUNDTOUCHDLL_API void __cdecl soundtouch_setChannels(ST_HANDLE h, unsigned int numChannels);
/// Sets sample rate.
SOUNDTOUCHDLL_API void __cdecl soundtouch_setSampleRate(ST_HANDLE h, unsigned int srate);
/// Flushes the last samples from the processing pipeline to the output.
/// Clears also the internal processing buffers.
//
/// Note: This function is meant for extracting the last samples of a sound
/// stream. This function may introduce additional blank samples in the end
/// of the sound stream, and thus it's not recommended to call this function
/// in the middle of a sound stream.
SOUNDTOUCHDLL_API void __cdecl soundtouch_flush(ST_HANDLE h);
/// Adds 'numSamples' pcs of samples from the 'samples' memory position into
/// the input of the object. Notice that sample rate _has_to_ be set before
/// calling this function, otherwise throws a runtime_error exception.
SOUNDTOUCHDLL_API void __cdecl soundtouch_putSamples(ST_HANDLE h,
const float *samples, ///< Pointer to sample buffer.
unsigned int numSamples ///< Number of sample frames in buffer. Notice
///< that in case of multi-channel sound a single
///< sample frame contains data for all channels.
);
/// Clears all the samples in the object's output and internal processing
/// buffers.
SOUNDTOUCHDLL_API void __cdecl soundtouch_clear(ST_HANDLE h);
/// Changes a setting controlling the processing system behaviour. See the
/// 'SETTING_...' defines for available setting ID's.
///
/// \return 'nonzero' if the setting was successfully changed, otherwise zero
SOUNDTOUCHDLL_API int __cdecl soundtouch_setSetting(ST_HANDLE h,
int settingId, ///< Setting ID number. see SETTING_... defines.
int value ///< New setting value.
);
/// Reads a setting controlling the processing system behaviour. See the
/// 'SETTING_...' defines for available setting ID's.
///
/// \return the setting value.
SOUNDTOUCHDLL_API int __cdecl soundtouch_getSetting(ST_HANDLE h,
int settingId ///< Setting ID number, see SETTING_... defines.
);
/// Returns number of samples currently unprocessed.
SOUNDTOUCHDLL_API unsigned int __cdecl soundtouch_numUnprocessedSamples(ST_HANDLE h);
/// Adjusts book-keeping so that given number of samples are removed from beginning of the
/// sample buffer without copying them anywhere.
///
/// Used to reduce the number of samples in the buffer when accessing the sample buffer directly
/// with 'ptrBegin' function.
SOUNDTOUCHDLL_API unsigned int __cdecl soundtouch_receiveSamples(ST_HANDLE h,
float *outBuffer, ///< Buffer where to copy output samples.
unsigned int maxSamples ///< How many samples to receive at max.
);
/// Returns number of samples currently available.
SOUNDTOUCHDLL_API unsigned int __cdecl soundtouch_numSamples(ST_HANDLE h);
/// Returns nonzero if there aren't any samples available for outputting.
SOUNDTOUCHDLL_API int __cdecl soundtouch_isEmpty(ST_HANDLE h);
#endif // _SoundTouchDLL_h_
+66
View File
@@ -0,0 +1,66 @@
# SoundTouch library
## About
SoundTouch is an open-source audio processing library that allows changing the sound tempo, pitch and playback rate parameters independently from each other:
* Change **tempo** while maintaining the original pitch
* Change **pitch** while maintaining the original tempo
* Change **playback rate** that affects both tempo and pitch at the
same time
* Change any combination of tempo/pitch/rate
Visit [SoundTouch website](https://www.surina.net/soundtouch) and see the [README file](https://www.surina.net/soundtouch/readme.html) for more information and audio examples.
### The latest stable release is 2.3.3
## Example
Use SoundStretch example app for modifying wav audio files, for example as follows:
```
soundstretch my_original_file.wav output_file.wav -tempo=+15 -pitch=-3
```
See the [README file](http://soundtouch.surina.net/README.html) for more usage examples and instructions how to build SoundTouch + SoundStretch.
Ready [SoundStretch application executables](https://www.surina.net/soundtouch/download.html) are available for download for Windows and Mac OS.
## Language & Platforms
SoundTouch is written in C++ and compiles in virtually any platform:
* Windows
* Mac OS
* Linux & Unices (including also Raspberry, Beaglebone, Yocto etc embedded Linux flavors)
* Android
* iOS
* embedded systems
The source code package includes dynamic library import modules for C#, Java and Pascal/Delphi languages.
## Tarballs
Source code release tarballs:
* https://www.surina.net/soundtouch/soundtouch-2.3.3.tar.gz
* https://www.surina.net/soundtouch/soundtouch-2.3.2.tar.gz
* https://www.surina.net/soundtouch/soundtouch-2.3.1.tar.gz
* https://www.surina.net/soundtouch/soundtouch-2.3.0.tar.gz
* https://www.surina.net/soundtouch/soundtouch-2.2.0.tar.gz
* https://www.surina.net/soundtouch/soundtouch-2.1.2.tar.gz
* https://www.surina.net/soundtouch/soundtouch-2.1.1.tar.gz
* https://www.surina.net/soundtouch/soundtouch-2.0.0.tar.gz
## License
SoundTouch is released under LGPL v2.1:
This library is free software; you can redistribute it and/or modify it under the terms of the GNU Lesser General Public License version 2.1 as published by the Free Software Foundation.
This library is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more details.
You should have received a copy of the GNU Lesser General Public License along with this library; if not, write to the Free Software Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
See [LGPL v2.1 full license text ](https://www.gnu.org/licenses/old-licenses/lgpl-2.1.html) for details.
--
Also commercial license free of GPL limitations available upon request
@@ -0,0 +1,222 @@
////////////////////////////////////////////////////////////////////////////////
///
/// FIR low-pass (anti-alias) filter with filter coefficient design routine and
/// MMX optimization.
///
/// Anti-alias filter is used to prevent folding of high frequencies when
/// transposing the sample rate with interpolation.
///
/// Author : Copyright (c) Olli Parviainen
/// Author e-mail : oparviai 'at' iki.fi
/// SoundTouch WWW: http://www.surina.net/soundtouch
///
////////////////////////////////////////////////////////////////////////////////
//
// License :
//
// SoundTouch audio processing library
// Copyright (c) Olli Parviainen
//
// This library is free software; you can redistribute it and/or
// modify it under the terms of the GNU Lesser General Public
// License as published by the Free Software Foundation; either
// version 2.1 of the License, or (at your option) any later version.
//
// This library is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
// Lesser General Public License for more details.
//
// You should have received a copy of the GNU Lesser General Public
// License along with this library; if not, write to the Free Software
// Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
//
////////////////////////////////////////////////////////////////////////////////
#include <memory.h>
#include <assert.h>
#include <math.h>
#include <stdlib.h>
#include "AAFilter.h"
#include "FIRFilter.h"
using namespace soundtouch;
#define PI 3.14159265358979323846
#define TWOPI (2 * PI)
// define this to save AA filter coefficients to a file
// #define _DEBUG_SAVE_AAFILTER_COEFFICIENTS 1
#ifdef _DEBUG_SAVE_AAFILTER_COEFFICIENTS
#include <stdio.h>
static void _DEBUG_SAVE_AAFIR_COEFFS(SAMPLETYPE *coeffs, int len)
{
FILE *fptr = fopen("aa_filter_coeffs.txt", "wt");
if (fptr == nullptr) return;
for (int i = 0; i < len; i ++)
{
double temp = coeffs[i];
fprintf(fptr, "%lf\n", temp);
}
fclose(fptr);
}
#else
#define _DEBUG_SAVE_AAFIR_COEFFS(x, y)
#endif
/*****************************************************************************
*
* Implementation of the class 'AAFilter'
*
*****************************************************************************/
AAFilter::AAFilter(uint len)
{
pFIR = FIRFilter::newInstance();
cutoffFreq = 0.5;
setLength(len);
}
AAFilter::~AAFilter()
{
delete pFIR;
}
// Sets new anti-alias filter cut-off edge frequency, scaled to
// sampling frequency (nyquist frequency = 0.5).
// The filter will cut frequencies higher than the given frequency.
void AAFilter::setCutoffFreq(double newCutoffFreq)
{
cutoffFreq = newCutoffFreq;
calculateCoeffs();
}
// Sets number of FIR filter taps
void AAFilter::setLength(uint newLength)
{
length = newLength;
calculateCoeffs();
}
// Calculates coefficients for a low-pass FIR filter using Hamming window
void AAFilter::calculateCoeffs()
{
uint i;
double cntTemp, temp, tempCoeff,h, w;
double wc;
double scaleCoeff, sum;
double *work;
SAMPLETYPE *coeffs;
assert(length >= 2);
assert(length % 4 == 0);
assert(cutoffFreq >= 0);
assert(cutoffFreq <= 0.5);
work = new double[length];
coeffs = new SAMPLETYPE[length];
wc = 2.0 * PI * cutoffFreq;
tempCoeff = TWOPI / (double)length;
sum = 0;
for (i = 0; i < length; i ++)
{
cntTemp = (double)i - (double)(length / 2);
temp = cntTemp * wc;
if (temp != 0)
{
h = sin(temp) / temp; // sinc function
}
else
{
h = 1.0;
}
w = 0.54 + 0.46 * cos(tempCoeff * cntTemp); // hamming window
temp = w * h;
work[i] = temp;
// calc net sum of coefficients
sum += temp;
}
// ensure the sum of coefficients is larger than zero
assert(sum > 0);
// ensure we've really designed a lowpass filter...
assert(work[length/2] > 0);
assert(work[length/2 + 1] > -1e-6);
assert(work[length/2 - 1] > -1e-6);
// Calculate a scaling coefficient in such a way that the result can be
// divided by 16384
scaleCoeff = 16384.0f / sum;
for (i = 0; i < length; i ++)
{
temp = work[i] * scaleCoeff;
// scale & round to nearest integer
temp += (temp >= 0) ? 0.5 : -0.5;
// ensure no overfloods
assert(temp >= -32768 && temp <= 32767);
coeffs[i] = (SAMPLETYPE)temp;
}
// Set coefficients. Use divide factor 14 => divide result by 2^14 = 16384
pFIR->setCoefficients(coeffs, length, 14);
_DEBUG_SAVE_AAFIR_COEFFS(coeffs, length);
delete[] work;
delete[] coeffs;
}
// Applies the filter to the given sequence of samples.
// Note : The amount of outputted samples is by value of 'filter length'
// smaller than the amount of input samples.
uint AAFilter::evaluate(SAMPLETYPE *dest, const SAMPLETYPE *src, uint numSamples, uint numChannels) const
{
return pFIR->evaluate(dest, src, numSamples, numChannels);
}
/// Applies the filter to the given src & dest pipes, so that processed amount of
/// samples get removed from src, and produced amount added to dest
/// Note : The amount of outputted samples is by value of 'filter length'
/// smaller than the amount of input samples.
uint AAFilter::evaluate(FIFOSampleBuffer &dest, FIFOSampleBuffer &src) const
{
SAMPLETYPE *pdest;
const SAMPLETYPE *psrc;
uint numSrcSamples;
uint result;
int numChannels = src.getChannels();
assert(numChannels == dest.getChannels());
numSrcSamples = src.numSamples();
psrc = src.ptrBegin();
pdest = dest.ptrEnd(numSrcSamples);
result = pFIR->evaluate(pdest, psrc, numSrcSamples, numChannels);
src.receiveSamples(result);
dest.putSamples(result);
return result;
}
uint AAFilter::getLength() const
{
return pFIR->getLength();
}
@@ -0,0 +1,93 @@
////////////////////////////////////////////////////////////////////////////////
///
/// Sampled sound tempo changer/time stretch algorithm. Changes the sound tempo
/// while maintaining the original pitch by using a time domain WSOLA-like method
/// with several performance-increasing tweaks.
///
/// Anti-alias filter is used to prevent folding of high frequencies when
/// transposing the sample rate with interpolation.
///
/// Author : Copyright (c) Olli Parviainen
/// Author e-mail : oparviai 'at' iki.fi
/// SoundTouch WWW: http://www.surina.net/soundtouch
///
////////////////////////////////////////////////////////////////////////////////
//
// License :
//
// SoundTouch audio processing library
// Copyright (c) Olli Parviainen
//
// This library is free software; you can redistribute it and/or
// modify it under the terms of the GNU Lesser General Public
// License as published by the Free Software Foundation; either
// version 2.1 of the License, or (at your option) any later version.
//
// This library is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
// Lesser General Public License for more details.
//
// You should have received a copy of the GNU Lesser General Public
// License along with this library; if not, write to the Free Software
// Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
//
////////////////////////////////////////////////////////////////////////////////
#ifndef AAFilter_H
#define AAFilter_H
#include "STTypes.h"
#include "FIFOSampleBuffer.h"
namespace soundtouch
{
class AAFilter
{
protected:
class FIRFilter *pFIR;
/// Low-pass filter cut-off frequency, negative = invalid
double cutoffFreq;
/// num of filter taps
uint length;
/// Calculate the FIR coefficients realizing the given cutoff-frequency
void calculateCoeffs();
public:
AAFilter(uint length);
~AAFilter();
/// Sets new anti-alias filter cut-off edge frequency, scaled to sampling
/// frequency (nyquist frequency = 0.5). The filter will cut off the
/// frequencies than that.
void setCutoffFreq(double newCutoffFreq);
/// Sets number of FIR filter taps, i.e. ~filter complexity
void setLength(uint newLength);
uint getLength() const;
/// Applies the filter to the given sequence of samples.
/// Note : The amount of outputted samples is by value of 'filter length'
/// smaller than the amount of input samples.
uint evaluate(SAMPLETYPE *dest,
const SAMPLETYPE *src,
uint numSamples,
uint numChannels) const;
/// Applies the filter to the given src & dest pipes, so that processed amount of
/// samples get removed from src, and produced amount added to dest
/// Note : The amount of outputted samples is by value of 'filter length'
/// smaller than the amount of input samples.
uint evaluate(FIFOSampleBuffer &dest,
FIFOSampleBuffer &src) const;
};
}
#endif
@@ -0,0 +1,569 @@
////////////////////////////////////////////////////////////////////////////////
///
/// Beats-per-minute (BPM) detection routine.
///
/// The beat detection algorithm works as follows:
/// - Use function 'inputSamples' to input a chunks of samples to the class for
/// analysis. It's a good idea to enter a large sound file or stream in smallish
/// chunks of around few kilosamples in order not to extinguish too much RAM memory.
/// - Inputted sound data is decimated to approx 500 Hz to reduce calculation burden,
/// which is basically ok as low (bass) frequencies mostly determine the beat rate.
/// Simple averaging is used for anti-alias filtering because the resulting signal
/// quality isn't of that high importance.
/// - Decimated sound data is enveloped, i.e. the amplitude shape is detected by
/// taking absolute value that's smoothed by sliding average. Signal levels that
/// are below a couple of times the general RMS amplitude level are cut away to
/// leave only notable peaks there.
/// - Repeating sound patterns (e.g. beats) are detected by calculating short-term
/// autocorrelation function of the enveloped signal.
/// - After whole sound data file has been analyzed as above, the bpm level is
/// detected by function 'getBpm' that finds the highest peak of the autocorrelation
/// function, calculates it's precise location and converts this reading to bpm's.
///
/// Author : Copyright (c) Olli Parviainen
/// Author e-mail : oparviai 'at' iki.fi
/// SoundTouch WWW: http://www.surina.net/soundtouch
///
////////////////////////////////////////////////////////////////////////////////
//
// License :
//
// SoundTouch audio processing library
// Copyright (c) Olli Parviainen
//
// This library is free software; you can redistribute it and/or
// modify it under the terms of the GNU Lesser General Public
// License as published by the Free Software Foundation; either
// version 2.1 of the License, or (at your option) any later version.
//
// This library is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
// Lesser General Public License for more details.
//
// You should have received a copy of the GNU Lesser General Public
// License along with this library; if not, write to the Free Software
// Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
//
////////////////////////////////////////////////////////////////////////////////
#define _USE_MATH_DEFINES
#include <math.h>
#include <assert.h>
#include <string.h>
#include <stdio.h>
#include <cfloat>
#include "FIFOSampleBuffer.h"
#include "PeakFinder.h"
#include "BPMDetect.h"
using namespace soundtouch;
// algorithm input sample block size
static const int INPUT_BLOCK_SIZE = 2048;
// decimated sample block size
static const int DECIMATED_BLOCK_SIZE = 256;
/// Target sample rate after decimation
static const int TARGET_SRATE = 1000;
/// XCorr update sequence size, update in about 200msec chunks
static const int XCORR_UPDATE_SEQUENCE = (int)(TARGET_SRATE / 5);
/// Moving average N size
static const int MOVING_AVERAGE_N = 15;
/// XCorr decay time constant, decay to half in 30 seconds
/// If it's desired to have the system adapt quicker to beat rate
/// changes within a continuing music stream, then the
/// 'xcorr_decay_time_constant' value can be reduced, yet that
/// can increase possibility of glitches in bpm detection.
static const double XCORR_DECAY_TIME_CONSTANT = 30.0;
/// Data overlap factor for beat detection algorithm
static const int OVERLAP_FACTOR = 4;
static const double TWOPI = (2 * M_PI);
////////////////////////////////////////////////////////////////////////////////
// Enable following define to create bpm analysis file:
//#define _CREATE_BPM_DEBUG_FILE
#ifdef _CREATE_BPM_DEBUG_FILE
static void _SaveDebugData(const char *name, const float *data, int minpos, int maxpos, double coeff)
{
FILE *fptr = fopen(name, "wt");
int i;
if (fptr)
{
printf("\nWriting BPM debug data into file %s\n", name);
for (i = minpos; i < maxpos; i ++)
{
fprintf(fptr, "%d\t%.1lf\t%f\n", i, coeff / (double)i, data[i]);
}
fclose(fptr);
}
}
void _SaveDebugBeatPos(const char *name, const std::vector<BEAT> &beats)
{
printf("\nWriting beat detections data into file %s\n", name);
FILE *fptr = fopen(name, "wt");
if (fptr)
{
for (uint i = 0; i < beats.size(); i++)
{
BEAT b = beats[i];
fprintf(fptr, "%lf\t%lf\n", b.pos, b.strength);
}
fclose(fptr);
}
}
#else
#define _SaveDebugData(name, a,b,c,d)
#define _SaveDebugBeatPos(name, b)
#endif
// Hamming window
void hamming(float *w, int N)
{
for (int i = 0; i < N; i++)
{
w[i] = (float)(0.54 - 0.46 * cos(TWOPI * i / (N - 1)));
}
}
////////////////////////////////////////////////////////////////////////////////
//
// IIR2_filter - 2nd order IIR filter
IIR2_filter::IIR2_filter(const double *lpf_coeffs)
{
memcpy(coeffs, lpf_coeffs, 5 * sizeof(double));
memset(prev, 0, sizeof(prev));
}
float IIR2_filter::update(float x)
{
prev[0] = x;
double y = x * coeffs[0];
for (int i = 4; i >= 1; i--)
{
y += coeffs[i] * prev[i];
prev[i] = prev[i - 1];
}
prev[3] = y;
return (float)y;
}
// IIR low-pass filter coefficients, calculated with matlab/octave cheby2(2,40,0.05)
const double _LPF_coeffs[5] = { 0.00996655391939, -0.01944529148401, 0.00996655391939, 1.96867605796247, -0.96916387431724 };
////////////////////////////////////////////////////////////////////////////////
BPMDetect::BPMDetect(int numChannels, int aSampleRate) :
beat_lpf(_LPF_coeffs)
{
beats.reserve(250); // initial reservation to prevent frequent reallocation
this->sampleRate = aSampleRate;
this->channels = numChannels;
decimateSum = 0;
decimateCount = 0;
// choose decimation factor so that result is approx. 1000 Hz
decimateBy = sampleRate / TARGET_SRATE;
if ((decimateBy <= 0) || (decimateBy * DECIMATED_BLOCK_SIZE < INPUT_BLOCK_SIZE))
{
ST_THROW_RT_ERROR("Too small samplerate");
}
// Calculate window length & starting item according to desired min & max bpms
windowLen = (60 * sampleRate) / (decimateBy * MIN_BPM);
windowStart = (60 * sampleRate) / (decimateBy * MAX_BPM_RANGE);
assert(windowLen > windowStart);
// allocate new working objects
xcorr = new float[windowLen];
memset(xcorr, 0, windowLen * sizeof(float));
pos = 0;
peakPos = 0;
peakVal = 0;
init_scaler = 1;
beatcorr_ringbuffpos = 0;
beatcorr_ringbuff = new float[windowLen];
memset(beatcorr_ringbuff, 0, windowLen * sizeof(float));
// allocate processing buffer
buffer = new FIFOSampleBuffer();
// we do processing in mono mode
buffer->setChannels(1);
buffer->clear();
// calculate hamming windows
hamw = new float[XCORR_UPDATE_SEQUENCE];
hamming(hamw, XCORR_UPDATE_SEQUENCE);
hamw2 = new float[XCORR_UPDATE_SEQUENCE / 2];
hamming(hamw2, XCORR_UPDATE_SEQUENCE / 2);
}
BPMDetect::~BPMDetect()
{
delete[] xcorr;
delete[] beatcorr_ringbuff;
delete[] hamw;
delete[] hamw2;
delete buffer;
}
/// convert to mono, low-pass filter & decimate to about 500 Hz.
/// return number of outputted samples.
///
/// Decimation is used to remove the unnecessary frequencies and thus to reduce
/// the amount of data needed to be processed as calculating autocorrelation
/// function is a very-very heavy operation.
///
/// Anti-alias filtering is done simply by averaging the samples. This is really a
/// poor-man's anti-alias filtering, but it's not so critical in this kind of application
/// (it'd also be difficult to design a high-quality filter with steep cut-off at very
/// narrow band)
int BPMDetect::decimate(SAMPLETYPE *dest, const SAMPLETYPE *src, int numsamples)
{
int count, outcount;
LONG_SAMPLETYPE out;
assert(channels > 0);
assert(decimateBy > 0);
outcount = 0;
for (count = 0; count < numsamples; count ++)
{
int j;
// convert to mono and accumulate
for (j = 0; j < channels; j ++)
{
decimateSum += src[j];
}
src += j;
decimateCount ++;
if (decimateCount >= decimateBy)
{
// Store every Nth sample only
out = (LONG_SAMPLETYPE)(decimateSum / (decimateBy * channels));
decimateSum = 0;
decimateCount = 0;
#ifdef SOUNDTOUCH_INTEGER_SAMPLES
// check ranges for sure (shouldn't actually be necessary)
if (out > 32767)
{
out = 32767;
}
else if (out < -32768)
{
out = -32768;
}
#endif // SOUNDTOUCH_INTEGER_SAMPLES
dest[outcount] = (SAMPLETYPE)out;
outcount ++;
}
}
return outcount;
}
// Calculates autocorrelation function of the sample history buffer
void BPMDetect::updateXCorr(int process_samples)
{
int offs;
SAMPLETYPE *pBuffer;
assert(buffer->numSamples() >= (uint)(process_samples + windowLen));
assert(process_samples == XCORR_UPDATE_SEQUENCE);
pBuffer = buffer->ptrBegin();
// calculate decay factor for xcorr filtering
float xcorr_decay = (float)pow(0.5, 1.0 / (XCORR_DECAY_TIME_CONSTANT * TARGET_SRATE / process_samples));
// prescale pbuffer
float tmp[XCORR_UPDATE_SEQUENCE];
for (int i = 0; i < process_samples; i++)
{
tmp[i] = hamw[i] * hamw[i] * pBuffer[i];
}
for (offs = windowStart; offs < windowLen; offs ++)
{
float sum;
int i;
sum = 0;
for (i = 0; i < process_samples; i ++)
{
sum += tmp[i] * pBuffer[i + offs]; // scaling the sub-result shouldn't be necessary
}
xcorr[offs] *= xcorr_decay; // decay 'xcorr' here with suitable time constant.
xcorr[offs] += (float)fabs(sum);
}
}
// Detect individual beat positions
void BPMDetect::updateBeatPos(int process_samples)
{
SAMPLETYPE *pBuffer;
assert(buffer->numSamples() >= (uint)(process_samples + windowLen));
pBuffer = buffer->ptrBegin();
assert(process_samples == XCORR_UPDATE_SEQUENCE / 2);
// static double thr = 0.0003;
double posScale = (double)this->decimateBy / (double)this->sampleRate;
int resetDur = (int)(0.12 / posScale + 0.5);
// prescale pbuffer
float tmp[XCORR_UPDATE_SEQUENCE / 2];
for (int i = 0; i < process_samples; i++)
{
tmp[i] = hamw2[i] * hamw2[i] * pBuffer[i];
}
for (int offs = windowStart; offs < windowLen; offs++)
{
float sum = 0;
for (int i = 0; i < process_samples; i++)
{
sum += tmp[i] * pBuffer[offs + i];
}
beatcorr_ringbuff[(beatcorr_ringbuffpos + offs) % windowLen] += (float)((sum > 0) ? sum : 0); // accumulate only positive correlations
}
int skipstep = XCORR_UPDATE_SEQUENCE / OVERLAP_FACTOR;
// compensate empty buffer at beginning by scaling coefficient
float scale = (float)windowLen / (float)(skipstep * init_scaler);
if (scale > 1.0f)
{
init_scaler++;
}
else
{
scale = 1.0f;
}
// detect beats
for (int i = 0; i < skipstep; i++)
{
float sum = beatcorr_ringbuff[beatcorr_ringbuffpos];
sum -= beat_lpf.update(sum);
if (sum > peakVal)
{
// found new local largest value
peakVal = sum;
peakPos = pos;
}
if (pos > peakPos + resetDur)
{
// largest value not updated for 200msec => accept as beat
peakPos += skipstep;
if (peakVal > 0)
{
// add detected beat to end of "beats" vector
BEAT temp = { (float)(peakPos * posScale), (float)(peakVal * scale) };
beats.push_back(temp);
}
peakVal = 0;
peakPos = pos;
}
beatcorr_ringbuff[beatcorr_ringbuffpos] = 0;
pos++;
beatcorr_ringbuffpos = (beatcorr_ringbuffpos + 1) % windowLen;
}
}
#define max(x,y) ((x) > (y) ? (x) : (y))
void BPMDetect::inputSamples(const SAMPLETYPE *samples, int numSamples)
{
SAMPLETYPE decimated[DECIMATED_BLOCK_SIZE];
// iterate so that max INPUT_BLOCK_SAMPLES processed per iteration
while (numSamples > 0)
{
int block;
int decSamples;
block = (numSamples > INPUT_BLOCK_SIZE) ? INPUT_BLOCK_SIZE : numSamples;
// decimate. note that converts to mono at the same time
decSamples = decimate(decimated, samples, block);
samples += block * channels;
numSamples -= block;
buffer->putSamples(decimated, decSamples);
}
// when the buffer has enough samples for processing...
int req = max(windowLen + XCORR_UPDATE_SEQUENCE, 2 * XCORR_UPDATE_SEQUENCE);
while ((int)buffer->numSamples() >= req)
{
// ... update autocorrelations...
updateXCorr(XCORR_UPDATE_SEQUENCE);
// ...update beat position calculation...
updateBeatPos(XCORR_UPDATE_SEQUENCE / 2);
// ... and remove proceessed samples from the buffer
int n = XCORR_UPDATE_SEQUENCE / OVERLAP_FACTOR;
buffer->receiveSamples(n);
}
}
void BPMDetect::removeBias()
{
int i;
// Remove linear bias: calculate linear regression coefficient
// 1. calc mean of 'xcorr' and 'i'
double mean_i = 0;
double mean_x = 0;
for (i = windowStart; i < windowLen; i++)
{
mean_x += xcorr[i];
}
mean_x /= (windowLen - windowStart);
mean_i = 0.5 * (windowLen - 1 + windowStart);
// 2. calculate linear regression coefficient
double b = 0;
double div = 0;
for (i = windowStart; i < windowLen; i++)
{
double xt = xcorr[i] - mean_x;
double xi = i - mean_i;
b += xt * xi;
div += xi * xi;
}
b /= div;
// subtract linear regression and resolve min. value bias
float minval = FLT_MAX; // arbitrary large number
for (i = windowStart; i < windowLen; i ++)
{
xcorr[i] -= (float)(b * i);
if (xcorr[i] < minval)
{
minval = xcorr[i];
}
}
// subtract min.value
for (i = windowStart; i < windowLen; i ++)
{
xcorr[i] -= minval;
}
}
// Calculate N-point moving average for "source" values
void MAFilter(float *dest, const float *source, int start, int end, int N)
{
for (int i = start; i < end; i++)
{
int i1 = i - N / 2;
int i2 = i + N / 2 + 1;
if (i1 < start) i1 = start;
if (i2 > end) i2 = end;
double sum = 0;
for (int j = i1; j < i2; j ++)
{
sum += source[j];
}
dest[i] = (float)(sum / (i2 - i1));
}
}
float BPMDetect::getBpm()
{
double peakPos;
double coeff;
PeakFinder peakFinder;
// remove bias from xcorr data
removeBias();
coeff = 60.0 * ((double)sampleRate / (double)decimateBy);
// save bpm debug data if debug data writing enabled
_SaveDebugData("soundtouch-bpm-xcorr.txt", xcorr, windowStart, windowLen, coeff);
// Smoothen by N-point moving-average
float *data = new float[windowLen];
memset(data, 0, sizeof(float) * windowLen);
MAFilter(data, xcorr, windowStart, windowLen, MOVING_AVERAGE_N);
// find peak position
peakPos = peakFinder.detectPeak(data, windowStart, windowLen);
// save bpm debug data if debug data writing enabled
_SaveDebugData("soundtouch-bpm-smoothed.txt", data, windowStart, windowLen, coeff);
delete[] data;
assert(decimateBy != 0);
if (peakPos < 1e-9) return 0.0; // detection failed.
_SaveDebugBeatPos("soundtouch-detected-beats.txt", beats);
// calculate BPM
float bpm = (float)(coeff / peakPos);
return (bpm >= MIN_BPM && bpm <= MAX_BPM_VALID) ? bpm : 0;
}
/// Get beat position arrays. Note: The array includes also really low beat detection values
/// in absence of clear strong beats. Consumer may wish to filter low values away.
/// - "pos" receive array of beat positions
/// - "values" receive array of beat detection strengths
/// - max_num indicates max.size of "pos" and "values" array.
///
/// You can query a suitable array sized by calling this with nullptr in "pos" & "values".
///
/// \return number of beats in the arrays.
int BPMDetect::getBeats(float *pos, float *values, int max_num)
{
int num = (int)beats.size();
if ((!pos) || (!values)) return num; // pos or values nullptr, return just size
for (int i = 0; (i < num) && (i < max_num); i++)
{
pos[i] = beats[i].pos;
values[i] = beats[i].strength;
}
return num;
}
@@ -0,0 +1,275 @@
////////////////////////////////////////////////////////////////////////////////
///
/// A buffer class for temporarily storaging sound samples, operates as a
/// first-in-first-out pipe.
///
/// Samples are added to the end of the sample buffer with the 'putSamples'
/// function, and are received from the beginning of the buffer by calling
/// the 'receiveSamples' function. The class automatically removes the
/// outputted samples from the buffer, as well as grows the buffer size
/// whenever necessary.
///
/// Author : Copyright (c) Olli Parviainen
/// Author e-mail : oparviai 'at' iki.fi
/// SoundTouch WWW: http://www.surina.net/soundtouch
///
////////////////////////////////////////////////////////////////////////////////
//
// License :
//
// SoundTouch audio processing library
// Copyright (c) Olli Parviainen
//
// This library is free software; you can redistribute it and/or
// modify it under the terms of the GNU Lesser General Public
// License as published by the Free Software Foundation; either
// version 2.1 of the License, or (at your option) any later version.
//
// This library is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
// Lesser General Public License for more details.
//
// You should have received a copy of the GNU Lesser General Public
// License along with this library; if not, write to the Free Software
// Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
//
////////////////////////////////////////////////////////////////////////////////
#include <stdlib.h>
#include <memory.h>
#include <string.h>
#include <assert.h>
#include "FIFOSampleBuffer.h"
using namespace soundtouch;
// Constructor
FIFOSampleBuffer::FIFOSampleBuffer(int numChannels)
{
assert(numChannels > 0);
sizeInBytes = 0; // reasonable initial value
buffer = nullptr;
bufferUnaligned = nullptr;
samplesInBuffer = 0;
bufferPos = 0;
channels = (uint)numChannels;
ensureCapacity(32); // allocate initial capacity
}
// destructor
FIFOSampleBuffer::~FIFOSampleBuffer()
{
delete[] bufferUnaligned;
bufferUnaligned = nullptr;
buffer = nullptr;
}
// Sets number of channels, 1 = mono, 2 = stereo
void FIFOSampleBuffer::setChannels(int numChannels)
{
uint usedBytes;
if (!verifyNumberOfChannels(numChannels)) return;
usedBytes = channels * samplesInBuffer;
channels = (uint)numChannels;
samplesInBuffer = usedBytes / channels;
}
// if output location pointer 'bufferPos' isn't zero, 'rewinds' the buffer and
// zeroes this pointer by copying samples from the 'bufferPos' pointer
// location on to the beginning of the buffer.
void FIFOSampleBuffer::rewind()
{
if (buffer && bufferPos)
{
memmove(buffer, ptrBegin(), sizeof(SAMPLETYPE) * channels * samplesInBuffer);
bufferPos = 0;
}
}
// Adds 'numSamples' pcs of samples from the 'samples' memory position to
// the sample buffer.
void FIFOSampleBuffer::putSamples(const SAMPLETYPE *samples, uint nSamples)
{
memcpy(ptrEnd(nSamples), samples, sizeof(SAMPLETYPE) * nSamples * channels);
samplesInBuffer += nSamples;
}
// Increases the number of samples in the buffer without copying any actual
// samples.
//
// This function is used to update the number of samples in the sample buffer
// when accessing the buffer directly with 'ptrEnd' function. Please be
// careful though!
void FIFOSampleBuffer::putSamples(uint nSamples)
{
uint req;
req = samplesInBuffer + nSamples;
ensureCapacity(req);
samplesInBuffer += nSamples;
}
// Returns a pointer to the end of the used part of the sample buffer (i.e.
// where the new samples are to be inserted). This function may be used for
// inserting new samples into the sample buffer directly. Please be careful!
//
// Parameter 'slackCapacity' tells the function how much free capacity (in
// terms of samples) there _at least_ should be, in order to the caller to
// successfully insert all the required samples to the buffer. When necessary,
// the function grows the buffer size to comply with this requirement.
//
// When using this function as means for inserting new samples, also remember
// to increase the sample count afterwards, by calling the
// 'putSamples(numSamples)' function.
SAMPLETYPE *FIFOSampleBuffer::ptrEnd(uint slackCapacity)
{
ensureCapacity(samplesInBuffer + slackCapacity);
return buffer + samplesInBuffer * channels;
}
// Returns a pointer to the beginning of the currently non-outputted samples.
// This function is provided for accessing the output samples directly.
// Please be careful!
//
// When using this function to output samples, also remember to 'remove' the
// outputted samples from the buffer by calling the
// 'receiveSamples(numSamples)' function
SAMPLETYPE *FIFOSampleBuffer::ptrBegin()
{
assert(buffer);
return buffer + bufferPos * channels;
}
// Ensures that the buffer has enough capacity, i.e. space for _at least_
// 'capacityRequirement' number of samples. The buffer is grown in steps of
// 4 kilobytes to eliminate the need for frequently growing up the buffer,
// as well as to round the buffer size up to the virtual memory page size.
void FIFOSampleBuffer::ensureCapacity(uint capacityRequirement)
{
SAMPLETYPE *tempUnaligned, *temp;
if (capacityRequirement > getCapacity())
{
// enlarge the buffer in 4kbyte steps (round up to next 4k boundary)
sizeInBytes = (capacityRequirement * channels * sizeof(SAMPLETYPE) + 4095) & (uint)-4096;
assert(sizeInBytes % 2 == 0);
tempUnaligned = new SAMPLETYPE[sizeInBytes / sizeof(SAMPLETYPE) + 16 / sizeof(SAMPLETYPE)];
if (tempUnaligned == nullptr)
{
ST_THROW_RT_ERROR("Couldn't allocate memory!\n");
}
// Align the buffer to begin at 16byte cache line boundary for optimal performance
temp = (SAMPLETYPE *)SOUNDTOUCH_ALIGN_POINTER_16(tempUnaligned);
if (samplesInBuffer)
{
memcpy(temp, ptrBegin(), samplesInBuffer * channels * sizeof(SAMPLETYPE));
}
delete[] bufferUnaligned;
buffer = temp;
bufferUnaligned = tempUnaligned;
bufferPos = 0;
}
else
{
// simply rewind the buffer (if necessary)
rewind();
}
}
// Returns the current buffer capacity in terms of samples
uint FIFOSampleBuffer::getCapacity() const
{
return sizeInBytes / (channels * sizeof(SAMPLETYPE));
}
// Returns the number of samples currently in the buffer
uint FIFOSampleBuffer::numSamples() const
{
return samplesInBuffer;
}
// Output samples from beginning of the sample buffer. Copies demanded number
// of samples to output and removes them from the sample buffer. If there
// are less than 'numsample' samples in the buffer, returns all available.
//
// Returns number of samples copied.
uint FIFOSampleBuffer::receiveSamples(SAMPLETYPE *output, uint maxSamples)
{
uint num;
num = (maxSamples > samplesInBuffer) ? samplesInBuffer : maxSamples;
memcpy(output, ptrBegin(), channels * sizeof(SAMPLETYPE) * num);
return receiveSamples(num);
}
// Removes samples from the beginning of the sample buffer without copying them
// anywhere. Used to reduce the number of samples in the buffer, when accessing
// the sample buffer with the 'ptrBegin' function.
uint FIFOSampleBuffer::receiveSamples(uint maxSamples)
{
if (maxSamples >= samplesInBuffer)
{
uint temp;
temp = samplesInBuffer;
samplesInBuffer = 0;
return temp;
}
samplesInBuffer -= maxSamples;
bufferPos += maxSamples;
return maxSamples;
}
// Returns nonzero if the sample buffer is empty
int FIFOSampleBuffer::isEmpty() const
{
return (samplesInBuffer == 0) ? 1 : 0;
}
// Clears the sample buffer
void FIFOSampleBuffer::clear()
{
samplesInBuffer = 0;
bufferPos = 0;
}
/// allow trimming (downwards) amount of samples in pipeline.
/// Returns adjusted amount of samples
uint FIFOSampleBuffer::adjustAmountOfSamples(uint numSamples)
{
if (numSamples < samplesInBuffer)
{
samplesInBuffer = numSamples;
}
return samplesInBuffer;
}
/// Add silence to end of buffer
void FIFOSampleBuffer::addSilent(uint nSamples)
{
memset(ptrEnd(nSamples), 0, sizeof(SAMPLETYPE) * nSamples * channels);
samplesInBuffer += nSamples;
}
@@ -0,0 +1,312 @@
////////////////////////////////////////////////////////////////////////////////
///
/// General FIR digital filter routines with MMX optimization.
///
/// Notes : MMX optimized functions reside in a separate, platform-specific file,
/// e.g. 'mmx_win.cpp' or 'mmx_gcc.cpp'
///
/// This source file contains OpenMP optimizations that allow speeding up the
/// corss-correlation algorithm by executing it in several threads / CPU cores
/// in parallel. See the following article link for more detailed discussion
/// about SoundTouch OpenMP optimizations:
/// http://www.softwarecoven.com/parallel-computing-in-embedded-mobile-devices
///
/// Author : Copyright (c) Olli Parviainen
/// Author e-mail : oparviai 'at' iki.fi
/// SoundTouch WWW: http://www.surina.net/soundtouch
///
////////////////////////////////////////////////////////////////////////////////
//
// License :
//
// SoundTouch audio processing library
// Copyright (c) Olli Parviainen
//
// This library is free software; you can redistribute it and/or
// modify it under the terms of the GNU Lesser General Public
// License as published by the Free Software Foundation; either
// version 2.1 of the License, or (at your option) any later version.
//
// This library is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
// Lesser General Public License for more details.
//
// You should have received a copy of the GNU Lesser General Public
// License along with this library; if not, write to the Free Software
// Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
//
////////////////////////////////////////////////////////////////////////////////
#include <memory.h>
#include <assert.h>
#include <math.h>
#include <stdlib.h>
#include "FIRFilter.h"
#include "cpu_detect.h"
using namespace soundtouch;
/*****************************************************************************
*
* Implementation of the class 'FIRFilter'
*
*****************************************************************************/
FIRFilter::FIRFilter()
{
resultDivFactor = 0;
resultDivider = 0;
length = 0;
lengthDiv8 = 0;
filterCoeffs = nullptr;
filterCoeffsStereo = nullptr;
}
FIRFilter::~FIRFilter()
{
delete[] filterCoeffs;
delete[] filterCoeffsStereo;
}
// Usual C-version of the filter routine for stereo sound
uint FIRFilter::evaluateFilterStereo(SAMPLETYPE *dest, const SAMPLETYPE *src, uint numSamples) const
{
int j, end;
// hint compiler autovectorization that loop length is divisible by 8
uint ilength = length & -8;
assert((length != 0) && (length == ilength) && (src != nullptr) && (dest != nullptr) && (filterCoeffs != nullptr));
assert(numSamples > ilength);
end = 2 * (numSamples - ilength);
for (j = 0; j < end; j += 2)
{
const SAMPLETYPE *ptr;
LONG_SAMPLETYPE suml, sumr;
suml = sumr = 0;
ptr = src + j;
for (uint i = 0; i < ilength; i ++)
{
suml += ptr[2 * i] * filterCoeffsStereo[2 * i];
sumr += ptr[2 * i + 1] * filterCoeffsStereo[2 * i + 1];
}
#ifdef SOUNDTOUCH_INTEGER_SAMPLES
suml >>= resultDivFactor;
sumr >>= resultDivFactor;
// saturate to 16 bit integer limits
suml = (suml < -32768) ? -32768 : (suml > 32767) ? 32767 : suml;
// saturate to 16 bit integer limits
sumr = (sumr < -32768) ? -32768 : (sumr > 32767) ? 32767 : sumr;
#endif // SOUNDTOUCH_INTEGER_SAMPLES
dest[j] = (SAMPLETYPE)suml;
dest[j + 1] = (SAMPLETYPE)sumr;
}
return numSamples - ilength;
}
// Usual C-version of the filter routine for mono sound
uint FIRFilter::evaluateFilterMono(SAMPLETYPE *dest, const SAMPLETYPE *src, uint numSamples) const
{
int j, end;
// hint compiler autovectorization that loop length is divisible by 8
int ilength = length & -8;
assert(ilength != 0);
end = numSamples - ilength;
for (j = 0; j < end; j ++)
{
const SAMPLETYPE *pSrc = src + j;
LONG_SAMPLETYPE sum;
int i;
sum = 0;
for (i = 0; i < ilength; i ++)
{
sum += pSrc[i] * filterCoeffs[i];
}
#ifdef SOUNDTOUCH_INTEGER_SAMPLES
sum >>= resultDivFactor;
// saturate to 16 bit integer limits
sum = (sum < -32768) ? -32768 : (sum > 32767) ? 32767 : sum;
#endif // SOUNDTOUCH_INTEGER_SAMPLES
dest[j] = (SAMPLETYPE)sum;
}
return end;
}
uint FIRFilter::evaluateFilterMulti(SAMPLETYPE *dest, const SAMPLETYPE *src, uint numSamples, uint numChannels)
{
int j, end;
assert(length != 0);
assert(src != nullptr);
assert(dest != nullptr);
assert(filterCoeffs != nullptr);
assert(numChannels < 16);
// hint compiler autovectorization that loop length is divisible by 8
int ilength = length & -8;
end = numChannels * (numSamples - ilength);
for (j = 0; j < end; j += numChannels)
{
const SAMPLETYPE *ptr;
LONG_SAMPLETYPE sums[16];
uint c;
int i;
for (c = 0; c < numChannels; c ++)
{
sums[c] = 0;
}
ptr = src + j;
for (i = 0; i < ilength; i ++)
{
SAMPLETYPE coef=filterCoeffs[i];
for (c = 0; c < numChannels; c ++)
{
sums[c] += ptr[0] * coef;
ptr ++;
}
}
for (c = 0; c < numChannels; c ++)
{
#ifdef SOUNDTOUCH_INTEGER_SAMPLES
sums[c] >>= resultDivFactor;
#endif // SOUNDTOUCH_INTEGER_SAMPLES
dest[j+c] = (SAMPLETYPE)sums[c];
}
}
return numSamples - ilength;
}
// Set filter coeffiecients and length.
//
// Throws an exception if filter length isn't divisible by 8
void FIRFilter::setCoefficients(const SAMPLETYPE *coeffs, uint newLength, uint uResultDivFactor)
{
assert(newLength > 0);
if (newLength % 8) ST_THROW_RT_ERROR("FIR filter length not divisible by 8");
#ifdef SOUNDTOUCH_FLOAT_SAMPLES
// scale coefficients already here if using floating samples
double scale = 1.0 / resultDivider;
#else
short scale = 1;
#endif
lengthDiv8 = newLength / 8;
length = lengthDiv8 * 8;
assert(length == newLength);
resultDivFactor = uResultDivFactor;
resultDivider = (SAMPLETYPE)::pow(2.0, (int)resultDivFactor);
delete[] filterCoeffs;
filterCoeffs = new SAMPLETYPE[length];
delete[] filterCoeffsStereo;
filterCoeffsStereo = new SAMPLETYPE[length*2];
for (uint i = 0; i < length; i ++)
{
filterCoeffs[i] = (SAMPLETYPE)(coeffs[i] * scale);
// create also stereo set of filter coefficients: this allows compiler
// to autovectorize filter evaluation much more efficiently
filterCoeffsStereo[2 * i] = (SAMPLETYPE)(coeffs[i] * scale);
filterCoeffsStereo[2 * i + 1] = (SAMPLETYPE)(coeffs[i] * scale);
}
}
uint FIRFilter::getLength() const
{
return length;
}
// Applies the filter to the given sequence of samples.
//
// Note : The amount of outputted samples is by value of 'filter_length'
// smaller than the amount of input samples.
uint FIRFilter::evaluate(SAMPLETYPE *dest, const SAMPLETYPE *src, uint numSamples, uint numChannels)
{
assert(length > 0);
assert(lengthDiv8 * 8 == length);
if (numSamples < length) return 0;
#ifndef USE_MULTICH_ALWAYS
if (numChannels == 1)
{
return evaluateFilterMono(dest, src, numSamples);
}
else if (numChannels == 2)
{
return evaluateFilterStereo(dest, src, numSamples);
}
else
#endif // USE_MULTICH_ALWAYS
{
assert(numChannels > 0);
return evaluateFilterMulti(dest, src, numSamples, numChannels);
}
}
// Operator 'new' is overloaded so that it automatically creates a suitable instance
// depending on if we've a MMX-capable CPU available or not.
void * FIRFilter::operator new(size_t)
{
// Notice! don't use "new FIRFilter" directly, use "newInstance" to create a new instance instead!
ST_THROW_RT_ERROR("Error in FIRFilter::new: Don't use 'new FIRFilter', use 'newInstance' member instead!");
return newInstance();
}
FIRFilter * FIRFilter::newInstance()
{
uint uExtensions;
uExtensions = detectCPUextensions();
(void)uExtensions;
// Check if MMX/SSE instruction set extensions supported by CPU
#ifdef SOUNDTOUCH_ALLOW_MMX
// MMX routines available only with integer sample types
if (uExtensions & SUPPORT_MMX)
{
return ::new FIRFilterMMX;
}
else
#endif // SOUNDTOUCH_ALLOW_MMX
#ifdef SOUNDTOUCH_ALLOW_SSE
if (uExtensions & SUPPORT_SSE)
{
// SSE support
return ::new FIRFilterSSE;
}
else
#endif // SOUNDTOUCH_ALLOW_SSE
{
// ISA optimizations not supported, use plain C version
return ::new FIRFilter;
}
}
@@ -0,0 +1,140 @@
////////////////////////////////////////////////////////////////////////////////
///
/// General FIR digital filter routines with MMX optimization.
///
/// Note : MMX optimized functions reside in a separate, platform-specific file,
/// e.g. 'mmx_win.cpp' or 'mmx_gcc.cpp'
///
/// Author : Copyright (c) Olli Parviainen
/// Author e-mail : oparviai 'at' iki.fi
/// SoundTouch WWW: http://www.surina.net/soundtouch
///
////////////////////////////////////////////////////////////////////////////////
//
// License :
//
// SoundTouch audio processing library
// Copyright (c) Olli Parviainen
//
// This library is free software; you can redistribute it and/or
// modify it under the terms of the GNU Lesser General Public
// License as published by the Free Software Foundation; either
// version 2.1 of the License, or (at your option) any later version.
//
// This library is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
// Lesser General Public License for more details.
//
// You should have received a copy of the GNU Lesser General Public
// License along with this library; if not, write to the Free Software
// Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
//
////////////////////////////////////////////////////////////////////////////////
#ifndef FIRFilter_H
#define FIRFilter_H
#include <stddef.h>
#include "STTypes.h"
namespace soundtouch
{
class FIRFilter
{
protected:
// Number of FIR filter taps
uint length;
// Number of FIR filter taps divided by 8
uint lengthDiv8;
// Result divider factor in 2^k format
uint resultDivFactor;
// Result divider value.
SAMPLETYPE resultDivider;
// Memory for filter coefficients
SAMPLETYPE *filterCoeffs;
SAMPLETYPE *filterCoeffsStereo;
virtual uint evaluateFilterStereo(SAMPLETYPE *dest,
const SAMPLETYPE *src,
uint numSamples) const;
virtual uint evaluateFilterMono(SAMPLETYPE *dest,
const SAMPLETYPE *src,
uint numSamples) const;
virtual uint evaluateFilterMulti(SAMPLETYPE *dest, const SAMPLETYPE *src, uint numSamples, uint numChannels);
public:
FIRFilter();
virtual ~FIRFilter();
/// Operator 'new' is overloaded so that it automatically creates a suitable instance
/// depending on if we've a MMX-capable CPU available or not.
static void * operator new(size_t s);
static FIRFilter *newInstance();
/// Applies the filter to the given sequence of samples.
/// Note : The amount of outputted samples is by value of 'filter_length'
/// smaller than the amount of input samples.
///
/// \return Number of samples copied to 'dest'.
uint evaluate(SAMPLETYPE *dest,
const SAMPLETYPE *src,
uint numSamples,
uint numChannels);
uint getLength() const;
virtual void setCoefficients(const SAMPLETYPE *coeffs,
uint newLength,
uint uResultDivFactor);
};
// Optional subclasses that implement CPU-specific optimizations:
#ifdef SOUNDTOUCH_ALLOW_MMX
/// Class that implements MMX optimized functions exclusive for 16bit integer samples type.
class FIRFilterMMX : public FIRFilter
{
protected:
short *filterCoeffsUnalign;
short *filterCoeffsAlign;
virtual uint evaluateFilterStereo(short *dest, const short *src, uint numSamples) const override;
public:
FIRFilterMMX();
~FIRFilterMMX();
virtual void setCoefficients(const short *coeffs, uint newLength, uint uResultDivFactor) override;
};
#endif // SOUNDTOUCH_ALLOW_MMX
#ifdef SOUNDTOUCH_ALLOW_SSE
/// Class that implements SSE optimized functions exclusive for floating point samples type.
class FIRFilterSSE : public FIRFilter
{
protected:
float *filterCoeffsUnalign;
float *filterCoeffsAlign;
virtual uint evaluateFilterStereo(float *dest, const float *src, uint numSamples) const override;
public:
FIRFilterSSE();
~FIRFilterSSE();
virtual void setCoefficients(const float *coeffs, uint newLength, uint uResultDivFactor) override;
};
#endif // SOUNDTOUCH_ALLOW_SSE
}
#endif // FIRFilter_H
@@ -0,0 +1,196 @@
////////////////////////////////////////////////////////////////////////////////
///
/// Cubic interpolation routine.
///
/// Author : Copyright (c) Olli Parviainen
/// Author e-mail : oparviai 'at' iki.fi
/// SoundTouch WWW: http://www.surina.net/soundtouch
///
////////////////////////////////////////////////////////////////////////////////
//
// License :
//
// SoundTouch audio processing library
// Copyright (c) Olli Parviainen
//
// This library is free software; you can redistribute it and/or
// modify it under the terms of the GNU Lesser General Public
// License as published by the Free Software Foundation; either
// version 2.1 of the License, or (at your option) any later version.
//
// This library is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
// Lesser General Public License for more details.
//
// You should have received a copy of the GNU Lesser General Public
// License along with this library; if not, write to the Free Software
// Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
//
////////////////////////////////////////////////////////////////////////////////
#include <stddef.h>
#include <math.h>
#include "InterpolateCubic.h"
#include "STTypes.h"
using namespace soundtouch;
// cubic interpolation coefficients
static const float _coeffs[]=
{ -0.5f, 1.0f, -0.5f, 0.0f,
1.5f, -2.5f, 0.0f, 1.0f,
-1.5f, 2.0f, 0.5f, 0.0f,
0.5f, -0.5f, 0.0f, 0.0f};
InterpolateCubic::InterpolateCubic()
{
fract = 0;
}
void InterpolateCubic::resetRegisters()
{
fract = 0;
}
/// Transpose mono audio. Returns number of produced output samples, and
/// updates "srcSamples" to amount of consumed source samples
int InterpolateCubic::transposeMono(SAMPLETYPE *pdest,
const SAMPLETYPE *psrc,
int &srcSamples)
{
int i;
int srcSampleEnd = srcSamples - 4;
int srcCount = 0;
i = 0;
while (srcCount < srcSampleEnd)
{
float out;
const float x3 = 1.0f;
const float x2 = (float)fract; // x
const float x1 = x2*x2; // x^2
const float x0 = x1*x2; // x^3
float y0, y1, y2, y3;
assert(fract < 1.0);
y0 = _coeffs[0] * x0 + _coeffs[1] * x1 + _coeffs[2] * x2 + _coeffs[3] * x3;
y1 = _coeffs[4] * x0 + _coeffs[5] * x1 + _coeffs[6] * x2 + _coeffs[7] * x3;
y2 = _coeffs[8] * x0 + _coeffs[9] * x1 + _coeffs[10] * x2 + _coeffs[11] * x3;
y3 = _coeffs[12] * x0 + _coeffs[13] * x1 + _coeffs[14] * x2 + _coeffs[15] * x3;
out = y0 * psrc[0] + y1 * psrc[1] + y2 * psrc[2] + y3 * psrc[3];
pdest[i] = (SAMPLETYPE)out;
i ++;
// update position fraction
fract += rate;
// update whole positions
int whole = (int)fract;
fract -= whole;
psrc += whole;
srcCount += whole;
}
srcSamples = srcCount;
return i;
}
/// Transpose stereo audio. Returns number of produced output samples, and
/// updates "srcSamples" to amount of consumed source samples
int InterpolateCubic::transposeStereo(SAMPLETYPE *pdest,
const SAMPLETYPE *psrc,
int &srcSamples)
{
int i;
int srcSampleEnd = srcSamples - 4;
int srcCount = 0;
i = 0;
while (srcCount < srcSampleEnd)
{
const float x3 = 1.0f;
const float x2 = (float)fract; // x
const float x1 = x2*x2; // x^2
const float x0 = x1*x2; // x^3
float y0, y1, y2, y3;
float out0, out1;
assert(fract < 1.0);
y0 = _coeffs[0] * x0 + _coeffs[1] * x1 + _coeffs[2] * x2 + _coeffs[3] * x3;
y1 = _coeffs[4] * x0 + _coeffs[5] * x1 + _coeffs[6] * x2 + _coeffs[7] * x3;
y2 = _coeffs[8] * x0 + _coeffs[9] * x1 + _coeffs[10] * x2 + _coeffs[11] * x3;
y3 = _coeffs[12] * x0 + _coeffs[13] * x1 + _coeffs[14] * x2 + _coeffs[15] * x3;
out0 = y0 * psrc[0] + y1 * psrc[2] + y2 * psrc[4] + y3 * psrc[6];
out1 = y0 * psrc[1] + y1 * psrc[3] + y2 * psrc[5] + y3 * psrc[7];
pdest[2*i] = (SAMPLETYPE)out0;
pdest[2*i+1] = (SAMPLETYPE)out1;
i ++;
// update position fraction
fract += rate;
// update whole positions
int whole = (int)fract;
fract -= whole;
psrc += 2*whole;
srcCount += whole;
}
srcSamples = srcCount;
return i;
}
/// Transpose multi-channel audio. Returns number of produced output samples, and
/// updates "srcSamples" to amount of consumed source samples
int InterpolateCubic::transposeMulti(SAMPLETYPE *pdest,
const SAMPLETYPE *psrc,
int &srcSamples)
{
int i;
int srcSampleEnd = srcSamples - 4;
int srcCount = 0;
i = 0;
while (srcCount < srcSampleEnd)
{
const float x3 = 1.0f;
const float x2 = (float)fract; // x
const float x1 = x2*x2; // x^2
const float x0 = x1*x2; // x^3
float y0, y1, y2, y3;
assert(fract < 1.0);
y0 = _coeffs[0] * x0 + _coeffs[1] * x1 + _coeffs[2] * x2 + _coeffs[3] * x3;
y1 = _coeffs[4] * x0 + _coeffs[5] * x1 + _coeffs[6] * x2 + _coeffs[7] * x3;
y2 = _coeffs[8] * x0 + _coeffs[9] * x1 + _coeffs[10] * x2 + _coeffs[11] * x3;
y3 = _coeffs[12] * x0 + _coeffs[13] * x1 + _coeffs[14] * x2 + _coeffs[15] * x3;
for (int c = 0; c < numChannels; c ++)
{
float out;
out = y0 * psrc[c] + y1 * psrc[c + numChannels] + y2 * psrc[c + 2 * numChannels] + y3 * psrc[c + 3 * numChannels];
pdest[0] = (SAMPLETYPE)out;
pdest ++;
}
i ++;
// update position fraction
fract += rate;
// update whole positions
int whole = (int)fract;
fract -= whole;
psrc += numChannels*whole;
srcCount += whole;
}
srcSamples = srcCount;
return i;
}
@@ -0,0 +1,69 @@
////////////////////////////////////////////////////////////////////////////////
///
/// Cubic interpolation routine.
///
/// Author : Copyright (c) Olli Parviainen
/// Author e-mail : oparviai 'at' iki.fi
/// SoundTouch WWW: http://www.surina.net/soundtouch
///
////////////////////////////////////////////////////////////////////////////////
//
// License :
//
// SoundTouch audio processing library
// Copyright (c) Olli Parviainen
//
// This library is free software; you can redistribute it and/or
// modify it under the terms of the GNU Lesser General Public
// License as published by the Free Software Foundation; either
// version 2.1 of the License, or (at your option) any later version.
//
// This library is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
// Lesser General Public License for more details.
//
// You should have received a copy of the GNU Lesser General Public
// License along with this library; if not, write to the Free Software
// Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
//
////////////////////////////////////////////////////////////////////////////////
#ifndef _InterpolateCubic_H_
#define _InterpolateCubic_H_
#include "RateTransposer.h"
#include "STTypes.h"
namespace soundtouch
{
class InterpolateCubic : public TransposerBase
{
protected:
virtual int transposeMono(SAMPLETYPE *dest,
const SAMPLETYPE *src,
int &srcSamples) override;
virtual int transposeStereo(SAMPLETYPE *dest,
const SAMPLETYPE *src,
int &srcSamples) override;
virtual int transposeMulti(SAMPLETYPE *dest,
const SAMPLETYPE *src,
int &srcSamples) override;
double fract;
public:
InterpolateCubic();
virtual void resetRegisters() override;
virtual int getLatency() const override
{
return 1;
}
};
}
#endif
@@ -0,0 +1,296 @@
////////////////////////////////////////////////////////////////////////////////
///
/// Linear interpolation algorithm.
///
/// Author : Copyright (c) Olli Parviainen
/// Author e-mail : oparviai 'at' iki.fi
/// SoundTouch WWW: http://www.surina.net/soundtouch
///
////////////////////////////////////////////////////////////////////////////////
//
// License :
//
// SoundTouch audio processing library
// Copyright (c) Olli Parviainen
//
// This library is free software; you can redistribute it and/or
// modify it under the terms of the GNU Lesser General Public
// License as published by the Free Software Foundation; either
// version 2.1 of the License, or (at your option) any later version.
//
// This library is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
// Lesser General Public License for more details.
//
// You should have received a copy of the GNU Lesser General Public
// License along with this library; if not, write to the Free Software
// Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
//
////////////////////////////////////////////////////////////////////////////////
#include <assert.h>
#include <stdlib.h>
#include "InterpolateLinear.h"
using namespace soundtouch;
//////////////////////////////////////////////////////////////////////////////
//
// InterpolateLinearInteger - integer arithmetic implementation
//
/// fixed-point interpolation routine precision
#define SCALE 65536
// Constructor
InterpolateLinearInteger::InterpolateLinearInteger() : TransposerBase()
{
// Notice: use local function calling syntax for sake of clarity,
// to indicate the fact that C++ constructor can't call virtual functions.
resetRegisters();
setRate(1.0f);
}
void InterpolateLinearInteger::resetRegisters()
{
iFract = 0;
}
// Transposes the sample rate of the given samples using linear interpolation.
// 'Mono' version of the routine. Returns the number of samples returned in
// the "dest" buffer
int InterpolateLinearInteger::transposeMono(SAMPLETYPE *dest, const SAMPLETYPE *src, int &srcSamples)
{
int i;
int srcSampleEnd = srcSamples - 1;
int srcCount = 0;
i = 0;
while (srcCount < srcSampleEnd)
{
LONG_SAMPLETYPE temp;
assert(iFract < SCALE);
temp = (SCALE - iFract) * src[0] + iFract * src[1];
dest[i] = (SAMPLETYPE)(temp / SCALE);
i++;
iFract += iRate;
int iWhole = iFract / SCALE;
iFract -= iWhole * SCALE;
srcCount += iWhole;
src += iWhole;
}
srcSamples = srcCount;
return i;
}
// Transposes the sample rate of the given samples using linear interpolation.
// 'Stereo' version of the routine. Returns the number of samples returned in
// the "dest" buffer
int InterpolateLinearInteger::transposeStereo(SAMPLETYPE *dest, const SAMPLETYPE *src, int &srcSamples)
{
int i;
int srcSampleEnd = srcSamples - 1;
int srcCount = 0;
i = 0;
while (srcCount < srcSampleEnd)
{
LONG_SAMPLETYPE temp0;
LONG_SAMPLETYPE temp1;
assert(iFract < SCALE);
temp0 = (SCALE - iFract) * src[0] + iFract * src[2];
temp1 = (SCALE - iFract) * src[1] + iFract * src[3];
dest[0] = (SAMPLETYPE)(temp0 / SCALE);
dest[1] = (SAMPLETYPE)(temp1 / SCALE);
dest += 2;
i++;
iFract += iRate;
int iWhole = iFract / SCALE;
iFract -= iWhole * SCALE;
srcCount += iWhole;
src += 2*iWhole;
}
srcSamples = srcCount;
return i;
}
int InterpolateLinearInteger::transposeMulti(SAMPLETYPE *dest, const SAMPLETYPE *src, int &srcSamples)
{
int i;
int srcSampleEnd = srcSamples - 1;
int srcCount = 0;
i = 0;
while (srcCount < srcSampleEnd)
{
LONG_SAMPLETYPE temp, vol1;
assert(iFract < SCALE);
vol1 = (LONG_SAMPLETYPE)(SCALE - iFract);
for (int c = 0; c < numChannels; c ++)
{
temp = vol1 * src[c] + iFract * src[c + numChannels];
dest[0] = (SAMPLETYPE)(temp / SCALE);
dest ++;
}
i++;
iFract += iRate;
int iWhole = iFract / SCALE;
iFract -= iWhole * SCALE;
srcCount += iWhole;
src += iWhole * numChannels;
}
srcSamples = srcCount;
return i;
}
// Sets new target iRate. Normal iRate = 1.0, smaller values represent slower
// iRate, larger faster iRates.
void InterpolateLinearInteger::setRate(double newRate)
{
iRate = (int)(newRate * SCALE + 0.5);
TransposerBase::setRate(newRate);
}
//////////////////////////////////////////////////////////////////////////////
//
// InterpolateLinearFloat - floating point arithmetic implementation
//
//////////////////////////////////////////////////////////////////////////////
// Constructor
InterpolateLinearFloat::InterpolateLinearFloat() : TransposerBase()
{
// Notice: use local function calling syntax for sake of clarity,
// to indicate the fact that C++ constructor can't call virtual functions.
resetRegisters();
setRate(1.0);
}
void InterpolateLinearFloat::resetRegisters()
{
fract = 0;
}
// Transposes the sample rate of the given samples using linear interpolation.
// 'Mono' version of the routine. Returns the number of samples returned in
// the "dest" buffer
int InterpolateLinearFloat::transposeMono(SAMPLETYPE *dest, const SAMPLETYPE *src, int &srcSamples)
{
int i;
int srcSampleEnd = srcSamples - 1;
int srcCount = 0;
i = 0;
while (srcCount < srcSampleEnd)
{
double out;
assert(fract < 1.0);
out = (1.0 - fract) * src[0] + fract * src[1];
dest[i] = (SAMPLETYPE)out;
i ++;
// update position fraction
fract += rate;
// update whole positions
int whole = (int)fract;
fract -= whole;
src += whole;
srcCount += whole;
}
srcSamples = srcCount;
return i;
}
// Transposes the sample rate of the given samples using linear interpolation.
// 'Mono' version of the routine. Returns the number of samples returned in
// the "dest" buffer
int InterpolateLinearFloat::transposeStereo(SAMPLETYPE *dest, const SAMPLETYPE *src, int &srcSamples)
{
int i;
int srcSampleEnd = srcSamples - 1;
int srcCount = 0;
i = 0;
while (srcCount < srcSampleEnd)
{
double out0, out1;
assert(fract < 1.0);
out0 = (1.0 - fract) * src[0] + fract * src[2];
out1 = (1.0 - fract) * src[1] + fract * src[3];
dest[2*i] = (SAMPLETYPE)out0;
dest[2*i+1] = (SAMPLETYPE)out1;
i ++;
// update position fraction
fract += rate;
// update whole positions
int whole = (int)fract;
fract -= whole;
src += 2*whole;
srcCount += whole;
}
srcSamples = srcCount;
return i;
}
int InterpolateLinearFloat::transposeMulti(SAMPLETYPE *dest, const SAMPLETYPE *src, int &srcSamples)
{
int i;
int srcSampleEnd = srcSamples - 1;
int srcCount = 0;
i = 0;
while (srcCount < srcSampleEnd)
{
float temp, vol1, fract_float;
vol1 = (float)(1.0 - fract);
fract_float = (float)fract;
for (int c = 0; c < numChannels; c ++)
{
temp = vol1 * src[c] + fract_float * src[c + numChannels];
*dest = (SAMPLETYPE)temp;
dest ++;
}
i++;
fract += rate;
int iWhole = (int)fract;
fract -= iWhole;
srcCount += iWhole;
src += iWhole * numChannels;
}
srcSamples = srcCount;
return i;
}
@@ -0,0 +1,98 @@
////////////////////////////////////////////////////////////////////////////////
///
/// Linear interpolation routine.
///
/// Author : Copyright (c) Olli Parviainen
/// Author e-mail : oparviai 'at' iki.fi
/// SoundTouch WWW: http://www.surina.net/soundtouch
///
////////////////////////////////////////////////////////////////////////////////
//
// License :
//
// SoundTouch audio processing library
// Copyright (c) Olli Parviainen
//
// This library is free software; you can redistribute it and/or
// modify it under the terms of the GNU Lesser General Public
// License as published by the Free Software Foundation; either
// version 2.1 of the License, or (at your option) any later version.
//
// This library is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
// Lesser General Public License for more details.
//
// You should have received a copy of the GNU Lesser General Public
// License along with this library; if not, write to the Free Software
// Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
//
////////////////////////////////////////////////////////////////////////////////
#ifndef _InterpolateLinear_H_
#define _InterpolateLinear_H_
#include "RateTransposer.h"
#include "STTypes.h"
namespace soundtouch
{
/// Linear transposer class that uses integer arithmetic
class InterpolateLinearInteger final : public TransposerBase
{
protected:
int iFract;
int iRate;
virtual int transposeMono(SAMPLETYPE *dest,
const SAMPLETYPE *src,
int &srcSamples) override;
virtual int transposeStereo(SAMPLETYPE *dest,
const SAMPLETYPE *src,
int &srcSamples) override;
virtual int transposeMulti(SAMPLETYPE *dest, const SAMPLETYPE *src, int &srcSamples) override;
public:
InterpolateLinearInteger();
/// Sets new target rate. Normal rate = 1.0, smaller values represent slower
/// rate, larger faster rates.
virtual void setRate(double newRate) override;
virtual void resetRegisters() override;
virtual int getLatency() const override
{
return 0;
}
};
/// Linear transposer class that uses floating point arithmetic
class InterpolateLinearFloat final : public TransposerBase
{
protected:
double fract;
virtual int transposeMono(SAMPLETYPE *dest,
const SAMPLETYPE *src,
int &srcSamples);
virtual int transposeStereo(SAMPLETYPE *dest,
const SAMPLETYPE *src,
int &srcSamples);
virtual int transposeMulti(SAMPLETYPE *dest, const SAMPLETYPE *src, int &srcSamples);
public:
InterpolateLinearFloat();
virtual void resetRegisters();
int getLatency() const
{
return 0;
}
};
}
#endif
@@ -0,0 +1,181 @@
////////////////////////////////////////////////////////////////////////////////
///
/// Sample interpolation routine using 8-tap band-limited Shannon interpolation
/// with kaiser window.
///
/// Notice. This algorithm is remarkably much heavier than linear or cubic
/// interpolation, and not remarkably better than cubic algorithm. Thus mostly
/// for experimental purposes
///
/// Author : Copyright (c) Olli Parviainen
/// Author e-mail : oparviai 'at' iki.fi
/// SoundTouch WWW: http://www.surina.net/soundtouch
///
////////////////////////////////////////////////////////////////////////////////
//
// License :
//
// SoundTouch audio processing library
// Copyright (c) Olli Parviainen
//
// This library is free software; you can redistribute it and/or
// modify it under the terms of the GNU Lesser General Public
// License as published by the Free Software Foundation; either
// version 2.1 of the License, or (at your option) any later version.
//
// This library is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
// Lesser General Public License for more details.
//
// You should have received a copy of the GNU Lesser General Public
// License along with this library; if not, write to the Free Software
// Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
//
////////////////////////////////////////////////////////////////////////////////
#include <math.h>
#include "InterpolateShannon.h"
#include "STTypes.h"
using namespace soundtouch;
/// Kaiser window with beta = 2.0
/// Values scaled down by 5% to avoid overflows
static const double _kaiser8[8] =
{
0.41778693317814,
0.64888025049173,
0.83508562409944,
0.93887857733412,
0.93887857733412,
0.83508562409944,
0.64888025049173,
0.41778693317814
};
InterpolateShannon::InterpolateShannon()
{
fract = 0;
}
void InterpolateShannon::resetRegisters()
{
fract = 0;
}
#define PI 3.1415926536
#define sinc(x) (sin(PI * (x)) / (PI * (x)))
/// Transpose mono audio. Returns number of produced output samples, and
/// updates "srcSamples" to amount of consumed source samples
int InterpolateShannon::transposeMono(SAMPLETYPE *pdest,
const SAMPLETYPE *psrc,
int &srcSamples)
{
int i;
int srcSampleEnd = srcSamples - 8;
int srcCount = 0;
i = 0;
while (srcCount < srcSampleEnd)
{
double out;
assert(fract < 1.0);
out = psrc[0] * sinc(-3.0 - fract) * _kaiser8[0];
out += psrc[1] * sinc(-2.0 - fract) * _kaiser8[1];
out += psrc[2] * sinc(-1.0 - fract) * _kaiser8[2];
if (fract < 1e-6)
{
out += psrc[3] * _kaiser8[3]; // sinc(0) = 1
}
else
{
out += psrc[3] * sinc(- fract) * _kaiser8[3];
}
out += psrc[4] * sinc( 1.0 - fract) * _kaiser8[4];
out += psrc[5] * sinc( 2.0 - fract) * _kaiser8[5];
out += psrc[6] * sinc( 3.0 - fract) * _kaiser8[6];
out += psrc[7] * sinc( 4.0 - fract) * _kaiser8[7];
pdest[i] = (SAMPLETYPE)out;
i ++;
// update position fraction
fract += rate;
// update whole positions
int whole = (int)fract;
fract -= whole;
psrc += whole;
srcCount += whole;
}
srcSamples = srcCount;
return i;
}
/// Transpose stereo audio. Returns number of produced output samples, and
/// updates "srcSamples" to amount of consumed source samples
int InterpolateShannon::transposeStereo(SAMPLETYPE *pdest,
const SAMPLETYPE *psrc,
int &srcSamples)
{
int i;
int srcSampleEnd = srcSamples - 8;
int srcCount = 0;
i = 0;
while (srcCount < srcSampleEnd)
{
double out0, out1, w;
assert(fract < 1.0);
w = sinc(-3.0 - fract) * _kaiser8[0];
out0 = psrc[0] * w; out1 = psrc[1] * w;
w = sinc(-2.0 - fract) * _kaiser8[1];
out0 += psrc[2] * w; out1 += psrc[3] * w;
w = sinc(-1.0 - fract) * _kaiser8[2];
out0 += psrc[4] * w; out1 += psrc[5] * w;
w = _kaiser8[3] * ((fract < 1e-5) ? 1.0 : sinc(- fract)); // sinc(0) = 1
out0 += psrc[6] * w; out1 += psrc[7] * w;
w = sinc( 1.0 - fract) * _kaiser8[4];
out0 += psrc[8] * w; out1 += psrc[9] * w;
w = sinc( 2.0 - fract) * _kaiser8[5];
out0 += psrc[10] * w; out1 += psrc[11] * w;
w = sinc( 3.0 - fract) * _kaiser8[6];
out0 += psrc[12] * w; out1 += psrc[13] * w;
w = sinc( 4.0 - fract) * _kaiser8[7];
out0 += psrc[14] * w; out1 += psrc[15] * w;
pdest[2*i] = (SAMPLETYPE)out0;
pdest[2*i+1] = (SAMPLETYPE)out1;
i ++;
// update position fraction
fract += rate;
// update whole positions
int whole = (int)fract;
fract -= whole;
psrc += 2*whole;
srcCount += whole;
}
srcSamples = srcCount;
return i;
}
/// Transpose stereo audio. Returns number of produced output samples, and
/// updates "srcSamples" to amount of consumed source samples
int InterpolateShannon::transposeMulti(SAMPLETYPE *,
const SAMPLETYPE *,
int &)
{
// not implemented
assert(false);
return 0;
}
@@ -0,0 +1,74 @@
////////////////////////////////////////////////////////////////////////////////
///
/// Sample interpolation routine using 8-tap band-limited Shannon interpolation
/// with kaiser window.
///
/// Notice. This algorithm is remarkably much heavier than linear or cubic
/// interpolation, and not remarkably better than cubic algorithm. Thus mostly
/// for experimental purposes
///
/// Author : Copyright (c) Olli Parviainen
/// Author e-mail : oparviai 'at' iki.fi
/// SoundTouch WWW: http://www.surina.net/soundtouch
///
////////////////////////////////////////////////////////////////////////////////
//
// License :
//
// SoundTouch audio processing library
// Copyright (c) Olli Parviainen
//
// This library is free software; you can redistribute it and/or
// modify it under the terms of the GNU Lesser General Public
// License as published by the Free Software Foundation; either
// version 2.1 of the License, or (at your option) any later version.
//
// This library is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
// Lesser General Public License for more details.
//
// You should have received a copy of the GNU Lesser General Public
// License along with this library; if not, write to the Free Software
// Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
//
////////////////////////////////////////////////////////////////////////////////
#ifndef _InterpolateShannon_H_
#define _InterpolateShannon_H_
#include "RateTransposer.h"
#include "STTypes.h"
namespace soundtouch
{
class InterpolateShannon final : public TransposerBase
{
protected:
int transposeMono(SAMPLETYPE *dest,
const SAMPLETYPE *src,
int &srcSamples) override;
int transposeStereo(SAMPLETYPE *dest,
const SAMPLETYPE *src,
int &srcSamples) override;
int transposeMulti(SAMPLETYPE *dest,
const SAMPLETYPE *src,
int &srcSamples) override;
double fract;
public:
InterpolateShannon();
void resetRegisters() override;
virtual int getLatency() const override
{
return 3;
}
};
}
#endif
@@ -0,0 +1,277 @@
////////////////////////////////////////////////////////////////////////////////
///
/// Peak detection routine.
///
/// The routine detects highest value on an array of values and calculates the
/// precise peak location as a mass-center of the 'hump' around the peak value.
///
/// Author : Copyright (c) Olli Parviainen
/// Author e-mail : oparviai 'at' iki.fi
/// SoundTouch WWW: http://www.surina.net/soundtouch
///
////////////////////////////////////////////////////////////////////////////////
//
// License :
//
// SoundTouch audio processing library
// Copyright (c) Olli Parviainen
//
// This library is free software; you can redistribute it and/or
// modify it under the terms of the GNU Lesser General Public
// License as published by the Free Software Foundation; either
// version 2.1 of the License, or (at your option) any later version.
//
// This library is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
// Lesser General Public License for more details.
//
// You should have received a copy of the GNU Lesser General Public
// License along with this library; if not, write to the Free Software
// Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
//
////////////////////////////////////////////////////////////////////////////////
#include <math.h>
#include <assert.h>
#include "PeakFinder.h"
using namespace soundtouch;
#define max(x, y) (((x) > (y)) ? (x) : (y))
PeakFinder::PeakFinder()
{
minPos = maxPos = 0;
}
// Finds real 'top' of a peak hump from neighnourhood of the given 'peakpos'.
int PeakFinder::findTop(const float *data, int peakpos) const
{
int i;
int start, end;
float refvalue;
refvalue = data[peakpos];
// seek within ±10 points
start = peakpos - 10;
if (start < minPos) start = minPos;
end = peakpos + 10;
if (end > maxPos) end = maxPos;
for (i = start; i <= end; i ++)
{
if (data[i] > refvalue)
{
peakpos = i;
refvalue = data[i];
}
}
// failure if max value is at edges of seek range => it's not peak, it's at slope.
if ((peakpos == start) || (peakpos == end)) return 0;
return peakpos;
}
// Finds 'ground level' of a peak hump by starting from 'peakpos' and proceeding
// to direction defined by 'direction' until next 'hump' after minimum value will
// begin
int PeakFinder::findGround(const float *data, int peakpos, int direction) const
{
int lowpos;
int pos;
int climb_count;
float refvalue;
float delta;
climb_count = 0;
refvalue = data[peakpos];
lowpos = peakpos;
pos = peakpos;
while ((pos > minPos+1) && (pos < maxPos-1))
{
int prevpos;
prevpos = pos;
pos += direction;
// calculate derivate
delta = data[pos] - data[prevpos];
if (delta <= 0)
{
// going downhill, ok
if (climb_count)
{
climb_count --; // decrease climb count
}
// check if new minimum found
if (data[pos] < refvalue)
{
// new minimum found
lowpos = pos;
refvalue = data[pos];
}
}
else
{
// going uphill, increase climbing counter
climb_count ++;
if (climb_count > 5) break; // we've been climbing too long => it's next uphill => quit
}
}
return lowpos;
}
// Find offset where the value crosses the given level, when starting from 'peakpos' and
// proceeds to direction defined in 'direction'
int PeakFinder::findCrossingLevel(const float *data, float level, int peakpos, int direction) const
{
float peaklevel;
int pos;
peaklevel = data[peakpos];
assert(peaklevel >= level);
pos = peakpos;
while ((pos >= minPos) && (pos + direction < maxPos))
{
if (data[pos + direction] < level) return pos; // crossing found
pos += direction;
}
return -1; // not found
}
// Calculates the center of mass location of 'data' array items between 'firstPos' and 'lastPos'
double PeakFinder::calcMassCenter(const float *data, int firstPos, int lastPos) const
{
int i;
float sum;
float wsum;
sum = 0;
wsum = 0;
for (i = firstPos; i <= lastPos; i ++)
{
sum += (float)i * data[i];
wsum += data[i];
}
if (wsum < 1e-6) return 0;
return sum / wsum;
}
/// get exact center of peak near given position by calculating local mass of center
double PeakFinder::getPeakCenter(const float *data, int peakpos) const
{
float peakLevel; // peak level
int crosspos1, crosspos2; // position where the peak 'hump' crosses cutting level
float cutLevel; // cutting value
float groundLevel; // ground level of the peak
int gp1, gp2; // bottom positions of the peak 'hump'
// find ground positions.
gp1 = findGround(data, peakpos, -1);
gp2 = findGround(data, peakpos, 1);
peakLevel = data[peakpos];
if (gp1 == gp2)
{
// avoid rounding errors when all are equal
assert(gp1 == peakpos);
cutLevel = groundLevel = peakLevel;
} else {
// get average of the ground levels
groundLevel = 0.5f * (data[gp1] + data[gp2]);
// calculate 70%-level of the peak
cutLevel = 0.70f * peakLevel + 0.30f * groundLevel;
}
// find mid-level crossings
crosspos1 = findCrossingLevel(data, cutLevel, peakpos, -1);
crosspos2 = findCrossingLevel(data, cutLevel, peakpos, 1);
if ((crosspos1 < 0) || (crosspos2 < 0)) return 0; // no crossing, no peak..
// calculate mass center of the peak surroundings
return calcMassCenter(data, crosspos1, crosspos2);
}
double PeakFinder::detectPeak(const float *data, int aminPos, int amaxPos)
{
int i;
int peakpos; // position of peak level
double highPeak, peak;
this->minPos = aminPos;
this->maxPos = amaxPos;
// find absolute peak
peakpos = minPos;
peak = data[minPos];
for (i = minPos + 1; i < maxPos; i ++)
{
if (data[i] > peak)
{
peak = data[i];
peakpos = i;
}
}
// Calculate exact location of the highest peak mass center
highPeak = getPeakCenter(data, peakpos);
peak = highPeak;
// Now check if the highest peak were in fact harmonic of the true base beat peak
// - sometimes the highest peak can be Nth harmonic of the true base peak yet
// just a slightly higher than the true base
for (i = 1; i < 3; i ++)
{
double peaktmp, harmonic;
int i1,i2;
harmonic = (double)pow(2.0, i);
peakpos = (int)(highPeak / harmonic + 0.5f);
if (peakpos < minPos) break;
peakpos = findTop(data, peakpos); // seek true local maximum index
if (peakpos == 0) continue; // no local max here
// calculate mass-center of possible harmonic peak
peaktmp = getPeakCenter(data, peakpos);
// accept harmonic peak if
// (a) it is found
// (b) is within ±4% of the expected harmonic interval
// (c) has at least half x-corr value of the max. peak
double diff = harmonic * peaktmp / highPeak;
if ((diff < 0.96) || (diff > 1.04)) continue; // peak too afar from expected
// now compare to highest detected peak
i1 = (int)(highPeak + 0.5);
i2 = (int)(peaktmp + 0.5);
if (data[i2] >= 0.4*data[i1])
{
// The harmonic is at least half as high primary peak,
// thus use the harmonic peak instead
peak = peaktmp;
}
}
return peak;
}
@@ -0,0 +1,90 @@
////////////////////////////////////////////////////////////////////////////////
///
/// The routine detects highest value on an array of values and calculates the
/// precise peak location as a mass-center of the 'hump' around the peak value.
///
/// Author : Copyright (c) Olli Parviainen
/// Author e-mail : oparviai 'at' iki.fi
/// SoundTouch WWW: http://www.surina.net/soundtouch
///
////////////////////////////////////////////////////////////////////////////////
//
// License :
//
// SoundTouch audio processing library
// Copyright (c) Olli Parviainen
//
// This library is free software; you can redistribute it and/or
// modify it under the terms of the GNU Lesser General Public
// License as published by the Free Software Foundation; either
// version 2.1 of the License, or (at your option) any later version.
//
// This library is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
// Lesser General Public License for more details.
//
// You should have received a copy of the GNU Lesser General Public
// License along with this library; if not, write to the Free Software
// Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
//
////////////////////////////////////////////////////////////////////////////////
#ifndef _PeakFinder_H_
#define _PeakFinder_H_
namespace soundtouch
{
class PeakFinder
{
protected:
/// Min, max allowed peak positions within the data vector
int minPos, maxPos;
/// Calculates the mass center between given vector items.
double calcMassCenter(const float *data, ///< Data vector.
int firstPos, ///< Index of first vector item belonging to the peak.
int lastPos ///< Index of last vector item belonging to the peak.
) const;
/// Finds the data vector index where the monotoniously decreasing signal crosses the
/// given level.
int findCrossingLevel(const float *data, ///< Data vector.
float level, ///< Goal crossing level.
int peakpos, ///< Peak position index within the data vector.
int direction /// Direction where to proceed from the peak: 1 = right, -1 = left.
) const;
// Finds real 'top' of a peak hump from neighnourhood of the given 'peakpos'.
int findTop(const float *data, int peakpos) const;
/// Finds the 'ground' level, i.e. smallest level between two neighbouring peaks, to right-
/// or left-hand side of the given peak position.
int findGround(const float *data, /// Data vector.
int peakpos, /// Peak position index within the data vector.
int direction /// Direction where to proceed from the peak: 1 = right, -1 = left.
) const;
/// get exact center of peak near given position by calculating local mass of center
double getPeakCenter(const float *data, int peakpos) const;
public:
/// Constructor.
PeakFinder();
/// Detect exact peak position of the data vector by finding the largest peak 'hump'
/// and calculating the mass-center location of the peak hump.
///
/// \return The location of the largest base harmonic peak hump.
double detectPeak(const float *data, /// Data vector to be analyzed. The data vector has
/// to be at least 'maxPos' items long.
int minPos, ///< Min allowed peak location within the vector data.
int maxPos ///< Max allowed peak location within the vector data.
);
};
}
#endif // _PeakFinder_H_
@@ -0,0 +1,313 @@
////////////////////////////////////////////////////////////////////////////////
///
/// Sample rate transposer. Changes sample rate by using linear interpolation
/// together with anti-alias filtering (first order interpolation with anti-
/// alias filtering should be quite adequate for this application)
///
/// Author : Copyright (c) Olli Parviainen
/// Author e-mail : oparviai 'at' iki.fi
/// SoundTouch WWW: http://www.surina.net/soundtouch
///
////////////////////////////////////////////////////////////////////////////////
//
// License :
//
// SoundTouch audio processing library
// Copyright (c) Olli Parviainen
//
// This library is free software; you can redistribute it and/or
// modify it under the terms of the GNU Lesser General Public
// License as published by the Free Software Foundation; either
// version 2.1 of the License, or (at your option) any later version.
//
// This library is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
// Lesser General Public License for more details.
//
// You should have received a copy of the GNU Lesser General Public
// License along with this library; if not, write to the Free Software
// Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
//
////////////////////////////////////////////////////////////////////////////////
#include <memory.h>
#include <assert.h>
#include <stdlib.h>
#include <stdio.h>
#include "RateTransposer.h"
#include "InterpolateLinear.h"
#include "InterpolateCubic.h"
#include "InterpolateShannon.h"
#include "AAFilter.h"
using namespace soundtouch;
// Define default interpolation algorithm here
TransposerBase::ALGORITHM TransposerBase::algorithm = TransposerBase::CUBIC;
// Constructor
RateTransposer::RateTransposer() : FIFOProcessor(&outputBuffer)
{
bUseAAFilter =
#ifndef SOUNDTOUCH_PREVENT_CLICK_AT_RATE_CROSSOVER
true;
#else
// Disable Anti-alias filter if desirable to avoid click at rate change zero value crossover
false;
#endif
// Instantiates the anti-alias filter
pAAFilter = new AAFilter(64);
pTransposer = TransposerBase::newInstance();
clear();
}
RateTransposer::~RateTransposer()
{
delete pAAFilter;
delete pTransposer;
}
/// Enables/disables the anti-alias filter. Zero to disable, nonzero to enable
void RateTransposer::enableAAFilter(bool newMode)
{
#ifndef SOUNDTOUCH_PREVENT_CLICK_AT_RATE_CROSSOVER
// Disable Anti-alias filter if desirable to avoid click at rate change zero value crossover
bUseAAFilter = newMode;
clear();
#endif
}
/// Returns nonzero if anti-alias filter is enabled.
bool RateTransposer::isAAFilterEnabled() const
{
return bUseAAFilter;
}
AAFilter *RateTransposer::getAAFilter()
{
return pAAFilter;
}
// Sets new target iRate. Normal iRate = 1.0, smaller values represent slower
// iRate, larger faster iRates.
void RateTransposer::setRate(double newRate)
{
double fCutoff;
pTransposer->setRate(newRate);
// design a new anti-alias filter
if (newRate > 1.0)
{
fCutoff = 0.5 / newRate;
}
else
{
fCutoff = 0.5 * newRate;
}
pAAFilter->setCutoffFreq(fCutoff);
}
// Adds 'nSamples' pcs of samples from the 'samples' memory position into
// the input of the object.
void RateTransposer::putSamples(const SAMPLETYPE *samples, uint nSamples)
{
processSamples(samples, nSamples);
}
// Transposes sample rate by applying anti-alias filter to prevent folding.
// Returns amount of samples returned in the "dest" buffer.
// The maximum amount of samples that can be returned at a time is set by
// the 'set_returnBuffer_size' function.
void RateTransposer::processSamples(const SAMPLETYPE *src, uint nSamples)
{
if (nSamples == 0) return;
// Store samples to input buffer
inputBuffer.putSamples(src, nSamples);
// If anti-alias filter is turned off, simply transpose without applying
// the filter
if (bUseAAFilter == false)
{
(void)pTransposer->transpose(outputBuffer, inputBuffer);
return;
}
assert(pAAFilter);
// Transpose with anti-alias filter
if (pTransposer->rate < 1.0f)
{
// If the parameter 'Rate' value is smaller than 1, first transpose
// the samples and then apply the anti-alias filter to remove aliasing.
// Transpose the samples, store the result to end of "midBuffer"
pTransposer->transpose(midBuffer, inputBuffer);
// Apply the anti-alias filter for transposed samples in midBuffer
pAAFilter->evaluate(outputBuffer, midBuffer);
}
else
{
// If the parameter 'Rate' value is larger than 1, first apply the
// anti-alias filter to remove high frequencies (prevent them from folding
// over the lover frequencies), then transpose.
// Apply the anti-alias filter for samples in inputBuffer
pAAFilter->evaluate(midBuffer, inputBuffer);
// Transpose the AA-filtered samples in "midBuffer"
pTransposer->transpose(outputBuffer, midBuffer);
}
}
// Sets the number of channels, 1 = mono, 2 = stereo
void RateTransposer::setChannels(int nChannels)
{
if (!verifyNumberOfChannels(nChannels) ||
(pTransposer->numChannels == nChannels)) return;
pTransposer->setChannels(nChannels);
inputBuffer.setChannels(nChannels);
midBuffer.setChannels(nChannels);
outputBuffer.setChannels(nChannels);
}
// Clears all the samples in the object
void RateTransposer::clear()
{
outputBuffer.clear();
midBuffer.clear();
inputBuffer.clear();
pTransposer->resetRegisters();
// prefill buffer to avoid losing first samples at beginning of stream
int prefill = getLatency();
inputBuffer.addSilent(prefill);
}
// Returns nonzero if there aren't any samples available for outputting.
int RateTransposer::isEmpty() const
{
int res;
res = FIFOProcessor::isEmpty();
if (res == 0) return 0;
return inputBuffer.isEmpty();
}
/// Return approximate initial input-output latency
int RateTransposer::getLatency() const
{
return pTransposer->getLatency() +
((bUseAAFilter) ? (pAAFilter->getLength() / 2) : 0);
}
//////////////////////////////////////////////////////////////////////////////
//
// TransposerBase - Base class for interpolation
//
// static function to set interpolation algorithm
void TransposerBase::setAlgorithm(TransposerBase::ALGORITHM a)
{
TransposerBase::algorithm = a;
}
// Transposes the sample rate of the given samples using linear interpolation.
// Returns the number of samples returned in the "dest" buffer
int TransposerBase::transpose(FIFOSampleBuffer &dest, FIFOSampleBuffer &src)
{
int numSrcSamples = src.numSamples();
int sizeDemand = (int)((double)numSrcSamples / rate) + 8;
int numOutput;
SAMPLETYPE *psrc = src.ptrBegin();
SAMPLETYPE *pdest = dest.ptrEnd(sizeDemand);
#ifndef USE_MULTICH_ALWAYS
if (numChannels == 1)
{
numOutput = transposeMono(pdest, psrc, numSrcSamples);
}
else if (numChannels == 2)
{
numOutput = transposeStereo(pdest, psrc, numSrcSamples);
}
else
#endif // USE_MULTICH_ALWAYS
{
assert(numChannels > 0);
numOutput = transposeMulti(pdest, psrc, numSrcSamples);
}
dest.putSamples(numOutput);
src.receiveSamples(numSrcSamples);
return numOutput;
}
TransposerBase::TransposerBase()
{
numChannels = 0;
rate = 1.0f;
}
TransposerBase::~TransposerBase()
{
}
void TransposerBase::setChannels(int channels)
{
numChannels = channels;
resetRegisters();
}
void TransposerBase::setRate(double newRate)
{
rate = newRate;
}
// static factory function
TransposerBase *TransposerBase::newInstance()
{
#ifdef SOUNDTOUCH_INTEGER_SAMPLES
// Notice: For integer arithmetic support only linear algorithm (due to simplest calculus)
return ::new InterpolateLinearInteger;
#else
switch (algorithm)
{
case LINEAR:
return new InterpolateLinearFloat;
case CUBIC:
return new InterpolateCubic;
case SHANNON:
return new InterpolateShannon;
default:
assert(false);
return nullptr;
}
#endif
}
@@ -0,0 +1,164 @@
////////////////////////////////////////////////////////////////////////////////
///
/// Sample rate transposer. Changes sample rate by using linear interpolation
/// together with anti-alias filtering (first order interpolation with anti-
/// alias filtering should be quite adequate for this application).
///
/// Use either of the derived classes of 'RateTransposerInteger' or
/// 'RateTransposerFloat' for corresponding integer/floating point tranposing
/// algorithm implementation.
///
/// Author : Copyright (c) Olli Parviainen
/// Author e-mail : oparviai 'at' iki.fi
/// SoundTouch WWW: http://www.surina.net/soundtouch
///
////////////////////////////////////////////////////////////////////////////////
//
// License :
//
// SoundTouch audio processing library
// Copyright (c) Olli Parviainen
//
// This library is free software; you can redistribute it and/or
// modify it under the terms of the GNU Lesser General Public
// License as published by the Free Software Foundation; either
// version 2.1 of the License, or (at your option) any later version.
//
// This library is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
// Lesser General Public License for more details.
//
// You should have received a copy of the GNU Lesser General Public
// License along with this library; if not, write to the Free Software
// Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
//
////////////////////////////////////////////////////////////////////////////////
#ifndef RateTransposer_H
#define RateTransposer_H
#include <stddef.h>
#include "AAFilter.h"
#include "FIFOSamplePipe.h"
#include "FIFOSampleBuffer.h"
#include "STTypes.h"
namespace soundtouch
{
/// Abstract base class for transposer implementations (linear, advanced vs integer, float etc)
class TransposerBase
{
public:
enum ALGORITHM {
LINEAR = 0,
CUBIC,
SHANNON
};
protected:
virtual int transposeMono(SAMPLETYPE *dest,
const SAMPLETYPE *src,
int &srcSamples) = 0;
virtual int transposeStereo(SAMPLETYPE *dest,
const SAMPLETYPE *src,
int &srcSamples) = 0;
virtual int transposeMulti(SAMPLETYPE *dest,
const SAMPLETYPE *src,
int &srcSamples) = 0;
static ALGORITHM algorithm;
public:
double rate;
int numChannels;
TransposerBase();
virtual ~TransposerBase();
virtual int transpose(FIFOSampleBuffer &dest, FIFOSampleBuffer &src);
virtual void setRate(double newRate);
virtual void setChannels(int channels);
virtual int getLatency() const = 0;
virtual void resetRegisters() = 0;
// static factory function
static TransposerBase *newInstance();
// static function to set interpolation algorithm
static void setAlgorithm(ALGORITHM a);
};
/// A common linear samplerate transposer class.
///
class RateTransposer final : public FIFOProcessor
{
protected:
/// Anti-alias filter object
AAFilter *pAAFilter;
TransposerBase *pTransposer;
/// Buffer for collecting samples to feed the anti-alias filter between
/// two batches
FIFOSampleBuffer inputBuffer;
/// Buffer for keeping samples between transposing & anti-alias filter
FIFOSampleBuffer midBuffer;
/// Output sample buffer
FIFOSampleBuffer outputBuffer;
bool bUseAAFilter;
/// Transposes sample rate by applying anti-alias filter to prevent folding.
/// Returns amount of samples returned in the "dest" buffer.
/// The maximum amount of samples that can be returned at a time is set by
/// the 'set_returnBuffer_size' function.
void processSamples(const SAMPLETYPE *src,
uint numSamples);
public:
RateTransposer();
virtual ~RateTransposer() override;
/// Returns the output buffer object
FIFOSamplePipe *getOutput() { return &outputBuffer; };
/// Return anti-alias filter object
AAFilter *getAAFilter();
/// Enables/disables the anti-alias filter. Zero to disable, nonzero to enable
void enableAAFilter(bool newMode);
/// Returns nonzero if anti-alias filter is enabled.
bool isAAFilterEnabled() const;
/// Sets new target rate. Normal rate = 1.0, smaller values represent slower
/// rate, larger faster rates.
virtual void setRate(double newRate);
/// Sets the number of channels, 1 = mono, 2 = stereo
void setChannels(int channels);
/// Adds 'numSamples' pcs of samples from the 'samples' memory position into
/// the input of the object.
void putSamples(const SAMPLETYPE *samples, uint numSamples) override;
/// Clears all the samples in the object
void clear() override;
/// Returns nonzero if there aren't any samples available for outputting.
int isEmpty() const override;
/// Return approximate initial input-output latency
int getLatency() const;
};
}
#endif
@@ -0,0 +1,538 @@
//////////////////////////////////////////////////////////////////////////////
///
/// SoundTouch - main class for tempo/pitch/rate adjusting routines.
///
/// Notes:
/// - Initialize the SoundTouch object instance by setting up the sound stream
/// parameters with functions 'setSampleRate' and 'setChannels', then set
/// desired tempo/pitch/rate settings with the corresponding functions.
///
/// - The SoundTouch class behaves like a first-in-first-out pipeline: The
/// samples that are to be processed are fed into one of the pipe by calling
/// function 'putSamples', while the ready processed samples can be read
/// from the other end of the pipeline with function 'receiveSamples'.
///
/// - The SoundTouch processing classes require certain sized 'batches' of
/// samples in order to process the sound. For this reason the classes buffer
/// incoming samples until there are enough of samples available for
/// processing, then they carry out the processing step and consequently
/// make the processed samples available for outputting.
///
/// - For the above reason, the processing routines introduce a certain
/// 'latency' between the input and output, so that the samples input to
/// SoundTouch may not be immediately available in the output, and neither
/// the amount of outputtable samples may not immediately be in direct
/// relationship with the amount of previously input samples.
///
/// - The tempo/pitch/rate control parameters can be altered during processing.
/// Please notice though that they aren't currently protected by semaphores,
/// so in multi-thread application external semaphore protection may be
/// required.
///
/// - This class utilizes classes 'TDStretch' for tempo change (without modifying
/// pitch) and 'RateTransposer' for changing the playback rate (that is, both
/// tempo and pitch in the same ratio) of the sound. The third available control
/// 'pitch' (change pitch but maintain tempo) is produced by a combination of
/// combining the two other controls.
///
/// Author : Copyright (c) Olli Parviainen
/// Author e-mail : oparviai 'at' iki.fi
/// SoundTouch WWW: http://www.surina.net/soundtouch
///
////////////////////////////////////////////////////////////////////////////////
//
// License :
//
// SoundTouch audio processing library
// Copyright (c) Olli Parviainen
//
// This library is free software; you can redistribute it and/or
// modify it under the terms of the GNU Lesser General Public
// License as published by the Free Software Foundation; either
// version 2.1 of the License, or (at your option) any later version.
//
// This library is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
// Lesser General Public License for more details.
//
// You should have received a copy of the GNU Lesser General Public
// License along with this library; if not, write to the Free Software
// Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
//
////////////////////////////////////////////////////////////////////////////////
#include <assert.h>
#include <stdlib.h>
#include <memory.h>
#include <math.h>
#include <stdio.h>
#include "SoundTouch.h"
#include "TDStretch.h"
#include "RateTransposer.h"
#include "cpu_detect.h"
using namespace soundtouch;
/// test if two floating point numbers are equal
#define TEST_FLOAT_EQUAL(a, b) (fabs(a - b) < 1e-10)
/// Print library version string for autoconf
extern "C" void soundtouch_ac_test()
{
printf("SoundTouch Version: %s\n",SOUNDTOUCH_VERSION);
}
SoundTouch::SoundTouch()
{
// Initialize rate transposer and tempo changer instances
pRateTransposer = new RateTransposer();
pTDStretch = TDStretch::newInstance();
setOutPipe(pTDStretch);
rate = tempo = 0;
virtualPitch =
virtualRate =
virtualTempo = 1.0;
calcEffectiveRateAndTempo();
samplesExpectedOut = 0;
samplesOutput = 0;
channels = 0;
bSrateSet = false;
}
SoundTouch::~SoundTouch()
{
delete pRateTransposer;
delete pTDStretch;
}
/// Get SoundTouch library version string
const char *SoundTouch::getVersionString()
{
static const char *_version = SOUNDTOUCH_VERSION;
return _version;
}
/// Get SoundTouch library version Id
uint SoundTouch::getVersionId()
{
return SOUNDTOUCH_VERSION_ID;
}
// Sets the number of channels, 1 = mono, 2 = stereo
void SoundTouch::setChannels(uint numChannels)
{
if (!verifyNumberOfChannels(numChannels)) return;
channels = numChannels;
pRateTransposer->setChannels((int)numChannels);
pTDStretch->setChannels((int)numChannels);
}
// Sets new rate control value. Normal rate = 1.0, smaller values
// represent slower rate, larger faster rates.
void SoundTouch::setRate(double newRate)
{
virtualRate = newRate;
calcEffectiveRateAndTempo();
}
// Sets new rate control value as a difference in percents compared
// to the original rate (-50 .. +100 %)
void SoundTouch::setRateChange(double newRate)
{
virtualRate = 1.0 + 0.01 * newRate;
calcEffectiveRateAndTempo();
}
// Sets new tempo control value. Normal tempo = 1.0, smaller values
// represent slower tempo, larger faster tempo.
void SoundTouch::setTempo(double newTempo)
{
virtualTempo = newTempo;
calcEffectiveRateAndTempo();
}
// Sets new tempo control value as a difference in percents compared
// to the original tempo (-50 .. +100 %)
void SoundTouch::setTempoChange(double newTempo)
{
virtualTempo = 1.0 + 0.01 * newTempo;
calcEffectiveRateAndTempo();
}
// Sets new pitch control value. Original pitch = 1.0, smaller values
// represent lower pitches, larger values higher pitch.
void SoundTouch::setPitch(double newPitch)
{
virtualPitch = newPitch;
calcEffectiveRateAndTempo();
}
// Sets pitch change in octaves compared to the original pitch
// (-1.00 .. +1.00)
void SoundTouch::setPitchOctaves(double newPitch)
{
virtualPitch = exp(0.69314718056 * newPitch);
calcEffectiveRateAndTempo();
}
// Sets pitch change in semi-tones compared to the original pitch
// (-12 .. +12)
void SoundTouch::setPitchSemiTones(int newPitch)
{
setPitchOctaves((double)newPitch / 12.0);
}
void SoundTouch::setPitchSemiTones(double newPitch)
{
setPitchOctaves(newPitch / 12.0);
}
// Calculates 'effective' rate and tempo values from the
// nominal control values.
void SoundTouch::calcEffectiveRateAndTempo()
{
double oldTempo = tempo;
double oldRate = rate;
tempo = virtualTempo / virtualPitch;
rate = virtualPitch * virtualRate;
if (!TEST_FLOAT_EQUAL(rate,oldRate)) pRateTransposer->setRate(rate);
if (!TEST_FLOAT_EQUAL(tempo, oldTempo)) pTDStretch->setTempo(tempo);
#ifndef SOUNDTOUCH_PREVENT_CLICK_AT_RATE_CROSSOVER
if (rate <= 1.0f)
{
if (output != pTDStretch)
{
FIFOSamplePipe *tempoOut;
assert(output == pRateTransposer);
// move samples in the current output buffer to the output of pTDStretch
tempoOut = pTDStretch->getOutput();
tempoOut->moveSamples(*output);
// move samples in pitch transposer's store buffer to tempo changer's input
// deprecated : pTDStretch->moveSamples(*pRateTransposer->getStore());
output = pTDStretch;
}
}
else
#endif
{
if (output != pRateTransposer)
{
FIFOSamplePipe *transOut;
assert(output == pTDStretch);
// move samples in the current output buffer to the output of pRateTransposer
transOut = pRateTransposer->getOutput();
transOut->moveSamples(*output);
// move samples in tempo changer's input to pitch transposer's input
pRateTransposer->moveSamples(*pTDStretch->getInput());
output = pRateTransposer;
}
}
}
// Sets sample rate.
void SoundTouch::setSampleRate(uint srate)
{
// set sample rate, leave other tempo changer parameters as they are.
pTDStretch->setParameters((int)srate);
bSrateSet = true;
}
// Adds 'numSamples' pcs of samples from the 'samples' memory position into
// the input of the object.
void SoundTouch::putSamples(const SAMPLETYPE *samples, uint nSamples)
{
if (bSrateSet == false)
{
ST_THROW_RT_ERROR("SoundTouch : Sample rate not defined");
}
else if (channels == 0)
{
ST_THROW_RT_ERROR("SoundTouch : Number of channels not defined");
}
// accumulate how many samples are expected out from processing, given the current
// processing setting
samplesExpectedOut += (double)nSamples / ((double)rate * (double)tempo);
#ifndef SOUNDTOUCH_PREVENT_CLICK_AT_RATE_CROSSOVER
if (rate <= 1.0f)
{
// transpose the rate down, output the transposed sound to tempo changer buffer
assert(output == pTDStretch);
pRateTransposer->putSamples(samples, nSamples);
pTDStretch->moveSamples(*pRateTransposer);
}
else
#endif
{
// evaluate the tempo changer, then transpose the rate up,
assert(output == pRateTransposer);
pTDStretch->putSamples(samples, nSamples);
pRateTransposer->moveSamples(*pTDStretch);
}
}
// Flushes the last samples from the processing pipeline to the output.
// Clears also the internal processing buffers.
//
// Note: This function is meant for extracting the last samples of a sound
// stream. This function may introduce additional blank samples in the end
// of the sound stream, and thus it's not recommended to call this function
// in the middle of a sound stream.
void SoundTouch::flush()
{
int i;
int numStillExpected;
SAMPLETYPE *buff = new SAMPLETYPE[128 * channels];
// how many samples are still expected to output
numStillExpected = (int)((long)(samplesExpectedOut + 0.5) - samplesOutput);
if (numStillExpected < 0) numStillExpected = 0;
memset(buff, 0, 128 * channels * sizeof(SAMPLETYPE));
// "Push" the last active samples out from the processing pipeline by
// feeding blank samples into the processing pipeline until new,
// processed samples appear in the output (not however, more than
// 24ksamples in any case)
for (i = 0; (numStillExpected > (int)numSamples()) && (i < 200); i ++)
{
putSamples(buff, 128);
}
adjustAmountOfSamples(numStillExpected);
delete[] buff;
// Clear input buffers
pTDStretch->clearInput();
// yet leave the output intouched as that's where the
// flushed samples are!
}
// Changes a setting controlling the processing system behaviour. See the
// 'SETTING_...' defines for available setting ID's.
bool SoundTouch::setSetting(int settingId, int value)
{
int sampleRate, sequenceMs, seekWindowMs, overlapMs;
// read current tdstretch routine parameters
pTDStretch->getParameters(&sampleRate, &sequenceMs, &seekWindowMs, &overlapMs);
switch (settingId)
{
case SETTING_USE_AA_FILTER :
// enables / disabless anti-alias filter
pRateTransposer->enableAAFilter((value != 0) ? true : false);
return true;
case SETTING_AA_FILTER_LENGTH :
// sets anti-alias filter length
pRateTransposer->getAAFilter()->setLength(value);
return true;
case SETTING_USE_QUICKSEEK :
// enables / disables tempo routine quick seeking algorithm
pTDStretch->enableQuickSeek((value != 0) ? true : false);
return true;
case SETTING_SEQUENCE_MS:
// change time-stretch sequence duration parameter
pTDStretch->setParameters(sampleRate, value, seekWindowMs, overlapMs);
return true;
case SETTING_SEEKWINDOW_MS:
// change time-stretch seek window length parameter
pTDStretch->setParameters(sampleRate, sequenceMs, value, overlapMs);
return true;
case SETTING_OVERLAP_MS:
// change time-stretch overlap length parameter
pTDStretch->setParameters(sampleRate, sequenceMs, seekWindowMs, value);
return true;
default :
return false;
}
}
// Reads a setting controlling the processing system behaviour. See the
// 'SETTING_...' defines for available setting ID's.
//
// Returns the setting value.
int SoundTouch::getSetting(int settingId) const
{
int temp;
switch (settingId)
{
case SETTING_USE_AA_FILTER :
return (uint)pRateTransposer->isAAFilterEnabled();
case SETTING_AA_FILTER_LENGTH :
return pRateTransposer->getAAFilter()->getLength();
case SETTING_USE_QUICKSEEK :
return (uint)pTDStretch->isQuickSeekEnabled();
case SETTING_SEQUENCE_MS:
pTDStretch->getParameters(nullptr, &temp, nullptr, nullptr);
return temp;
case SETTING_SEEKWINDOW_MS:
pTDStretch->getParameters(nullptr, nullptr, &temp, nullptr);
return temp;
case SETTING_OVERLAP_MS:
pTDStretch->getParameters(nullptr, nullptr, nullptr, &temp);
return temp;
case SETTING_NOMINAL_INPUT_SEQUENCE :
{
int size = pTDStretch->getInputSampleReq();
#ifndef SOUNDTOUCH_PREVENT_CLICK_AT_RATE_CROSSOVER
if (rate <= 1.0)
{
// transposing done before timestretch, which impacts latency
return (int)(size * rate + 0.5);
}
#endif
return size;
}
case SETTING_NOMINAL_OUTPUT_SEQUENCE :
{
int size = pTDStretch->getOutputBatchSize();
if (rate > 1.0)
{
// transposing done after timestretch, which impacts latency
return (int)(size / rate + 0.5);
}
return size;
}
case SETTING_INITIAL_LATENCY:
{
double latency = pTDStretch->getLatency();
int latency_tr = pRateTransposer->getLatency();
#ifndef SOUNDTOUCH_PREVENT_CLICK_AT_RATE_CROSSOVER
if (rate <= 1.0)
{
// transposing done before timestretch, which impacts latency
latency = (latency + latency_tr) * rate;
}
else
#endif
{
latency += (double)latency_tr / rate;
}
return (int)(latency + 0.5);
}
default :
return 0;
}
}
// Clears all the samples in the object's output and internal processing
// buffers.
void SoundTouch::clear()
{
samplesExpectedOut = 0;
samplesOutput = 0;
pRateTransposer->clear();
pTDStretch->clear();
}
/// Returns number of samples currently unprocessed.
uint SoundTouch::numUnprocessedSamples() const
{
FIFOSamplePipe * psp;
if (pTDStretch)
{
psp = pTDStretch->getInput();
if (psp)
{
return psp->numSamples();
}
}
return 0;
}
/// Output samples from beginning of the sample buffer. Copies requested samples to
/// output buffer and removes them from the sample buffer. If there are less than
/// 'numsample' samples in the buffer, returns all that available.
///
/// \return Number of samples returned.
uint SoundTouch::receiveSamples(SAMPLETYPE *output, uint maxSamples)
{
uint ret = FIFOProcessor::receiveSamples(output, maxSamples);
samplesOutput += (long)ret;
return ret;
}
/// Adjusts book-keeping so that given number of samples are removed from beginning of the
/// sample buffer without copying them anywhere.
///
/// Used to reduce the number of samples in the buffer when accessing the sample buffer directly
/// with 'ptrBegin' function.
uint SoundTouch::receiveSamples(uint maxSamples)
{
uint ret = FIFOProcessor::receiveSamples(maxSamples);
samplesOutput += (long)ret;
return ret;
}
/// Get ratio between input and output audio durations, useful for calculating
/// processed output duration: if you'll process a stream of N samples, then
/// you can expect to get out N * getInputOutputSampleRatio() samples.
double SoundTouch::getInputOutputSampleRatio()
{
return 1.0 / (tempo * rate);
}
@@ -0,0 +1,29 @@
Microsoft Visual Studio Solution File, Format Version 12.00
# Visual Studio 14
VisualStudioVersion = 14.0.23107.0
MinimumVisualStudioVersion = 10.0.40219.1
Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "SoundTouch", "SoundTouch.vcxproj", "{68A5DD20-7057-448B-8FE0-B6AC8D205509}"
EndProject
Global
GlobalSection(SolutionConfigurationPlatforms) = preSolution
Debug|Win32 = Debug|Win32
Debug|x64 = Debug|x64
Release|Win32 = Release|Win32
Release|x64 = Release|x64
ReleaseX64|Win32 = ReleaseX64|Win32
ReleaseX64|x64 = ReleaseX64|x64
EndGlobalSection
GlobalSection(ProjectConfigurationPlatforms) = postSolution
{68A5DD20-7057-448B-8FE0-B6AC8D205509}.Debug|Win32.ActiveCfg = Debug|Win32
{68A5DD20-7057-448B-8FE0-B6AC8D205509}.Debug|x64.ActiveCfg = Debug|x64
{68A5DD20-7057-448B-8FE0-B6AC8D205509}.Release|Win32.ActiveCfg = Release|Win32
{68A5DD20-7057-448B-8FE0-B6AC8D205509}.Release|x64.ActiveCfg = Release|x64
{68A5DD20-7057-448B-8FE0-B6AC8D205509}.ReleaseX64|Win32.ActiveCfg = ReleaseX64|Win32
{68A5DD20-7057-448B-8FE0-B6AC8D205509}.ReleaseX64|Win32.Build.0 = ReleaseX64|Win32
{68A5DD20-7057-448B-8FE0-B6AC8D205509}.ReleaseX64|x64.ActiveCfg = ReleaseX64|x64
{68A5DD20-7057-448B-8FE0-B6AC8D205509}.ReleaseX64|x64.Build.0 = ReleaseX64|x64
EndGlobalSection
GlobalSection(SolutionProperties) = preSolution
HideSolutionNode = FALSE
EndGlobalSection
EndGlobal
@@ -0,0 +1,342 @@
<?xml version="1.0" encoding="utf-8"?>
<Project DefaultTargets="Build" ToolsVersion="14.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<ItemGroup Label="ProjectConfigurations">
<ProjectConfiguration Include="Debug|Win32">
<Configuration>Debug</Configuration>
<Platform>Win32</Platform>
</ProjectConfiguration>
<ProjectConfiguration Include="Debug|x64">
<Configuration>Debug</Configuration>
<Platform>x64</Platform>
</ProjectConfiguration>
<ProjectConfiguration Include="Release|Win32">
<Configuration>Release</Configuration>
<Platform>Win32</Platform>
</ProjectConfiguration>
<ProjectConfiguration Include="Release|x64">
<Configuration>Release</Configuration>
<Platform>x64</Platform>
</ProjectConfiguration>
</ItemGroup>
<PropertyGroup Label="Globals">
<ProjectGuid>{68A5DD20-7057-448B-8FE0-B6AC8D205509}</ProjectGuid>
<WindowsTargetPlatformVersion>10.0</WindowsTargetPlatformVersion>
</PropertyGroup>
<Import Project="$(VCTargetsPath)\Microsoft.Cpp.Default.props" />
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'" Label="Configuration">
<ConfigurationType>StaticLibrary</ConfigurationType>
<PlatformToolset>v142</PlatformToolset>
<UseOfMfc>false</UseOfMfc>
<CharacterSet>Unicode</CharacterSet>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|Win32'" Label="Configuration">
<ConfigurationType>StaticLibrary</ConfigurationType>
<PlatformToolset>v142</PlatformToolset>
<UseOfMfc>false</UseOfMfc>
<CharacterSet>Unicode</CharacterSet>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|x64'" Label="Configuration">
<ConfigurationType>StaticLibrary</ConfigurationType>
<PlatformToolset>v142</PlatformToolset>
<UseOfMfc>false</UseOfMfc>
<CharacterSet>Unicode</CharacterSet>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|x64'" Label="Configuration">
<ConfigurationType>StaticLibrary</ConfigurationType>
<PlatformToolset>v142</PlatformToolset>
<UseOfMfc>false</UseOfMfc>
<CharacterSet>Unicode</CharacterSet>
</PropertyGroup>
<Import Project="$(VCTargetsPath)\Microsoft.Cpp.props" />
<ImportGroup Label="ExtensionSettings">
</ImportGroup>
<ImportGroup Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'" Label="PropertySheets">
<Import Project="$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props" Condition="exists('$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props')" Label="LocalAppDataPlatform" />
<Import Project="$(VCTargetsPath)Microsoft.CPP.UpgradeFromVC71.props" />
</ImportGroup>
<ImportGroup Condition="'$(Configuration)|$(Platform)'=='Release|Win32'" Label="PropertySheets">
<Import Project="$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props" Condition="exists('$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props')" Label="LocalAppDataPlatform" />
<Import Project="$(VCTargetsPath)Microsoft.CPP.UpgradeFromVC71.props" />
</ImportGroup>
<ImportGroup Condition="'$(Configuration)|$(Platform)'=='Debug|x64'" Label="PropertySheets">
<Import Project="$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props" Condition="exists('$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props')" Label="LocalAppDataPlatform" />
<Import Project="$(VCTargetsPath)Microsoft.CPP.UpgradeFromVC71.props" />
</ImportGroup>
<ImportGroup Condition="'$(Configuration)|$(Platform)'=='Release|x64'" Label="PropertySheets">
<Import Project="$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props" Condition="exists('$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props')" Label="LocalAppDataPlatform" />
<Import Project="$(VCTargetsPath)Microsoft.CPP.UpgradeFromVC71.props" />
</ImportGroup>
<PropertyGroup Label="UserMacros" />
<PropertyGroup>
<_ProjectFileVersion>14.0.23107.0</_ProjectFileVersion>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|Win32'">
<OutDir>$(Platform)\$(Configuration)\</OutDir>
<IntDir>$(Platform)\$(Configuration)\</IntDir>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|x64'">
<OutDir>$(Platform)\$(Configuration)\</OutDir>
<IntDir>$(Platform)\$(Configuration)\</IntDir>
<TargetName>$(ProjectName)_x64</TargetName>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'">
<OutDir>$(Platform)\$(Configuration)\</OutDir>
<IntDir>$(Platform)\$(Configuration)\</IntDir>
<TargetName>$(ProjectName)D</TargetName>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|x64'">
<OutDir>$(Platform)\$(Configuration)\</OutDir>
<IntDir>$(Platform)\$(Configuration)\</IntDir>
<TargetName>$(ProjectName)D_x64</TargetName>
</PropertyGroup>
<ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Release|Win32'">
<ClCompile>
<Optimization>Full</Optimization>
<InlineFunctionExpansion>AnySuitable</InlineFunctionExpansion>
<IntrinsicFunctions>true</IntrinsicFunctions>
<AdditionalIncludeDirectories>..\..\include;%(AdditionalIncludeDirectories)</AdditionalIncludeDirectories>
<PreprocessorDefinitions>WIN32;NDEBUG;_LIB;%(PreprocessorDefinitions)</PreprocessorDefinitions>
<StringPooling>true</StringPooling>
<RuntimeLibrary>MultiThreaded</RuntimeLibrary>
<FunctionLevelLinking>true</FunctionLevelLinking>
<FloatingPointModel>Fast</FloatingPointModel>
<PrecompiledHeader />
<PrecompiledHeaderOutputFile>$(OutDir)$(TargetName).pch</PrecompiledHeaderOutputFile>
<AssemblerListingLocation>$(OutDir)</AssemblerListingLocation>
<ObjectFileName>$(OutDir)</ObjectFileName>
<ProgramDataBaseFileName>$(OutDir)</ProgramDataBaseFileName>
<WarningLevel>Level3</WarningLevel>
<SuppressStartupBanner>true</SuppressStartupBanner>
<DebugInformationFormat />
<CompileAs>Default</CompileAs>
<EnableEnhancedInstructionSet>StreamingSIMDExtensions2</EnableEnhancedInstructionSet>
<XMLDocumentationFileName>$(IntDir)</XMLDocumentationFileName>
<BrowseInformationFile>$(IntDir)</BrowseInformationFile>
<MultiProcessorCompilation>true</MultiProcessorCompilation>
</ClCompile>
<ResourceCompile>
<PreprocessorDefinitions>NDEBUG;%(PreprocessorDefinitions)</PreprocessorDefinitions>
<Culture>0x040b</Culture>
</ResourceCompile>
<Lib>
<OutputFile>$(OutDir)$(TargetName)$(TargetExt)</OutputFile>
<SuppressStartupBanner>true</SuppressStartupBanner>
</Lib>
<PostBuildEvent>
<Command>if not exist ..\..\lib mkdir ..\..\lib
copy $(OutDir)$(TargetName)$(TargetExt) ..\..\lib</Command>
</PostBuildEvent>
</ItemDefinitionGroup>
<ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Release|x64'">
<Midl>
<TargetEnvironment>X64</TargetEnvironment>
</Midl>
<ClCompile>
<Optimization>Full</Optimization>
<InlineFunctionExpansion>AnySuitable</InlineFunctionExpansion>
<IntrinsicFunctions>true</IntrinsicFunctions>
<AdditionalIncludeDirectories>..\..\include;%(AdditionalIncludeDirectories)</AdditionalIncludeDirectories>
<PreprocessorDefinitions>WIN32;NDEBUG;_LIB;%(PreprocessorDefinitions)</PreprocessorDefinitions>
<StringPooling>true</StringPooling>
<RuntimeLibrary>MultiThreaded</RuntimeLibrary>
<FunctionLevelLinking>true</FunctionLevelLinking>
<FloatingPointModel>Fast</FloatingPointModel>
<PrecompiledHeader />
<PrecompiledHeaderOutputFile>$(OutDir)$(TargetName).pch</PrecompiledHeaderOutputFile>
<AssemblerListingLocation>$(OutDir)</AssemblerListingLocation>
<ObjectFileName>$(OutDir)</ObjectFileName>
<ProgramDataBaseFileName>$(OutDir)</ProgramDataBaseFileName>
<WarningLevel>Level3</WarningLevel>
<SuppressStartupBanner>true</SuppressStartupBanner>
<DebugInformationFormat />
<CompileAs>Default</CompileAs>
<EnableEnhancedInstructionSet>
</EnableEnhancedInstructionSet>
<XMLDocumentationFileName>$(IntDir)</XMLDocumentationFileName>
<BrowseInformationFile>$(IntDir)</BrowseInformationFile>
<MultiProcessorCompilation>true</MultiProcessorCompilation>
</ClCompile>
<ResourceCompile>
<PreprocessorDefinitions>NDEBUG;%(PreprocessorDefinitions)</PreprocessorDefinitions>
<Culture>0x040b</Culture>
</ResourceCompile>
<Lib>
<OutputFile>$(OutDir)$(TargetName)$(TargetExt)</OutputFile>
<SuppressStartupBanner>true</SuppressStartupBanner>
</Lib>
<PostBuildEvent>
<Command>if not exist ..\..\lib mkdir ..\..\lib
copy $(OutDir)$(TargetName)$(TargetExt) ..\..\lib</Command>
</PostBuildEvent>
</ItemDefinitionGroup>
<ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'">
<ClCompile>
<Optimization>Disabled</Optimization>
<AdditionalIncludeDirectories>..\..\include;%(AdditionalIncludeDirectories)</AdditionalIncludeDirectories>
<PreprocessorDefinitions>WIN32;_DEBUG;_LIB;%(PreprocessorDefinitions)</PreprocessorDefinitions>
<BasicRuntimeChecks>EnableFastChecks</BasicRuntimeChecks>
<RuntimeLibrary>MultiThreadedDebug</RuntimeLibrary>
<FloatingPointModel>Fast</FloatingPointModel>
<PrecompiledHeader />
<PrecompiledHeaderOutputFile>$(OutDir)$(TargetName).pch</PrecompiledHeaderOutputFile>
<AssemblerListingLocation>$(OutDir)</AssemblerListingLocation>
<ObjectFileName>$(OutDir)</ObjectFileName>
<ProgramDataBaseFileName>$(OutDir)</ProgramDataBaseFileName>
<BrowseInformation>true</BrowseInformation>
<WarningLevel>Level3</WarningLevel>
<SuppressStartupBanner>true</SuppressStartupBanner>
<DebugInformationFormat>ProgramDatabase</DebugInformationFormat>
<CompileAs>Default</CompileAs>
<EnableEnhancedInstructionSet>StreamingSIMDExtensions2</EnableEnhancedInstructionSet>
<XMLDocumentationFileName>$(IntDir)</XMLDocumentationFileName>
<BrowseInformationFile>$(IntDir)</BrowseInformationFile>
<MultiProcessorCompilation>true</MultiProcessorCompilation>
</ClCompile>
<ResourceCompile>
<PreprocessorDefinitions>_DEBUG;%(PreprocessorDefinitions)</PreprocessorDefinitions>
<Culture>0x040b</Culture>
</ResourceCompile>
<Lib>
<OutputFile>$(OutDir)$(TargetName)$(TargetExt)</OutputFile>
<SuppressStartupBanner>true</SuppressStartupBanner>
</Lib>
<PostBuildEvent>
<Command>if not exist ..\..\lib mkdir ..\..\lib
copy $(OutDir)$(TargetName)$(TargetExt) ..\..\lib</Command>
</PostBuildEvent>
</ItemDefinitionGroup>
<ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Debug|x64'">
<Midl>
<TargetEnvironment>X64</TargetEnvironment>
</Midl>
<ClCompile>
<Optimization>Disabled</Optimization>
<AdditionalIncludeDirectories>..\..\include;%(AdditionalIncludeDirectories)</AdditionalIncludeDirectories>
<PreprocessorDefinitions>WIN32;_DEBUG;_LIB;%(PreprocessorDefinitions)</PreprocessorDefinitions>
<BasicRuntimeChecks>EnableFastChecks</BasicRuntimeChecks>
<RuntimeLibrary>MultiThreadedDebug</RuntimeLibrary>
<FloatingPointModel>Fast</FloatingPointModel>
<PrecompiledHeader />
<PrecompiledHeaderOutputFile>$(OutDir)$(TargetName).pch</PrecompiledHeaderOutputFile>
<AssemblerListingLocation>$(OutDir)</AssemblerListingLocation>
<ObjectFileName>$(OutDir)</ObjectFileName>
<ProgramDataBaseFileName>$(OutDir)</ProgramDataBaseFileName>
<BrowseInformation>true</BrowseInformation>
<WarningLevel>Level3</WarningLevel>
<SuppressStartupBanner>true</SuppressStartupBanner>
<DebugInformationFormat>ProgramDatabase</DebugInformationFormat>
<CompileAs>Default</CompileAs>
<EnableEnhancedInstructionSet>
</EnableEnhancedInstructionSet>
<XMLDocumentationFileName>$(IntDir)</XMLDocumentationFileName>
<BrowseInformationFile>$(IntDir)</BrowseInformationFile>
<MultiProcessorCompilation>true</MultiProcessorCompilation>
</ClCompile>
<ResourceCompile>
<PreprocessorDefinitions>_DEBUG;%(PreprocessorDefinitions)</PreprocessorDefinitions>
<Culture>0x040b</Culture>
</ResourceCompile>
<Lib>
<OutputFile>$(OutDir)$(TargetName)$(TargetExt)</OutputFile>
<SuppressStartupBanner>true</SuppressStartupBanner>
</Lib>
<PostBuildEvent>
<Command>if not exist ..\..\lib mkdir ..\..\lib
copy $(OutDir)$(TargetName)$(TargetExt) ..\..\lib</Command>
</PostBuildEvent>
</ItemDefinitionGroup>
<ItemGroup>
<ClCompile Include="AAFilter.cpp">
<Optimization Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'">Disabled</Optimization>
<BasicRuntimeChecks Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'">EnableFastChecks</BasicRuntimeChecks>
<BrowseInformation Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'">true</BrowseInformation>
<Optimization Condition="'$(Configuration)|$(Platform)'=='Debug|x64'">Disabled</Optimization>
<BasicRuntimeChecks Condition="'$(Configuration)|$(Platform)'=='Debug|x64'">EnableFastChecks</BasicRuntimeChecks>
<BrowseInformation Condition="'$(Configuration)|$(Platform)'=='Debug|x64'">true</BrowseInformation>
<Optimization Condition="'$(Configuration)|$(Platform)'=='Release|Win32'">MaxSpeed</Optimization>
<Optimization Condition="'$(Configuration)|$(Platform)'=='Release|x64'">MaxSpeed</Optimization>
</ClCompile>
<ClCompile Include="BPMDetect.cpp">
<DisableSpecificWarnings Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'">4996</DisableSpecificWarnings>
<DisableSpecificWarnings Condition="'$(Configuration)|$(Platform)'=='Release|Win32'">4996</DisableSpecificWarnings>
<DisableSpecificWarnings Condition="'$(Configuration)|$(Platform)'=='Debug|x64'">4996</DisableSpecificWarnings>
<DisableSpecificWarnings Condition="'$(Configuration)|$(Platform)'=='Release|x64'">4996</DisableSpecificWarnings>
</ClCompile>
<ClCompile Include="cpu_detect_x86.cpp" />
<ClCompile Include="FIFOSampleBuffer.cpp">
<Optimization Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'">Disabled</Optimization>
<BasicRuntimeChecks Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'">EnableFastChecks</BasicRuntimeChecks>
<BrowseInformation Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'">true</BrowseInformation>
<Optimization Condition="'$(Configuration)|$(Platform)'=='Debug|x64'">Disabled</Optimization>
<BasicRuntimeChecks Condition="'$(Configuration)|$(Platform)'=='Debug|x64'">EnableFastChecks</BasicRuntimeChecks>
<BrowseInformation Condition="'$(Configuration)|$(Platform)'=='Debug|x64'">true</BrowseInformation>
<Optimization Condition="'$(Configuration)|$(Platform)'=='Release|Win32'">MaxSpeed</Optimization>
<Optimization Condition="'$(Configuration)|$(Platform)'=='Release|x64'">MaxSpeed</Optimization>
</ClCompile>
<ClCompile Include="FIRFilter.cpp">
<Optimization Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'">Disabled</Optimization>
<BasicRuntimeChecks Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'">EnableFastChecks</BasicRuntimeChecks>
<BrowseInformation Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'">true</BrowseInformation>
<Optimization Condition="'$(Configuration)|$(Platform)'=='Debug|x64'">Disabled</Optimization>
<BasicRuntimeChecks Condition="'$(Configuration)|$(Platform)'=='Debug|x64'">EnableFastChecks</BasicRuntimeChecks>
<BrowseInformation Condition="'$(Configuration)|$(Platform)'=='Debug|x64'">true</BrowseInformation>
<Optimization Condition="'$(Configuration)|$(Platform)'=='Release|Win32'">MaxSpeed</Optimization>
<Optimization Condition="'$(Configuration)|$(Platform)'=='Release|x64'">MaxSpeed</Optimization>
</ClCompile>
<ClCompile Include="InterpolateCubic.cpp" />
<ClCompile Include="InterpolateLinear.cpp" />
<ClCompile Include="InterpolateShannon.cpp" />
<ClCompile Include="mmx_optimized.cpp" />
<ClCompile Include="PeakFinder.cpp" />
<ClCompile Include="RateTransposer.cpp">
<Optimization Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'">Disabled</Optimization>
<BasicRuntimeChecks Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'">EnableFastChecks</BasicRuntimeChecks>
<BrowseInformation Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'">true</BrowseInformation>
<Optimization Condition="'$(Configuration)|$(Platform)'=='Debug|x64'">Disabled</Optimization>
<BasicRuntimeChecks Condition="'$(Configuration)|$(Platform)'=='Debug|x64'">EnableFastChecks</BasicRuntimeChecks>
<BrowseInformation Condition="'$(Configuration)|$(Platform)'=='Debug|x64'">true</BrowseInformation>
<Optimization Condition="'$(Configuration)|$(Platform)'=='Release|Win32'">MaxSpeed</Optimization>
<Optimization Condition="'$(Configuration)|$(Platform)'=='Release|x64'">MaxSpeed</Optimization>
</ClCompile>
<ClCompile Include="SoundTouch.cpp">
<Optimization Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'">Disabled</Optimization>
<BasicRuntimeChecks Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'">EnableFastChecks</BasicRuntimeChecks>
<BrowseInformation Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'">true</BrowseInformation>
<Optimization Condition="'$(Configuration)|$(Platform)'=='Debug|x64'">Disabled</Optimization>
<BasicRuntimeChecks Condition="'$(Configuration)|$(Platform)'=='Debug|x64'">EnableFastChecks</BasicRuntimeChecks>
<BrowseInformation Condition="'$(Configuration)|$(Platform)'=='Debug|x64'">true</BrowseInformation>
<Optimization Condition="'$(Configuration)|$(Platform)'=='Release|Win32'">MaxSpeed</Optimization>
<Optimization Condition="'$(Configuration)|$(Platform)'=='Release|x64'">MaxSpeed</Optimization>
</ClCompile>
<ClCompile Include="sse_optimized.cpp" />
<ClCompile Include="TDStretch.cpp">
<Optimization Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'">Disabled</Optimization>
<BasicRuntimeChecks Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'">EnableFastChecks</BasicRuntimeChecks>
<BrowseInformation Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'">true</BrowseInformation>
<Optimization Condition="'$(Configuration)|$(Platform)'=='Debug|x64'">Disabled</Optimization>
<BasicRuntimeChecks Condition="'$(Configuration)|$(Platform)'=='Debug|x64'">EnableFastChecks</BasicRuntimeChecks>
<BrowseInformation Condition="'$(Configuration)|$(Platform)'=='Debug|x64'">true</BrowseInformation>
<Optimization Condition="'$(Configuration)|$(Platform)'=='Release|Win32'">MaxSpeed</Optimization>
<Optimization Condition="'$(Configuration)|$(Platform)'=='Release|x64'">MaxSpeed</Optimization>
</ClCompile>
</ItemGroup>
<ItemGroup>
<ClInclude Include="..\..\include\BPMDetect.h" />
<ClInclude Include="..\..\include\FIFOSampleBuffer.h" />
<ClInclude Include="..\..\include\FIFOSamplePipe.h" />
<ClInclude Include="..\..\include\SoundTouch.h" />
<ClInclude Include="..\..\include\STTypes.h" />
<ClInclude Include="AAFilter.h" />
<ClInclude Include="cpu_detect.h" />
<ClInclude Include="FIRFilter.h" />
<ClInclude Include="InterpolateCubic.h" />
<ClInclude Include="InterpolateLinear.h" />
<ClInclude Include="InterpolateShannon.h" />
<ClInclude Include="PeakFinder.h" />
<ClInclude Include="RateTransposer.h" />
<ClInclude Include="TDStretch.h" />
</ItemGroup>
<Import Project="$(VCTargetsPath)\Microsoft.Cpp.targets" />
<ImportGroup Label="ExtensionTargets">
</ImportGroup>
</Project>
File diff suppressed because it is too large. Load diff
@@ -0,0 +1,279 @@
////////////////////////////////////////////////////////////////////////////////
///
/// Sampled sound tempo changer/time stretch algorithm. Changes the sound tempo
/// while maintaining the original pitch by using a time domain WSOLA-like method
/// with several performance-increasing tweaks.
///
/// Note : MMX/SSE optimized functions reside in separate, platform-specific files
/// 'mmx_optimized.cpp' and 'sse_optimized.cpp'
///
/// Author : Copyright (c) Olli Parviainen
/// Author e-mail : oparviai 'at' iki.fi
/// SoundTouch WWW: http://www.surina.net/soundtouch
///
////////////////////////////////////////////////////////////////////////////////
//
// License :
//
// SoundTouch audio processing library
// Copyright (c) Olli Parviainen
//
// This library is free software; you can redistribute it and/or
// modify it under the terms of the GNU Lesser General Public
// License as published by the Free Software Foundation; either
// version 2.1 of the License, or (at your option) any later version.
//
// This library is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
// Lesser General Public License for more details.
//
// You should have received a copy of the GNU Lesser General Public
// License along with this library; if not, write to the Free Software
// Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
//
////////////////////////////////////////////////////////////////////////////////
#ifndef TDStretch_H
#define TDStretch_H
#include <stddef.h>
#include "STTypes.h"
#include "RateTransposer.h"
#include "FIFOSamplePipe.h"
namespace soundtouch
{
/// Default values for sound processing parameters:
/// Notice that the default parameters are tuned for contemporary popular music
/// processing. For speech processing applications these parameters suit better:
/// #define DEFAULT_SEQUENCE_MS 40
/// #define DEFAULT_SEEKWINDOW_MS 15
/// #define DEFAULT_OVERLAP_MS 8
///
/// Default length of a single processing sequence, in milliseconds. This determines to how
/// long sequences the original sound is chopped in the time-stretch algorithm.
///
/// The larger this value is, the lesser sequences are used in processing. In principle
/// a bigger value sounds better when slowing down tempo, but worse when increasing tempo
/// and vice versa.
///
/// Increasing this value reduces computational burden & vice versa.
//#define DEFAULT_SEQUENCE_MS 40
#define DEFAULT_SEQUENCE_MS USE_AUTO_SEQUENCE_LEN
/// Giving this value for the sequence length sets automatic parameter value
/// according to tempo setting (recommended)
#define USE_AUTO_SEQUENCE_LEN 0
/// Seeking window default length in milliseconds for algorithm that finds the best possible
/// overlapping location. This determines from how wide window the algorithm may look for an
/// optimal joining location when mixing the sound sequences back together.
///
/// The bigger this window setting is, the higher the possibility to find a better mixing
/// position will become, but at the same time large values may cause a "drifting" artifact
/// because consequent sequences will be taken at more uneven intervals.
///
/// If there's a disturbing artifact that sounds as if a constant frequency was drifting
/// around, try reducing this setting.
///
/// Increasing this value increases computational burden & vice versa.
//#define DEFAULT_SEEKWINDOW_MS 15
#define DEFAULT_SEEKWINDOW_MS USE_AUTO_SEEKWINDOW_LEN
/// Giving this value for the seek window length sets automatic parameter value
/// according to tempo setting (recommended)
#define USE_AUTO_SEEKWINDOW_LEN 0
/// Overlap length in milliseconds. When the chopped sound sequences are mixed back together,
/// to form a continuous sound stream, this parameter defines over how long period the two
/// consecutive sequences are let to overlap each other.
///
/// This shouldn't be that critical parameter. If you reduce the DEFAULT_SEQUENCE_MS setting
/// by a large amount, you might wish to try a smaller value on this.
///
/// Increasing this value increases computational burden & vice versa.
#define DEFAULT_OVERLAP_MS 8
/// Class that does the time-stretch (tempo change) effect for the processed
/// sound.
class TDStretch : public FIFOProcessor
{
protected:
int channels;
int sampleReq;
int overlapLength;
int seekLength;
int seekWindowLength;
int overlapDividerBitsNorm;
int overlapDividerBitsPure;
int slopingDivider;
int sampleRate;
int sequenceMs;
int seekWindowMs;
int overlapMs;
unsigned long maxnorm;
float maxnormf;
double tempo;
double nominalSkip;
double skipFract;
bool bQuickSeek;
bool bAutoSeqSetting;
bool bAutoSeekSetting;
bool isBeginning;
SAMPLETYPE *pMidBuffer;
SAMPLETYPE *pMidBufferUnaligned;
FIFOSampleBuffer outputBuffer;
FIFOSampleBuffer inputBuffer;
void acceptNewOverlapLength(int newOverlapLength);
virtual void clearCrossCorrState();
void calculateOverlapLength(int overlapMs);
virtual double calcCrossCorr(const SAMPLETYPE *mixingPos, const SAMPLETYPE *compare, double &norm);
virtual double calcCrossCorrAccumulate(const SAMPLETYPE *mixingPos, const SAMPLETYPE *compare, double &norm);
virtual int seekBestOverlapPositionFull(const SAMPLETYPE *refPos);
virtual int seekBestOverlapPositionQuick(const SAMPLETYPE *refPos);
virtual int seekBestOverlapPosition(const SAMPLETYPE *refPos);
virtual void overlapStereo(SAMPLETYPE *output, const SAMPLETYPE *input) const;
virtual void overlapMono(SAMPLETYPE *output, const SAMPLETYPE *input) const;
virtual void overlapMulti(SAMPLETYPE *output, const SAMPLETYPE *input) const;
void clearMidBuffer();
void overlap(SAMPLETYPE *output, const SAMPLETYPE *input, uint ovlPos) const;
void calcSeqParameters();
void adaptNormalizer();
/// Changes the tempo of the given sound samples.
/// Returns amount of samples returned in the "output" buffer.
/// The maximum amount of samples that can be returned at a time is set by
/// the 'set_returnBuffer_size' function.
void processSamples();
public:
TDStretch();
virtual ~TDStretch() override;
/// Operator 'new' is overloaded so that it automatically creates a suitable instance
/// depending on if we've a MMX/SSE/etc-capable CPU available or not.
static void *operator new(size_t s);
/// Use this function instead of "new" operator to create a new instance of this class.
/// This function automatically chooses a correct feature set depending on if the CPU
/// supports MMX/SSE/etc extensions.
static TDStretch *newInstance();
/// Returns the output buffer object
FIFOSamplePipe *getOutput() { return &outputBuffer; };
/// Returns the input buffer object
FIFOSamplePipe *getInput() { return &inputBuffer; };
/// Sets new target tempo. Normal tempo = 'SCALE', smaller values represent slower
/// tempo, larger faster tempo.
void setTempo(double newTempo);
/// Returns nonzero if there aren't any samples available for outputting.
virtual void clear() override;
/// Clears the input buffer
void clearInput();
/// Sets the number of channels, 1 = mono, 2 = stereo
void setChannels(int numChannels);
/// Enables/disables the quick position seeking algorithm. Zero to disable,
/// nonzero to enable
void enableQuickSeek(bool enable);
/// Returns nonzero if the quick seeking algorithm is enabled.
bool isQuickSeekEnabled() const;
/// Sets routine control parameters. These control are certain time constants
/// defining how the sound is stretched to the desired duration.
//
/// 'sampleRate' = sample rate of the sound
/// 'sequenceMS' = one processing sequence length in milliseconds
/// 'seekwindowMS' = seeking window length for scanning the best overlapping
/// position
/// 'overlapMS' = overlapping length
void setParameters(int sampleRate, ///< Samplerate of sound being processed (Hz)
int sequenceMS = -1, ///< Single processing sequence length (ms)
int seekwindowMS = -1, ///< Offset seeking window length (ms)
int overlapMS = -1 ///< Sequence overlapping length (ms)
);
/// Get routine control parameters, see setParameters() function.
/// Any of the parameters to this function can be nullptr, in such case corresponding parameter
/// value isn't returned.
void getParameters(int *pSampleRate, int *pSequenceMs, int *pSeekWindowMs, int *pOverlapMs) const;
/// Adds 'numsamples' pcs of samples from the 'samples' memory position into
/// the input of the object.
virtual void putSamples(
const SAMPLETYPE *samples, ///< Input sample data
uint numSamples ///< Number of samples in 'samples' so that one sample
///< contains both channels if stereo
) override;
/// return nominal input sample requirement for triggering a processing batch
int getInputSampleReq() const
{
return (int)(nominalSkip + 0.5);
}
/// return nominal output sample amount when running a processing batch
int getOutputBatchSize() const
{
return seekWindowLength - overlapLength;
}
/// return approximate initial input-output latency
int getLatency() const
{
return sampleReq;
}
};
// Implementation-specific class declarations:
#ifdef SOUNDTOUCH_ALLOW_MMX
/// Class that implements MMX optimized routines for 16bit integer samples type.
class TDStretchMMX : public TDStretch
{
protected:
double calcCrossCorr(const short *mixingPos, const short *compare, double &norm) override;
double calcCrossCorrAccumulate(const short *mixingPos, const short *compare, double &norm) override;
virtual void overlapStereo(short *output, const short *input) const override;
virtual void clearCrossCorrState() override;
};
#endif /// SOUNDTOUCH_ALLOW_MMX
#ifdef SOUNDTOUCH_ALLOW_SSE
/// Class that implements SSE optimized routines for floating point samples type.
class TDStretchSSE : public TDStretch
{
protected:
double calcCrossCorr(const float *mixingPos, const float *compare, double &norm) override;
double calcCrossCorrAccumulate(const float *mixingPos, const float *compare, double &norm) override;
};
#endif /// SOUNDTOUCH_ALLOW_SSE
}
#endif /// TDStretch_H
@@ -0,0 +1,55 @@
////////////////////////////////////////////////////////////////////////////////
///
/// A header file for detecting the Intel MMX instructions set extension.
///
/// Please see 'mmx_win.cpp', 'mmx_cpp.cpp' and 'mmx_non_x86.cpp' for the
/// routine implementations for x86 Windows, x86 gnu version and non-x86
/// platforms, respectively.
///
/// Author : Copyright (c) Olli Parviainen
/// Author e-mail : oparviai 'at' iki.fi
/// SoundTouch WWW: http://www.surina.net/soundtouch
///
////////////////////////////////////////////////////////////////////////////////
//
// License :
//
// SoundTouch audio processing library
// Copyright (c) Olli Parviainen
//
// This library is free software; you can redistribute it and/or
// modify it under the terms of the GNU Lesser General Public
// License as published by the Free Software Foundation; either
// version 2.1 of the License, or (at your option) any later version.
//
// This library is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
// Lesser General Public License for more details.
//
// You should have received a copy of the GNU Lesser General Public
// License along with this library; if not, write to the Free Software
// Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
//
////////////////////////////////////////////////////////////////////////////////
#ifndef _CPU_DETECT_H_
#define _CPU_DETECT_H_
#include "STTypes.h"
#define SUPPORT_MMX 0x0001
#define SUPPORT_3DNOW 0x0002
#define SUPPORT_ALTIVEC 0x0004
#define SUPPORT_SSE 0x0008
#define SUPPORT_SSE2 0x0010
/// Checks which instruction set extensions are supported by the CPU.
///
/// \return A bitmask of supported extensions, see SUPPORT_... defines.
uint detectCPUextensions(void);
/// Disables given set of instruction extensions. See SUPPORT_... defines.
void disableExtensions(uint wDisableMask);
#endif // _CPU_DETECT_H_
@@ -0,0 +1,130 @@
////////////////////////////////////////////////////////////////////////////////
///
/// Generic version of the x86 CPU extension detection routine.
///
/// This file is for GNU & other non-Windows compilers, see 'cpu_detect_x86_win.cpp'
/// for the Microsoft compiler version.
///
/// Author : Copyright (c) Olli Parviainen
/// Author e-mail : oparviai 'at' iki.fi
/// SoundTouch WWW: http://www.surina.net/soundtouch
///
////////////////////////////////////////////////////////////////////////////////
//
// License :
//
// SoundTouch audio processing library
// Copyright (c) Olli Parviainen
//
// This library is free software; you can redistribute it and/or
// modify it under the terms of the GNU Lesser General Public
// License as published by the Free Software Foundation; either
// version 2.1 of the License, or (at your option) any later version.
//
// This library is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
// Lesser General Public License for more details.
//
// You should have received a copy of the GNU Lesser General Public
// License along with this library; if not, write to the Free Software
// Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
//
////////////////////////////////////////////////////////////////////////////////
#include "cpu_detect.h"
#include "STTypes.h"
#if defined(SOUNDTOUCH_ALLOW_X86_OPTIMIZATIONS)
#if defined(__GNUC__) && defined(__i386__)
// gcc
#include "cpuid.h"
#elif defined(_M_IX86)
// windows non-gcc
#include <intrin.h>
#endif
#define bit_MMX (1 << 23)
#define bit_SSE (1 << 25)
#define bit_SSE2 (1 << 26)
#endif
//////////////////////////////////////////////////////////////////////////////
//
// processor instructions extension detection routines
//
//////////////////////////////////////////////////////////////////////////////
// Flag variable indicating whick ISA extensions are disabled (for debugging)
static uint _dwDisabledISA = 0x00; // 0xffffffff; //<- use this to disable all extensions
// Disables given set of instruction extensions. See SUPPORT_... defines.
void disableExtensions(uint dwDisableMask)
{
_dwDisabledISA = dwDisableMask;
}
/// Checks which instruction set extensions are supported by the CPU.
uint detectCPUextensions(void)
{
/// If building for a 64bit system (no Itanium) and the user wants optimizations.
/// Return the OR of SUPPORT_{MMX,SSE,SSE2}. 11001 or 0x19.
/// Keep the _dwDisabledISA test (2 more operations, could be eliminated).
#if ((defined(__GNUC__) && defined(__x86_64__)) \
|| defined(_M_X64)) \
&& defined(SOUNDTOUCH_ALLOW_X86_OPTIMIZATIONS)
return 0x19 & ~_dwDisabledISA;
/// If building for a 32bit system and the user wants optimizations.
/// Keep the _dwDisabledISA test (2 more operations, could be eliminated).
#elif ((defined(__GNUC__) && defined(__i386__)) \
|| defined(_M_IX86)) \
&& defined(SOUNDTOUCH_ALLOW_X86_OPTIMIZATIONS)
if (_dwDisabledISA == 0xffffffff) return 0;
uint res = 0;
#if defined(__GNUC__)
// GCC version of cpuid. Requires GCC 4.3.0 or later for __cpuid intrinsic support.
uint eax, ebx, ecx, edx; // unsigned int is the standard type. uint is defined by the compiler and not guaranteed to be portable.
// Check if no cpuid support.
if (!__get_cpuid (1, &eax, &ebx, &ecx, &edx)) return 0; // always disable extensions.
if (edx & bit_MMX) res = res | SUPPORT_MMX;
if (edx & bit_SSE) res = res | SUPPORT_SSE;
if (edx & bit_SSE2) res = res | SUPPORT_SSE2;
#else
// Window / VS version of cpuid. Notice that Visual Studio 2005 or later required
// for __cpuid intrinsic support.
int reg[4] = {-1};
// Check if no cpuid support.
__cpuid(reg,0);
if ((unsigned int)reg[0] == 0) return 0; // always disable extensions.
__cpuid(reg,1);
if ((unsigned int)reg[3] & bit_MMX) res = res | SUPPORT_MMX;
if ((unsigned int)reg[3] & bit_SSE) res = res | SUPPORT_SSE;
if ((unsigned int)reg[3] & bit_SSE2) res = res | SUPPORT_SSE2;
#endif
return res & ~_dwDisabledISA;
#else
/// One of these is true:
/// 1) We don't want optimizations.
/// 2) Using an unsupported compiler.
/// 3) Running on a non-x86 platform.
return 0;
#endif
}
@@ -0,0 +1,392 @@
////////////////////////////////////////////////////////////////////////////////
///
/// MMX optimized routines. All MMX optimized functions have been gathered into
/// this single source code file, regardless to their class or original source
/// code file, in order to ease porting the library to other compiler and
/// processor platforms.
///
/// The MMX-optimizations are programmed using MMX compiler intrinsics that
/// are supported both by Microsoft Visual C++ and GCC compilers, so this file
/// should compile with both toolsets.
///
/// NOTICE: If using Visual Studio 6.0, you'll need to install the "Visual C++
/// 6.0 processor pack" update to support compiler intrinsic syntax. The update
/// is available for download at Microsoft Developers Network, see here:
/// http://msdn.microsoft.com/en-us/vstudio/aa718349.aspx
///
/// Author : Copyright (c) Olli Parviainen
/// Author e-mail : oparviai 'at' iki.fi
/// SoundTouch WWW: http://www.surina.net/soundtouch
///
////////////////////////////////////////////////////////////////////////////////
//
// License :
//
// SoundTouch audio processing library
// Copyright (c) Olli Parviainen
//
// This library is free software; you can redistribute it and/or
// modify it under the terms of the GNU Lesser General Public
// License as published by the Free Software Foundation; either
// version 2.1 of the License, or (at your option) any later version.
//
// This library is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
// Lesser General Public License for more details.
//
// You should have received a copy of the GNU Lesser General Public
// License along with this library; if not, write to the Free Software
// Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
//
////////////////////////////////////////////////////////////////////////////////
#include "STTypes.h"
#ifdef SOUNDTOUCH_ALLOW_MMX
// MMX routines available only with integer sample type
using namespace soundtouch;
//////////////////////////////////////////////////////////////////////////////
//
// implementation of MMX optimized functions of class 'TDStretchMMX'
//
//////////////////////////////////////////////////////////////////////////////
#include "TDStretch.h"
#include <mmintrin.h>
#include <limits.h>
#include <math.h>
// Calculates cross correlation of two buffers
double TDStretchMMX::calcCrossCorr(const short *pV1, const short *pV2, double &dnorm)
{
const __m64 *pVec1, *pVec2;
__m64 shifter;
__m64 accu, normaccu;
long corr, norm;
int i;
pVec1 = (__m64*)pV1;
pVec2 = (__m64*)pV2;
shifter = _m_from_int(overlapDividerBitsNorm);
normaccu = accu = _mm_setzero_si64();
// Process 4 parallel sets of 2 * stereo samples or 4 * mono samples
// during each round for improved CPU-level parallellization.
for (i = 0; i < channels * overlapLength / 16; i ++)
{
__m64 temp, temp2;
// dictionary of instructions:
// _m_pmaddwd : 4*16bit multiply-add, resulting two 32bits = [a0*b0+a1*b1 ; a2*b2+a3*b3]
// _mm_add_pi32 : 2*32bit add
// _m_psrad : 32bit right-shift
temp = _mm_add_pi32(_mm_sra_pi32(_mm_madd_pi16(pVec1[0], pVec2[0]), shifter),
_mm_sra_pi32(_mm_madd_pi16(pVec1[1], pVec2[1]), shifter));
temp2 = _mm_add_pi32(_mm_sra_pi32(_mm_madd_pi16(pVec1[0], pVec1[0]), shifter),
_mm_sra_pi32(_mm_madd_pi16(pVec1[1], pVec1[1]), shifter));
accu = _mm_add_pi32(accu, temp);
normaccu = _mm_add_pi32(normaccu, temp2);
temp = _mm_add_pi32(_mm_sra_pi32(_mm_madd_pi16(pVec1[2], pVec2[2]), shifter),
_mm_sra_pi32(_mm_madd_pi16(pVec1[3], pVec2[3]), shifter));
temp2 = _mm_add_pi32(_mm_sra_pi32(_mm_madd_pi16(pVec1[2], pVec1[2]), shifter),
_mm_sra_pi32(_mm_madd_pi16(pVec1[3], pVec1[3]), shifter));
accu = _mm_add_pi32(accu, temp);
normaccu = _mm_add_pi32(normaccu, temp2);
pVec1 += 4;
pVec2 += 4;
}
// copy hi-dword of mm0 to lo-dword of mm1, then sum mmo+mm1
// and finally store the result into the variable "corr"
accu = _mm_add_pi32(accu, _mm_srli_si64(accu, 32));
corr = _m_to_int(accu);
normaccu = _mm_add_pi32(normaccu, _mm_srli_si64(normaccu, 32));
norm = _m_to_int(normaccu);
// Clear MMS state
_m_empty();
if (norm > (long)maxnorm)
{
// modify 'maxnorm' inside critical section to avoid multi-access conflict if in OpenMP mode
maxnorm = norm;
}
// Normalize result by dividing by sqrt(norm) - this step is easiest
// done using floating point operation
dnorm = (double)norm;
return (double)corr / sqrt(dnorm < 1e-9 ? 1.0 : dnorm);
// Note: Warning about the missing EMMS instruction is harmless
// as it'll be called elsewhere.
}
/// Update cross-correlation by accumulating "norm" coefficient by previously calculated value
double TDStretchMMX::calcCrossCorrAccumulate(const short *pV1, const short *pV2, double &dnorm)
{
const __m64 *pVec1, *pVec2;
__m64 shifter;
__m64 accu;
long corr, lnorm;
int i;
// cancel first normalizer tap from previous round
lnorm = 0;
for (i = 1; i <= channels; i ++)
{
lnorm -= (pV1[-i] * pV1[-i]) >> overlapDividerBitsNorm;
}
pVec1 = (__m64*)pV1;
pVec2 = (__m64*)pV2;
shifter = _m_from_int(overlapDividerBitsNorm);
accu = _mm_setzero_si64();
// Process 4 parallel sets of 2 * stereo samples or 4 * mono samples
// during each round for improved CPU-level parallellization.
for (i = 0; i < channels * overlapLength / 16; i ++)
{
__m64 temp;
// dictionary of instructions:
// _m_pmaddwd : 4*16bit multiply-add, resulting two 32bits = [a0*b0+a1*b1 ; a2*b2+a3*b3]
// _mm_add_pi32 : 2*32bit add
// _m_psrad : 32bit right-shift
temp = _mm_add_pi32(_mm_sra_pi32(_mm_madd_pi16(pVec1[0], pVec2[0]), shifter),
_mm_sra_pi32(_mm_madd_pi16(pVec1[1], pVec2[1]), shifter));
accu = _mm_add_pi32(accu, temp);
temp = _mm_add_pi32(_mm_sra_pi32(_mm_madd_pi16(pVec1[2], pVec2[2]), shifter),
_mm_sra_pi32(_mm_madd_pi16(pVec1[3], pVec2[3]), shifter));
accu = _mm_add_pi32(accu, temp);
pVec1 += 4;
pVec2 += 4;
}
// copy hi-dword of mm0 to lo-dword of mm1, then sum mmo+mm1
// and finally store the result into the variable "corr"
accu = _mm_add_pi32(accu, _mm_srli_si64(accu, 32));
corr = _m_to_int(accu);
// Clear MMS state
_m_empty();
// update normalizer with last samples of this round
pV1 = (short *)pVec1;
for (int j = 1; j <= channels; j ++)
{
lnorm += (pV1[-j] * pV1[-j]) >> overlapDividerBitsNorm;
}
dnorm += (double)lnorm;
if (lnorm > (long)maxnorm)
{
maxnorm = lnorm;
}
// Normalize result by dividing by sqrt(norm) - this step is easiest
// done using floating point operation
return (double)corr / sqrt((dnorm < 1e-9) ? 1.0 : dnorm);
}
void TDStretchMMX::clearCrossCorrState()
{
// Clear MMS state
_m_empty();
//_asm EMMS;
}
// MMX-optimized version of the function overlapStereo
void TDStretchMMX::overlapStereo(short *output, const short *input) const
{
const __m64 *pVinput, *pVMidBuf;
__m64 *pVdest;
__m64 mix1, mix2, adder, shifter;
int i;
pVinput = (const __m64*)input;
pVMidBuf = (const __m64*)pMidBuffer;
pVdest = (__m64*)output;
// mix1 = mixer values for 1st stereo sample
// mix1 = mixer values for 2nd stereo sample
// adder = adder for updating mixer values after each round
mix1 = _mm_set_pi16(0, overlapLength, 0, overlapLength);
adder = _mm_set_pi16(1, -1, 1, -1);
mix2 = _mm_add_pi16(mix1, adder);
adder = _mm_add_pi16(adder, adder);
// Overlaplength-division by shifter. "+1" is to account for "-1" deduced in
// overlapDividerBits calculation earlier.
shifter = _m_from_int(overlapDividerBitsPure + 1);
for (i = 0; i < overlapLength / 4; i ++)
{
__m64 temp1, temp2;
// load & shuffle data so that input & mixbuffer data samples are paired
temp1 = _mm_unpacklo_pi16(pVMidBuf[0], pVinput[0]); // = i0l m0l i0r m0r
temp2 = _mm_unpackhi_pi16(pVMidBuf[0], pVinput[0]); // = i1l m1l i1r m1r
// temp = (temp .* mix) >> shifter
temp1 = _mm_sra_pi32(_mm_madd_pi16(temp1, mix1), shifter);
temp2 = _mm_sra_pi32(_mm_madd_pi16(temp2, mix2), shifter);
pVdest[0] = _mm_packs_pi32(temp1, temp2); // pack 2*2*32bit => 4*16bit
// update mix += adder
mix1 = _mm_add_pi16(mix1, adder);
mix2 = _mm_add_pi16(mix2, adder);
// --- second round begins here ---
// load & shuffle data so that input & mixbuffer data samples are paired
temp1 = _mm_unpacklo_pi16(pVMidBuf[1], pVinput[1]); // = i2l m2l i2r m2r
temp2 = _mm_unpackhi_pi16(pVMidBuf[1], pVinput[1]); // = i3l m3l i3r m3r
// temp = (temp .* mix) >> shifter
temp1 = _mm_sra_pi32(_mm_madd_pi16(temp1, mix1), shifter);
temp2 = _mm_sra_pi32(_mm_madd_pi16(temp2, mix2), shifter);
pVdest[1] = _mm_packs_pi32(temp1, temp2); // pack 2*2*32bit => 4*16bit
// update mix += adder
mix1 = _mm_add_pi16(mix1, adder);
mix2 = _mm_add_pi16(mix2, adder);
pVinput += 2;
pVMidBuf += 2;
pVdest += 2;
}
_m_empty(); // clear MMS state
}
//////////////////////////////////////////////////////////////////////////////
//
// implementation of MMX optimized functions of class 'FIRFilter'
//
//////////////////////////////////////////////////////////////////////////////
#include "FIRFilter.h"
FIRFilterMMX::FIRFilterMMX() : FIRFilter()
{
filterCoeffsAlign = nullptr;
filterCoeffsUnalign = nullptr;
}
FIRFilterMMX::~FIRFilterMMX()
{
delete[] filterCoeffsUnalign;
}
// (overloaded) Calculates filter coefficients for MMX routine
void FIRFilterMMX::setCoefficients(const short *coeffs, uint newLength, uint uResultDivFactor)
{
uint i;
FIRFilter::setCoefficients(coeffs, newLength, uResultDivFactor);
// Ensure that filter coeffs array is aligned to 16-byte boundary
delete[] filterCoeffsUnalign;
filterCoeffsUnalign = new short[2 * newLength + 8];
filterCoeffsAlign = (short *)SOUNDTOUCH_ALIGN_POINTER_16(filterCoeffsUnalign);
// rearrange the filter coefficients for mmx routines
for (i = 0;i < length; i += 4)
{
filterCoeffsAlign[2 * i + 0] = coeffs[i + 0];
filterCoeffsAlign[2 * i + 1] = coeffs[i + 2];
filterCoeffsAlign[2 * i + 2] = coeffs[i + 0];
filterCoeffsAlign[2 * i + 3] = coeffs[i + 2];
filterCoeffsAlign[2 * i + 4] = coeffs[i + 1];
filterCoeffsAlign[2 * i + 5] = coeffs[i + 3];
filterCoeffsAlign[2 * i + 6] = coeffs[i + 1];
filterCoeffsAlign[2 * i + 7] = coeffs[i + 3];
}
}
// mmx-optimized version of the filter routine for stereo sound
uint FIRFilterMMX::evaluateFilterStereo(short *dest, const short *src, uint numSamples) const
{
// Create stack copies of the needed member variables for asm routines :
uint i, j;
__m64 *pVdest = (__m64*)dest;
if (length < 2) return 0;
for (i = 0; i < (numSamples - length) / 2; i ++)
{
__m64 accu1;
__m64 accu2;
const __m64 *pVsrc = (const __m64*)src;
const __m64 *pVfilter = (const __m64*)filterCoeffsAlign;
accu1 = accu2 = _mm_setzero_si64();
for (j = 0; j < lengthDiv8 * 2; j ++)
{
__m64 temp1, temp2;
temp1 = _mm_unpacklo_pi16(pVsrc[0], pVsrc[1]); // = l2 l0 r2 r0
temp2 = _mm_unpackhi_pi16(pVsrc[0], pVsrc[1]); // = l3 l1 r3 r1
accu1 = _mm_add_pi32(accu1, _mm_madd_pi16(temp1, pVfilter[0])); // += l2*f2+l0*f0 r2*f2+r0*f0
accu1 = _mm_add_pi32(accu1, _mm_madd_pi16(temp2, pVfilter[1])); // += l3*f3+l1*f1 r3*f3+r1*f1
temp1 = _mm_unpacklo_pi16(pVsrc[1], pVsrc[2]); // = l4 l2 r4 r2
accu2 = _mm_add_pi32(accu2, _mm_madd_pi16(temp2, pVfilter[0])); // += l3*f2+l1*f0 r3*f2+r1*f0
accu2 = _mm_add_pi32(accu2, _mm_madd_pi16(temp1, pVfilter[1])); // += l4*f3+l2*f1 r4*f3+r2*f1
// accu1 += l2*f2+l0*f0 r2*f2+r0*f0
// += l3*f3+l1*f1 r3*f3+r1*f1
// accu2 += l3*f2+l1*f0 r3*f2+r1*f0
// l4*f3+l2*f1 r4*f3+r2*f1
pVfilter += 2;
pVsrc += 2;
}
// accu >>= resultDivFactor
accu1 = _mm_srai_pi32(accu1, resultDivFactor);
accu2 = _mm_srai_pi32(accu2, resultDivFactor);
// pack 2*2*32bits => 4*16 bits
pVdest[0] = _mm_packs_pi32(accu1, accu2);
src += 4;
pVdest ++;
}
_m_empty(); // clear emms state
return (numSamples & 0xfffffffe) - length;
}
#else
// workaround to not complain about empty module
bool _dontcomplain_mmx_empty;
#endif // SOUNDTOUCH_ALLOW_MMX
@@ -0,0 +1,364 @@
////////////////////////////////////////////////////////////////////////////////
///
/// SSE optimized routines for Pentium-III, Athlon-XP and later CPUs. All SSE
/// optimized functions have been gathered into this single source
/// code file, regardless to their class or original source code file, in order
/// to ease porting the library to other compiler and processor platforms.
///
/// The SSE-optimizations are programmed using SSE compiler intrinsics that
/// are supported both by Microsoft Visual C++ and GCC compilers, so this file
/// should compile with both toolsets.
///
/// NOTICE: If using Visual Studio 6.0, you'll need to install the "Visual C++
/// 6.0 processor pack" update to support SSE instruction set. The update is
/// available for download at Microsoft Developers Network, see here:
/// http://msdn.microsoft.com/en-us/vstudio/aa718349.aspx
///
/// If the above URL is expired or removed, go to "http://msdn.microsoft.com" and
/// perform a search with keywords "processor pack".
///
/// Author : Copyright (c) Olli Parviainen
/// Author e-mail : oparviai 'at' iki.fi
/// SoundTouch WWW: http://www.surina.net/soundtouch
///
////////////////////////////////////////////////////////////////////////////////
//
// License :
//
// SoundTouch audio processing library
// Copyright (c) Olli Parviainen
//
// This library is free software; you can redistribute it and/or
// modify it under the terms of the GNU Lesser General Public
// License as published by the Free Software Foundation; either
// version 2.1 of the License, or (at your option) any later version.
//
// This library is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
// Lesser General Public License for more details.
//
// You should have received a copy of the GNU Lesser General Public
// License along with this library; if not, write to the Free Software
// Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
//
////////////////////////////////////////////////////////////////////////////////
#include "cpu_detect.h"
#include "STTypes.h"
using namespace soundtouch;
#ifdef SOUNDTOUCH_ALLOW_SSE
// SSE routines available only with float sample type
//////////////////////////////////////////////////////////////////////////////
//
// implementation of SSE optimized functions of class 'TDStretchSSE'
//
//////////////////////////////////////////////////////////////////////////////
#include "TDStretch.h"
#include <xmmintrin.h>
#include <math.h>
// Calculates cross correlation of two buffers
double TDStretchSSE::calcCrossCorr(const float *pV1, const float *pV2, double &anorm)
{
int i;
const float *pVec1;
const __m128 *pVec2;
__m128 vSum, vNorm;
// Note. It means a major slow-down if the routine needs to tolerate
// unaligned __m128 memory accesses. It's way faster if we can skip
// unaligned slots and use _mm_load_ps instruction instead of _mm_loadu_ps.
// This can mean up to ~ 10-fold difference (incl. part of which is
// due to skipping every second round for stereo sound though).
//
// Compile-time define SOUNDTOUCH_ALLOW_NONEXACT_SIMD_OPTIMIZATION is provided
// for choosing if this little cheating is allowed.
#ifdef ST_SIMD_AVOID_UNALIGNED
// Little cheating allowed, return valid correlation only for
// aligned locations, meaning every second round for stereo sound.
#define _MM_LOAD _mm_load_ps
if (((ulongptr)pV1) & 15) return -1e50; // skip unaligned locations
#else
// No cheating allowed, use unaligned load & take the resulting
// performance hit.
#define _MM_LOAD _mm_loadu_ps
#endif
// ensure overlapLength is divisible by 8
assert((overlapLength % 8) == 0);
// Calculates the cross-correlation value between 'pV1' and 'pV2' vectors
// Note: pV2 _must_ be aligned to 16-bit boundary, pV1 need not.
pVec1 = (const float*)pV1;
pVec2 = (const __m128*)pV2;
vSum = vNorm = _mm_setzero_ps();
// Unroll the loop by factor of 4 * 4 operations. Use same routine for
// stereo & mono, for mono it just means twice the amount of unrolling.
for (i = 0; i < channels * overlapLength / 16; i ++)
{
__m128 vTemp;
// vSum += pV1[0..3] * pV2[0..3]
vTemp = _MM_LOAD(pVec1);
vSum = _mm_add_ps(vSum, _mm_mul_ps(vTemp ,pVec2[0]));
vNorm = _mm_add_ps(vNorm, _mm_mul_ps(vTemp ,vTemp));
// vSum += pV1[4..7] * pV2[4..7]
vTemp = _MM_LOAD(pVec1 + 4);
vSum = _mm_add_ps(vSum, _mm_mul_ps(vTemp, pVec2[1]));
vNorm = _mm_add_ps(vNorm, _mm_mul_ps(vTemp ,vTemp));
// vSum += pV1[8..11] * pV2[8..11]
vTemp = _MM_LOAD(pVec1 + 8);
vSum = _mm_add_ps(vSum, _mm_mul_ps(vTemp, pVec2[2]));
vNorm = _mm_add_ps(vNorm, _mm_mul_ps(vTemp ,vTemp));
// vSum += pV1[12..15] * pV2[12..15]
vTemp = _MM_LOAD(pVec1 + 12);
vSum = _mm_add_ps(vSum, _mm_mul_ps(vTemp, pVec2[3]));
vNorm = _mm_add_ps(vNorm, _mm_mul_ps(vTemp ,vTemp));
pVec1 += 16;
pVec2 += 4;
}
// return value = vSum[0] + vSum[1] + vSum[2] + vSum[3]
float *pvNorm = (float*)&vNorm;
float norm = (pvNorm[0] + pvNorm[1] + pvNorm[2] + pvNorm[3]);
anorm = norm;
float *pvSum = (float*)&vSum;
return (double)(pvSum[0] + pvSum[1] + pvSum[2] + pvSum[3]) / sqrt(norm < 1e-9 ? 1.0 : norm);
/* This is approximately corresponding routine in C-language yet without normalization:
double corr, norm;
uint i;
// Calculates the cross-correlation value between 'pV1' and 'pV2' vectors
corr = norm = 0.0;
for (i = 0; i < channels * overlapLength / 16; i ++)
{
corr += pV1[0] * pV2[0] +
pV1[1] * pV2[1] +
pV1[2] * pV2[2] +
pV1[3] * pV2[3] +
pV1[4] * pV2[4] +
pV1[5] * pV2[5] +
pV1[6] * pV2[6] +
pV1[7] * pV2[7] +
pV1[8] * pV2[8] +
pV1[9] * pV2[9] +
pV1[10] * pV2[10] +
pV1[11] * pV2[11] +
pV1[12] * pV2[12] +
pV1[13] * pV2[13] +
pV1[14] * pV2[14] +
pV1[15] * pV2[15];
for (j = 0; j < 15; j ++) norm += pV1[j] * pV1[j];
pV1 += 16;
pV2 += 16;
}
return corr / sqrt(norm);
*/
}
double TDStretchSSE::calcCrossCorrAccumulate(const float *pV1, const float *pV2, double &norm)
{
// call usual calcCrossCorr function because SSE does not show big benefit of
// accumulating "norm" value, and also the "norm" rolling algorithm would get
// complicated due to SSE-specific alignment-vs-nonexact correlation rules.
return calcCrossCorr(pV1, pV2, norm);
}
//////////////////////////////////////////////////////////////////////////////
//
// implementation of SSE optimized functions of class 'FIRFilter'
//
//////////////////////////////////////////////////////////////////////////////
#include "FIRFilter.h"
FIRFilterSSE::FIRFilterSSE() : FIRFilter()
{
filterCoeffsAlign = nullptr;
filterCoeffsUnalign = nullptr;
}
FIRFilterSSE::~FIRFilterSSE()
{
delete[] filterCoeffsUnalign;
filterCoeffsAlign = nullptr;
filterCoeffsUnalign = nullptr;
}
// (overloaded) Calculates filter coefficients for SSE routine
void FIRFilterSSE::setCoefficients(const float *coeffs, uint newLength, uint uResultDivFactor)
{
uint i;
float fDivider;
FIRFilter::setCoefficients(coeffs, newLength, uResultDivFactor);
// Scale the filter coefficients so that it won't be necessary to scale the filtering result
// also rearrange coefficients suitably for SSE
// Ensure that filter coeffs array is aligned to 16-byte boundary
delete[] filterCoeffsUnalign;
filterCoeffsUnalign = new float[2 * newLength + 4];
filterCoeffsAlign = (float *)SOUNDTOUCH_ALIGN_POINTER_16(filterCoeffsUnalign);
fDivider = (float)resultDivider;
// rearrange the filter coefficients for mmx routines
for (i = 0; i < newLength; i ++)
{
filterCoeffsAlign[2 * i + 0] =
filterCoeffsAlign[2 * i + 1] = coeffs[i + 0] / fDivider;
}
}
// SSE-optimized version of the filter routine for stereo sound
uint FIRFilterSSE::evaluateFilterStereo(float *dest, const float *source, uint numSamples) const
{
int count = (int)((numSamples - length) & (uint)-2);
int j;
assert(count % 2 == 0);
if (count < 2) return 0;
assert(source != nullptr);
assert(dest != nullptr);
assert((length % 8) == 0);
assert(filterCoeffsAlign != nullptr);
assert(((ulongptr)filterCoeffsAlign) % 16 == 0);
// filter is evaluated for two stereo samples with each iteration, thus use of 'j += 2'
for (j = 0; j < count; j += 2)
{
const float *pSrc;
float *pDest;
const __m128 *pFil;
__m128 sum1, sum2;
uint i;
pSrc = (const float*)source + j * 2; // source audio data
pDest = dest + j * 2; // destination audio data
pFil = (const __m128*)filterCoeffsAlign; // filter coefficients. NOTE: Assumes coefficients
// are aligned to 16-byte boundary
sum1 = sum2 = _mm_setzero_ps();
for (i = 0; i < length / 8; i ++)
{
// Unroll loop for efficiency & calculate filter for 2*2 stereo samples
// at each pass
// sum1 is accu for 2*2 filtered stereo sound data at the primary sound data offset
// sum2 is accu for 2*2 filtered stereo sound data for the next sound sample offset.
sum1 = _mm_add_ps(sum1, _mm_mul_ps(_mm_loadu_ps(pSrc) , pFil[0]));
sum2 = _mm_add_ps(sum2, _mm_mul_ps(_mm_loadu_ps(pSrc + 2), pFil[0]));
sum1 = _mm_add_ps(sum1, _mm_mul_ps(_mm_loadu_ps(pSrc + 4), pFil[1]));
sum2 = _mm_add_ps(sum2, _mm_mul_ps(_mm_loadu_ps(pSrc + 6), pFil[1]));
sum1 = _mm_add_ps(sum1, _mm_mul_ps(_mm_loadu_ps(pSrc + 8) , pFil[2]));
sum2 = _mm_add_ps(sum2, _mm_mul_ps(_mm_loadu_ps(pSrc + 10), pFil[2]));
sum1 = _mm_add_ps(sum1, _mm_mul_ps(_mm_loadu_ps(pSrc + 12), pFil[3]));
sum2 = _mm_add_ps(sum2, _mm_mul_ps(_mm_loadu_ps(pSrc + 14), pFil[3]));
pSrc += 16;
pFil += 4;
}
// Now sum1 and sum2 both have a filtered 2-channel sample each, but we still need
// to sum the two hi- and lo-floats of these registers together.
// post-shuffle & add the filtered values and store to dest.
_mm_storeu_ps(pDest, _mm_add_ps(
_mm_shuffle_ps(sum1, sum2, _MM_SHUFFLE(1,0,3,2)), // s2_1 s2_0 s1_3 s1_2
_mm_shuffle_ps(sum1, sum2, _MM_SHUFFLE(3,2,1,0)) // s2_3 s2_2 s1_1 s1_0
));
}
// Ideas for further improvement:
// 1. If it could be guaranteed that 'source' were always aligned to 16-byte
// boundary, a faster aligned '_mm_load_ps' instruction could be used.
// 2. If it could be guaranteed that 'dest' were always aligned to 16-byte
// boundary, a faster '_mm_store_ps' instruction could be used.
return (uint)count;
/* original routine in C-language. please notice the C-version has differently
organized coefficients though.
double suml1, suml2;
double sumr1, sumr2;
uint i, j;
for (j = 0; j < count; j += 2)
{
const float *ptr;
const float *pFil;
suml1 = sumr1 = 0.0;
suml2 = sumr2 = 0.0;
ptr = src;
pFil = filterCoeffs;
for (i = 0; i < lengthLocal; i ++)
{
// unroll loop for efficiency.
suml1 += ptr[0] * pFil[0] +
ptr[2] * pFil[2] +
ptr[4] * pFil[4] +
ptr[6] * pFil[6];
sumr1 += ptr[1] * pFil[1] +
ptr[3] * pFil[3] +
ptr[5] * pFil[5] +
ptr[7] * pFil[7];
suml2 += ptr[8] * pFil[0] +
ptr[10] * pFil[2] +
ptr[12] * pFil[4] +
ptr[14] * pFil[6];
sumr2 += ptr[9] * pFil[1] +
ptr[11] * pFil[3] +
ptr[13] * pFil[5] +
ptr[15] * pFil[7];
ptr += 16;
pFil += 8;
}
dest[0] = (float)suml1;
dest[1] = (float)sumr1;
dest[2] = (float)suml2;
dest[3] = (float)sumr2;
src += 4;
dest += 4;
}
*/
}
#endif // SOUNDTOUCH_ALLOW_SSE
@@ -0,0 +1,250 @@
//////////////////////////////////////////////////////////////////////////////
///
/// SoundTouch DLL wrapper - wraps SoundTouch routines into a Dynamic Load
/// Library interface.
///
/// Author : Copyright (c) Olli Parviainen
/// Author e-mail : oparviai 'at' iki.fi
/// SoundTouch WWW: http://www.surina.net/soundtouch
///
////////////////////////////////////////////////////////////////////////////////
//
// License :
//
// SoundTouch audio processing library
// Copyright (c) Olli Parviainen
//
// This library is free software; you can redistribute it and/or
// modify it under the terms of the GNU Lesser General Public
// License as published by the Free Software Foundation; either
// version 2.1 of the License, or (at your option) any later version.
//
// This library is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
// Lesser General Public License for more details.
//
// You should have received a copy of the GNU Lesser General Public
// License along with this library; if not, write to the Free Software
// Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
//
////////////////////////////////////////////////////////////////////////////////
#if defined(_WIN32) || defined(WIN32)
#include <windows.h>
// DLL main in Windows compilation
BOOL APIENTRY DllMain( HANDLE hModule,
DWORD ul_reason_for_call,
LPVOID lpReserved
)
{
switch (ul_reason_for_call)
{
case DLL_PROCESS_ATTACH:
case DLL_THREAD_ATTACH:
case DLL_THREAD_DETACH:
case DLL_PROCESS_DETACH:
break;
}
return TRUE;
}
#endif
#include <limits.h>
#include <string.h>
#include "SoundTouchDLL.h"
#include "SoundTouch.h"
#include "BPMDetect.h"
using namespace soundtouch;
#ifdef SOUNDTOUCH_INTEGER_SAMPLES
#error "error - compile the dll version with float samples"
#endif // SOUNDTOUCH_INTEGER_SAMPLES
//////////////
SOUNDTOUCHDLL_API ST_HANDLE __cdecl soundtouch_createInstance()
{
return new SoundTouch();
}
SOUNDTOUCHDLL_API void __cdecl soundtouch_destroyInstance(ST_HANDLE h)
{
delete static_cast<SoundTouch*>(h);
}
/// Get SoundTouch library version string
SOUNDTOUCHDLL_API const char *__cdecl soundtouch_getVersionString()
{
return SoundTouch::getVersionString();
}
/// Get SoundTouch library version Id
SOUNDTOUCHDLL_API uint __cdecl soundtouch_getVersionId()
{
return SoundTouch::getVersionId();
}
/// Sets new rate control value. Normal rate = 1.0, smaller values
/// represent slower rate, larger faster rates.
SOUNDTOUCHDLL_API void __cdecl soundtouch_setRate(ST_HANDLE h, float newRate)
{
static_cast<SoundTouch*>(h)->setRate(newRate);
}
/// Sets new tempo control value. Normal tempo = 1.0, smaller values
/// represent slower tempo, larger faster tempo.
SOUNDTOUCHDLL_API void __cdecl soundtouch_setTempo(ST_HANDLE h, float newTempo)
{
static_cast<SoundTouch*>(h)->setTempo(newTempo);
}
/// Sets new rate control value as a difference in percents compared
/// to the original rate (-50 .. +100 %)
SOUNDTOUCHDLL_API void __cdecl soundtouch_setRateChange(ST_HANDLE h, float newRate)
{
static_cast<SoundTouch*>(h)->setRateChange(newRate);
}
/// Sets new tempo control value as a difference in percents compared
/// to the original tempo (-50 .. +100 %)
SOUNDTOUCHDLL_API void __cdecl soundtouch_setTempoChange(ST_HANDLE h, float newTempo)
{
static_cast<SoundTouch*>(h)->setTempoChange(newTempo);
}
/// Sets new pitch control value. Original pitch = 1.0, smaller values
/// represent lower pitches, larger values higher pitch.
SOUNDTOUCHDLL_API void __cdecl soundtouch_setPitch(ST_HANDLE h, float newPitch)
{
static_cast<SoundTouch*>(h)->setPitch(newPitch);
}
/// Sets pitch change in octaves compared to the original pitch
/// (-1.00 .. +1.00)
SOUNDTOUCHDLL_API void __cdecl soundtouch_setPitchOctaves(ST_HANDLE h, float newPitch)
{
static_cast<SoundTouch*>(h)->setPitchOctaves(newPitch);
}
/// Sets pitch change in semi-tones compared to the original pitch
/// (-12 .. +12)
SOUNDTOUCHDLL_API void __cdecl soundtouch_setPitchSemiTones(ST_HANDLE h, float newPitch)
{
static_cast<SoundTouch*>(h)->setPitchSemiTones(newPitch);
}
/// Sets the number of channels, 1 = mono, 2 = stereo
SOUNDTOUCHDLL_API void __cdecl soundtouch_setChannels(ST_HANDLE h, uint numChannels)
{
static_cast<SoundTouch*>(h)->setChannels(numChannels);
}
/// Sets sample rate.
SOUNDTOUCHDLL_API void __cdecl soundtouch_setSampleRate(ST_HANDLE h, uint srate)
{
static_cast<SoundTouch*>(h)->setSampleRate(srate);
}
/// Flushes the last samples from the processing pipeline to the output.
/// Clears also the internal processing buffers.
//
/// Note: This function is meant for extracting the last samples of a sound
/// stream. This function may introduce additional blank samples in the end
/// of the sound stream, and thus it's not recommended to call this function
/// in the middle of a sound stream.
SOUNDTOUCHDLL_API void __cdecl soundtouch_flush(ST_HANDLE h)
{
static_cast<SoundTouch*>(h)->flush();
}
/// Adds 'numSamples' pcs of samples from the 'samples' memory position into
/// the input of the object. Notice that sample rate _has_to_ be set before
/// calling this function, otherwise throws a runtime_error exception.
SOUNDTOUCHDLL_API void __cdecl soundtouch_putSamples(ST_HANDLE h,
const SAMPLETYPE *samples, ///< Pointer to sample buffer.
unsigned int numSamples ///< Number of samples in buffer. Notice
///< that in case of stereo-sound a single sample
///< contains data for both channels.
)
{
static_cast<SoundTouch*>(h)->putSamples(samples, numSamples);
}
/// Clears all the samples in the object's output and internal processing
/// buffers.
SOUNDTOUCHDLL_API void __cdecl soundtouch_clear(ST_HANDLE h)
{
return static_cast<SoundTouch*>(h)->clear();
}
/// Changes a setting controlling the processing system behaviour. See the
/// 'SETTING_...' defines for available setting ID's.
///
/// \return 'nonzero' if the setting was successfully changed
SOUNDTOUCHDLL_API int __cdecl soundtouch_setSetting(ST_HANDLE h,
int settingId, ///< Setting ID number. see SETTING_... defines.
int value ///< New setting value.
)
{
return static_cast<SoundTouch*>(h)->setSetting(settingId, value);
}
/// Reads a setting controlling the processing system behaviour. See the
/// 'SETTING_...' defines for available setting ID's.
///
/// \return the setting value.
SOUNDTOUCHDLL_API int __cdecl soundtouch_getSetting(ST_HANDLE h,
int settingId ///< Setting ID number, see SETTING_... defines.
)
{
return static_cast<SoundTouch*>(h)->getSetting(settingId);
}
/// Returns number of samples currently unprocessed.
SOUNDTOUCHDLL_API uint __cdecl soundtouch_numUnprocessedSamples(ST_HANDLE h)
{
return static_cast<SoundTouch*>(h)->numUnprocessedSamples();
}
/// Receive ready samples from the processing pipeline.
///
/// if called with outBuffer=nullptr, just reduces amount of ready samples within the pipeline.
SOUNDTOUCHDLL_API uint __cdecl soundtouch_receiveSamples(ST_HANDLE h,
SAMPLETYPE *outBuffer, ///< Buffer where to copy output samples.
unsigned int maxSamples ///< How many samples to receive at max.
)
{
if (outBuffer)
{
return static_cast<SoundTouch*>(h)->receiveSamples(outBuffer, maxSamples);
}
else
{
return static_cast<SoundTouch*>(h)->receiveSamples(maxSamples);
}
}
/// Returns number of samples currently available.
SOUNDTOUCHDLL_API uint __cdecl soundtouch_numSamples(ST_HANDLE h)
{
return static_cast<SoundTouch*>(h)->numSamples();
}
/// Returns nonzero if there aren't any samples available for outputting.
SOUNDTOUCHDLL_API int __cdecl soundtouch_isEmpty(ST_HANDLE h)
{
return static_cast<SoundTouch*>(h)->isEmpty();
}
@@ -0,0 +1,100 @@
// Microsoft Visual C++ generated resource script.
//
#include "resource.h"
#define APSTUDIO_READONLY_SYMBOLS
/////////////////////////////////////////////////////////////////////////////
//
// Generated from the TEXTINCLUDE 2 resource.
//
#include "winres.h"
/////////////////////////////////////////////////////////////////////////////
#undef APSTUDIO_READONLY_SYMBOLS
/////////////////////////////////////////////////////////////////////////////
// English (United States) resources
#if !defined(AFX_RESOURCE_DLL) || defined(AFX_TARG_ENU)
LANGUAGE LANG_ENGLISH, SUBLANG_ENGLISH_US
#pragma code_page(1252)
#ifdef APSTUDIO_INVOKED
/////////////////////////////////////////////////////////////////////////////
//
// TEXTINCLUDE
//
1 TEXTINCLUDE
BEGIN
"resource.h\0"
END
2 TEXTINCLUDE
BEGIN
"#include ""afxres.h""\r\n"
"\0"
END
3 TEXTINCLUDE
BEGIN
"\r\n"
"\0"
END
#endif // APSTUDIO_INVOKED
/////////////////////////////////////////////////////////////////////////////
//
// Version
//
VS_VERSION_INFO VERSIONINFO
FILEVERSION 2,3,2,0
PRODUCTVERSION 2,3,2,0
FILEFLAGSMASK 0x17L
#ifdef _DEBUG
FILEFLAGS 0x1L
#else
FILEFLAGS 0x0L
#endif
FILEOS 0x4L
FILETYPE 0x2L
FILESUBTYPE 0x0L
BEGIN
BLOCK "StringFileInfo"
BEGIN
BLOCK "000004b0"
BEGIN
VALUE "Comments", "SoundTouch Library licensed for 3rd party applications subject to LGPL license v2.1. Visit http://www.surina.net/soundtouch for more information about the SoundTouch library."
VALUE "FileDescription", "SoundTouch Dynamic Link Library"
VALUE "FileVersion", "2.3.3.0"
VALUE "InternalName", "SoundTouch"
VALUE "LegalCopyright", "Copyright (C) Olli Parviainen 2024"
VALUE "OriginalFilename", "SoundTouch.dll"
VALUE "ProductName", " SoundTouch Dynamic Link Library"
VALUE "ProductVersion", "2.3.3.0"
END
END
BLOCK "VarFileInfo"
BEGIN
VALUE "Translation", 0x0, 1200
END
END
#endif // English (United States) resources
/////////////////////////////////////////////////////////////////////////////
#ifndef APSTUDIO_INVOKED
/////////////////////////////////////////////////////////////////////////////
//
// Generated from the TEXTINCLUDE 3 resource.
//
/////////////////////////////////////////////////////////////////////////////
#endif // not APSTUDIO_INVOKED
@@ -0,0 +1,15 @@
//{{NO_DEPENDENCIES}}
// Microsoft Visual C++ generated include file.
// Used by SoundTouchDLL.rc
//
// Next default values for new objects
//
#ifdef APSTUDIO_INVOKED
#ifndef APSTUDIO_READONLY_SYMBOLS
#define _APS_NEXT_RESOURCE_VALUE 101
#define _APS_NEXT_COMMAND_VALUE 40001
#define _APS_NEXT_CONTROL_VALUE 1000
#define _APS_NEXT_SYMED_VALUE 101
#endif
#endif
+54 -3
View File
@@ -25,6 +25,7 @@ static void fin_deferred_cb(struct ll_queue* q, void* arg);
static void write_queue_fetch_cb(struct ll_queue* q, void* arg);
static int flush_write_buf(struct tcp_conn* tc);
static void tcp_conn_handle_error(struct tcp_conn* tc, int err);
static void connect_timeout_cb(void* arg);
static const uint8_t tcp_fin_sentinel;
struct tcp_conn* tcp_conn_create(
@@ -121,6 +122,8 @@ void tcp_conn_destroy(struct tcp_conn* tc) {
DEBUG_DEBUG(DEBUG_CATEGORY_SOCKET, "tcp_conn_destroy: fd=%d connected=%d error=%d fin_remote=%d fin_local=%d closed=%d",
(int)tc->sock, tc->connected, tc->error, tc->fin_remote, tc->fin_local, tc->closed);
if (tc->connect_timer) { uasync_cancel_timeout(tc->ua, tc->connect_timer); tc->connect_timer = NULL; }
if (tc->socket_id) {
uasync_remove_socket_t(tc->ua, tc->sock);
tc->socket_id = NULL;
@@ -219,6 +222,7 @@ static void read_cb(socket_t sock, void* arg) {
} else if (n == 0) {
memory_pool_free(tc->data_pool, buf);
queue_entry_free(e);
if (tc->fin_remote) return;
tc->fin_remote = 1;
DEBUG_DEBUG(DEBUG_CATEGORY_SOCKET, "tcp_io: FIN fd=%d", (int)tc->sock);
uasync_set_socket_read(tc->ua, tc->socket_id, 0);
@@ -383,6 +387,7 @@ static void write_cb(socket_t sock, void* arg) {
socklen_t len = sizeof(err);
if (getsockopt(tc->sock, SOL_SOCKET, SO_ERROR, (char*)&err, &len) == 0 && err == 0) {
tc->connected = 1;
if (tc->connect_timer) { uasync_cancel_timeout(tc->ua, tc->connect_timer); tc->connect_timer = NULL; }
DEBUG_DEBUG(DEBUG_CATEGORY_SOCKET, "tcp_io: connect ok fd=%d", (int)tc->sock);
} else {
DEBUG_ERROR(DEBUG_CATEGORY_SOCKET, "tcp_io: connect fail fd=%d err=%d", (int)tc->sock, err);
@@ -410,9 +415,32 @@ static void error_cb(socket_t sock, void* arg) {
(void)sock;
struct tcp_conn* tc = (struct tcp_conn*)arg;
if (!tc || tc->sock == SOCKET_INVALID) return;
DEBUG_ERROR(DEBUG_CATEGORY_SOCKET, "tcp_io: async error fd=%d connected=%d err=%d fin_r=%d fin_l=%d closed=%d wmon=%d",
(int)tc->sock, tc->connected, tc->error, tc->fin_remote, tc->fin_local, tc->closed, tc->write_monitor);
tcp_conn_handle_error(tc, -1);
int so_err = 0; socklen_t so_len = sizeof(so_err);
if (getsockopt(tc->sock, SOL_SOCKET, SO_ERROR, &so_err, &so_len) == 0 && so_err == 0) {
// Peer полностью закрыл свой сокет (EPOLLHUP) после того, как FIN уже прочитан (fin_remote=1).
// EPOLLHUP — level-triggered и «always reported»: если оставить сокет в epoll, каждая итерация
// epoll_wait мгновенно вернёт HUP и error_cb зациклится на 100% CPU. Закрываем сокет.
if (tc->fin_remote) {
DEBUG_DEBUG(DEBUG_CATEGORY_SOCKET, "tcp_io: async HUP (graceful) fd=%d — peer fully closed after FIN, closing",
(int)tc->sock);
tcp_conn_push_close(tc);
return;
}
// Грациозное закрытие peer (FIN, EPOLLHUP) при приостановленном чтении: FIN пришёл не как
// EPOLLIN (чтение отключено backpressure-ом), а как HUP. Возобновляем чтение, чтобы
// read_cb дослил остаток и прочитал FIN (recv()==0 → fin_remote → отложенный on_fin).
DEBUG_DEBUG(DEBUG_CATEGORY_SOCKET, "tcp_io: async HUP (graceful) fd=%d fin_r=%d fin_l=%d read_paused=%d rq=%d — resume read",
(int)tc->sock, tc->fin_remote, tc->fin_local, tc->read_paused,
tc->read_queue ? tc->read_queue->count : 0);
queue_waiter_cancel(tc->read_queue, &tc->read_waiter);
uasync_set_socket_read(tc->ua, tc->socket_id, 1);
tc->read_paused = 0;
read_cb(tc->sock, tc);
return;
}
DEBUG_ERROR(DEBUG_CATEGORY_SOCKET, "tcp_io: async error fd=%d connected=%d err=%d so_err=%d fin_r=%d fin_l=%d closed=%d wmon=%d",
(int)tc->sock, tc->connected, tc->error, so_err, tc->fin_remote, tc->fin_local, tc->closed, tc->write_monitor);
tcp_conn_handle_error(tc, so_err ? so_err : -1);
}
// ====================================================================
@@ -453,3 +481,26 @@ void tcp_conn_pause_read(struct tcp_conn* tc) {
queue_waiter_cancel(tc->read_queue, &tc->read_waiter);
tc->read_paused = 1;
}
// ====================================================================
// Таймаут установления соединения
// ====================================================================
static void connect_timeout_cb(void* arg) {
struct tcp_conn* tc = (struct tcp_conn*)arg;
if (!tc) return;
tc->connect_timer = NULL;
if (!tc->connected && tc->sock != SOCKET_INVALID) {
DEBUG_WARN(DEBUG_CATEGORY_SOCKET, "tcp_io: connect timeout fd=%d (limit %d ms)", (int)tc->sock, tc->connect_timeout_ms);
tcp_conn_handle_error(tc, ETIMEDOUT);
}
}
void tcp_conn_set_connect_timeout(struct tcp_conn* tc, int timeout_ms) {
if (!tc) return;
if (timeout_ms < 0) timeout_ms = 0;
tc->connect_timeout_ms = timeout_ms;
if (!tc->connected && timeout_ms > 0) {
tc->connect_timer = uasync_set_timeout(tc->ua, timeout_ms * 10, tc, connect_timeout_cb, "tcp_conn_ct");
}
}
+9
View File
@@ -53,6 +53,9 @@ struct tcp_conn {
uint8_t closed; // сокет полностью закрыт (close)
uint8_t destroyed; // 1 = tcp_conn_destroy вызван, tc ожидает отложенного free
int connect_timeout_ms; // таймаут установления соединения, 0 = без таймаута
void* connect_timer; // handle uasync таймера connect
// Частичная отправка (из data_pool, не в очереди — досылается первой)
uint8_t* write_buf;
size_t write_len;
@@ -113,6 +116,12 @@ void tcp_conn_set_flushed(struct tcp_conn* tc, void (*on_flushed)(struct tcp_con
// Принудительная остановка чтения: убирает EPOLLIN и отменяет read_waiter.
void tcp_conn_pause_read(struct tcp_conn* tc);
// Таймаут установления соединения (мс, 0 = без таймаута). Если соединение ещё не
// установлено (connected=0), взводит таймер; по истечении вызывает
// tcp_conn_handle_error(ETIMEDOUT) → on_error. Таймер гасится при успешном connect
// и при tcp_conn_destroy.
void tcp_conn_set_connect_timeout(struct tcp_conn* tc, int timeout_ms);
#ifdef __cplusplus
}
+62 -13
View File
@@ -518,6 +518,10 @@ static void get_next_timeout(struct UASYNC* ua, struct timeval* tv) {
uint64_t now_ms = timeval_to_ms(&now_tv);
if (entry.expiration <= now_ms) {
struct timeout_node* rn = (struct timeout_node*)entry.data;
DEBUG_DEBUG(DEBUG_CATEGORY_TIMERS, "get_next_timeout: root timer '%s' already expired (exp=%llu now=%llu delta=%lldms size=%zu)",
rn ? rn->name : "?", (unsigned long long)entry.expiration,
(unsigned long long)now_ms, (long long)(now_ms - entry.expiration), ua->timeout_heap->size);
tv->tv_sec = 0;
tv->tv_usec = 0;
return;
@@ -1050,6 +1054,13 @@ void uasync_poll(struct UASYNC* ua, int timeout_tb) {
}
}
/* Если ближайший таймер уже истёк (get_next_timeout вернул 0), но heap не пуст —
* опрашиваем немедленно (timeout_ms=0), иначе epoll_wait(-1) заблокирует навсегда,
* а process_timeouts (вызывается только после epoll_wait) никогда не запустится. */
if (timeout_ms == -1 && ua->timeout_heap->size > 0) {
timeout_ms = 0;
}
DEBUG_DEBUG(DEBUG_CATEGORY_SYS, "poll(%d sockets, %zu timers, timeout=%dms)",
socket_count, ua->timeout_heap->size, timeout_ms);
@@ -1157,28 +1168,37 @@ void uasync_poll(struct UASYNC* ua, int timeout_tb) {
int idx = ua->sockets->active_indices[i];
struct socket_node* node = &ua->sockets->sockets[idx];
if (!node->active) continue;
SOCKET s;
if (node->type == SOCKET_NODE_TYPE_SOCK) {
s = node->sock;
} else {
s = (SOCKET)node->fd;
}
int has_read = FD_ISSET(s, &read_fds);
int has_write = FD_ISSET(s, &write_fds);
int has_except = FD_ISSET(s, &except_fds);
if (!has_read && !has_write && !has_except) continue;
DEBUG_DEBUG(DEBUG_CATEGORY_SYS, "select→fd=%d r=%d w=%d e=%d", (int)s, has_read, has_write, has_except);
/* Коллбэки могут удалить/добавить сокет (realloc массива) — перед
* каждым последующим коллбэком пере-валидируем node по индексу. */
int cb_called = 0;
if (has_except) {
if (node->except_cbk) {
node->except_cbk(node->fd, node->user_data);
}
cb_called++;
}
if (has_read) {
if (cb_called) {
node = &ua->sockets->sockets[idx];
if (!node->active) continue;
}
if (node->type == SOCKET_NODE_TYPE_SOCK) {
if (node->read_cbk_sock) {
node->read_cbk_sock(node->sock, node->user_data);
@@ -1188,9 +1208,14 @@ void uasync_poll(struct UASYNC* ua, int timeout_tb) {
node->read_cbk(node->fd, node->user_data);
}
}
cb_called++;
}
if (has_write) {
if (cb_called) {
node = &ua->sockets->sockets[idx];
if (!node->active) continue;
}
if (node->type == SOCKET_NODE_TYPE_SOCK) {
if (node->write_cbk_sock) {
node->write_cbk_sock(node->sock, node->user_data);
@@ -1200,6 +1225,7 @@ void uasync_poll(struct UASYNC* ua, int timeout_tb) {
node->write_cbk(node->fd, node->user_data);
}
}
cb_called++;
}
}
}
@@ -1243,13 +1269,17 @@ void uasync_poll(struct UASYNC* ua, int timeout_tb) {
}
/* Socket event - lookup by fd */
struct socket_node* node = socket_array_get(ua->sockets, ua->poll_fds[i].fd);
int fd = ua->poll_fds[i].fd;
struct socket_node* node = socket_array_get(ua->sockets, fd);
if (!node) { // Try by socket_t (in case this is a socket)
socket_t lookup_sock = ua->poll_fds[i].fd;
node = socket_array_get_by_sock(ua->sockets, lookup_sock);
node = socket_array_get_by_sock(ua->sockets, fd);
}
if (!node) continue; // Socket may have been removed
/* Коллбэки могут удалить/добавить сокет (realloc массива) — перед
* каждым последующим коллбэком пере-валидируем node по fd. */
int cb_called = 0;
/* Read readiness BEFORE error — avoid losing data on combined IN+ERR/HUP events */
if (ua->poll_fds[i].revents & POLLIN) {
if (node->type == SOCKET_NODE_TYPE_SOCK) {
@@ -1261,10 +1291,16 @@ void uasync_poll(struct UASYNC* ua, int timeout_tb) {
node->read_cbk(node->fd, node->user_data);
}
}
cb_called++;
}
/* Write readiness BEFORE error — flush pending writes before handling HUP */
if (ua->poll_fds[i].revents & POLLOUT) {
if (cb_called) {
node = socket_array_get(ua->sockets, fd);
if (!node) node = socket_array_get_by_sock(ua->sockets, fd);
if (!node) continue;
}
if (node->type == SOCKET_NODE_TYPE_SOCK) {
if (node->write_cbk_sock) {
node->write_cbk_sock(node->sock, node->user_data);
@@ -1274,21 +1310,34 @@ void uasync_poll(struct UASYNC* ua, int timeout_tb) {
node->write_cbk(node->fd, node->user_data);
}
}
cb_called++;
}
/* Check for error conditions LAST — I/O handlers drain/process data first */
if (ua->poll_fds[i].revents & (POLLERR | POLLHUP | POLLNVAL)) {
if (cb_called) {
node = socket_array_get(ua->sockets, fd);
if (!node) node = socket_array_get_by_sock(ua->sockets, fd);
if (!node) continue;
}
/* Treat as exceptional condition */
if (node->except_cbk) {
node->except_cbk(node->fd, node->user_data);
}
cb_called++;
}
/* Exceptional data (out-of-band) */
if (ua->poll_fds[i].revents & POLLPRI) {
if (cb_called) {
node = socket_array_get(ua->sockets, fd);
if (!node) node = socket_array_get_by_sock(ua->sockets, fd);
if (!node) continue;
}
if (node->except_cbk) {
node->except_cbk(node->fd, node->user_data);
}
cb_called++;
}
}
}
+10
View File
@@ -6,6 +6,16 @@
"type": "local",
"command": ["/home/user/.local/bin/uvx", "kwin-mcp"],
"enabled": true
},
"gdb-mcp": {
"type": "local",
"command": ["/home/user/.local/bin/uvx", "--with", "mcp<2", "gdb-mcp"],
"enabled": true
},
"aiterm": {
"type": "local",
"command": ["/home/user/.npm-global/bin/aiterm-mcp"],
"enabled": true
}
}
}
+32
View File
@@ -15,6 +15,7 @@ utun_CORE_SOURCES = \
routing_layer/topo_node.c \
routing_layer/topo_node_sqlite.c \
routing_layer/route_connectivity.c \
routing_layer/sock_match.c \
routing_layer/conn_mgr_core.c \
routing_layer/conn_mgr_indirect.c \
routing_layer/conn_mgr_monitor.c \
@@ -36,11 +37,15 @@ utun_CORE_SOURCES = \
transport_layer/etcp_debug.c \
transport_layer/etcp_dump.c \
transport_layer/secure_channel.c \
transport_layer/reality.c \
transport_layer/reality_fingerprint.c \
transport_layer/reality_relay.c \
transport_layer/crc32.c \
transport_layer/stcp_link.c \
transport_layer/stcp.c \
transport_layer/stcp_server.c \
transport_layer/stcp_client.c \
transport_layer/socks_client.c \
transport_layer/pkt_normalizer.c \
transport_layer/packet_dump.c \
transport_layer/etcp_api.c \
@@ -56,11 +61,13 @@ utun_CORE_SOURCES = \
media_delivery/media_download.c \
media_delivery/media_index.c \
media_async/media_async.c \
video/video.c \
transport_layer/dummynet.c \
ntp_time.c \
ntp_node_time.c \
proxy/tcp_proxy_client.c \
routing_layer/etcp_router.c \
routing_layer/route_crypto.c \
proxy/tcp_proxy_server.c \
proxy/udp_proxy.c \
proxy/socks_proxy.c \
@@ -78,10 +85,16 @@ utun_CORE_SOURCES = \
chat/chat_profile.c \
chat/chat_status.c \
chat/chat_sync.c \
chat/chat_admin.c \
chat/chat_join.c \
chat/member_sync.c \
chat/merkle_sync.c \
chat/invite_link.c \
chat/invite_build.c \
chat/chat_headless_control.c \
dm/dm_core.c \
dm/dm_crypto.c \
dm/dm_mailbox.c \
broadcast.c
# libutun: all core sources except main()
@@ -97,6 +110,7 @@ libutun_a_SOURCES = \
routing_layer/topo_node.c \
routing_layer/topo_node_sqlite.c \
routing_layer/route_connectivity.c \
routing_layer/sock_match.c \
routing_layer/conn_mgr_core.c \
routing_layer/conn_mgr_indirect.c \
routing_layer/conn_mgr_monitor.c \
@@ -118,11 +132,15 @@ libutun_a_SOURCES = \
transport_layer/etcp_debug.c \
transport_layer/etcp_dump.c \
transport_layer/secure_channel.c \
transport_layer/reality.c \
transport_layer/reality_fingerprint.c \
transport_layer/reality_relay.c \
transport_layer/crc32.c \
transport_layer/stcp_link.c \
transport_layer/stcp.c \
transport_layer/stcp_server.c \
transport_layer/stcp_client.c \
transport_layer/socks_client.c \
transport_layer/pkt_normalizer.c \
transport_layer/packet_dump.c \
transport_layer/etcp_api.c \
@@ -138,11 +156,13 @@ libutun_a_SOURCES = \
media_delivery/media_download.c \
media_delivery/media_index.c \
media_async/media_async.c \
video/video.c \
transport_layer/dummynet.c \
ntp_time.c \
ntp_node_time.c \
proxy/tcp_proxy_client.c \
routing_layer/etcp_router.c \
routing_layer/route_crypto.c \
proxy/tcp_proxy_server.c \
proxy/udp_proxy.c \
proxy/socks_proxy.c \
@@ -159,10 +179,16 @@ libutun_a_SOURCES = \
chat/chat_profile.c \
chat/chat_status.c \
chat/chat_sync.c \
chat/chat_admin.c \
chat/chat_join.c \
chat/member_sync.c \
chat/merkle_sync.c \
chat/invite_link.c \
chat/invite_build.c \
chat/chat_headless_control.c \
dm/dm_core.c \
dm/dm_crypto.c \
dm/dm_mailbox.c \
broadcast.c
libutun_a_CFLAGS = $(utun_CFLAGS)
@@ -179,6 +205,7 @@ utun_CFLAGS = \
-I$(top_srcdir)/src/routing_layer \
-I$(top_srcdir)/src/media_delivery \
-I$(top_srcdir)/src/media_async \
-I$(top_srcdir)/src/video \
-I$(top_srcdir)/src/uip \
-g \
-DUSE_SQLITE \
@@ -200,6 +227,11 @@ utun_CFLAGS += -DHAVE_WHISPER
utun_LDADD += @WHISPER_LIBS@
endif
if HAVE_FFMPEG
utun_CFLAGS += -DHAVE_FFMPEG @FFMPEG_CFLAGS@
utun_LDADD += @FFMPEG_LIBS@
endif
# Copy binary to project root after building
all-local: copy-to-root
+176
View File
@@ -0,0 +1,176 @@
/*
* chat_admin.c — передача канального приватного ключа (прав админа) другому узлу
*
* Владелец канала передаёт оба ключа (ed25519_privkey + x25519_privkey) выбранному
* мемберу через etcp_router с ROUTE_CRYPTO_SIGN | ROUTE_CRYPTO_ENCRYPT.
* Получатель верифицирует ключи (вывод pubkey == сохранённые pubkey канала) и импортирует.
*/
#include "chat_admin.h"
#include "chat_core_priv.h"
#include "chat_event.h"
#include "../utun_instance.h"
#include "../routing_layer/topo_node_sqlite.h"
#include "../routing_layer/etcp_router.h"
#include "../routing_layer/route_crypto.h"
#include "../transport_layer/etcp_api.h"
#include "../transport_layer/etcp.h"
#include "../transport_layer/secure_channel.h"
#include "../../lib/mem.h"
#include "../../lib/ll_queue.h"
#include "../../lib/debug_config.h"
#include <string.h>
#include <stdio.h>
#include <stdlib.h>
#include <openssl/evp.h>
#define CA_ID "chat_admin"
/* ─── ed25519 pubkey из ed25519 privkey (канальные ключи — независимая пара) ─── */
static int ed25519_pub_from_priv(const uint8_t priv[32], uint8_t pub[32]) {
EVP_PKEY* pkey = EVP_PKEY_new_raw_private_key(EVP_PKEY_ED25519, NULL, priv, 32);
if (!pkey) return -1;
size_t plen = 32;
int ok = EVP_PKEY_get_raw_public_key(pkey, pub, &plen) == 1 && plen == 32;
EVP_PKEY_free(pkey);
return ok ? 0 : -1;
}
/* ─── приём переданного ключа ───
* payload: [group_id:8 LE][ed25519_priv:32][x25519_priv:32]
*/
static void admin_key_recv_cb(struct ETCP_CONN* conn, struct ll_entry* entry) {
if (!entry) return;
if (!conn || !entry->dgram || entry->len < ROUTER_SVC_PAYLOAD_OFF + 8 + 32 + 32) {
queue_dgram_free(entry); queue_entry_free(entry); return;
}
struct UTUN_INSTANCE* inst = conn->instance;
if (!inst) { queue_dgram_free(entry); queue_entry_free(entry); return; }
struct chat_core_ctx* cc = CC(inst);
if (!cc || !cc->initialized || !cc->db) { queue_dgram_free(entry); queue_entry_free(entry); return; }
const uint8_t* d = entry->dgram;
uint64_t from_node;
memcpy(&from_node, d + ROUTER_SVC_SRC_OFF, 8);
const uint8_t* payload = d + ROUTER_SVC_PAYLOAD_OFF;
uint64_t group_id; memcpy(&group_id, payload, 8);
const uint8_t* ed_priv = payload + 8;
const uint8_t* x_priv = payload + 8 + 32;
char ch_id[64]; snprintf(ch_id, sizeof(ch_id), "%llu", (unsigned long long)group_id);
/* проверить, что ключи соответствуют публичным ключам канала (защита от подлога) */
uint8_t ch_x_pub[32], ch_ed_pub[32];
if (topo_node_sqlite_channel_get(cc->db, ch_id, NULL, 0, NULL, ch_x_pub, ch_ed_pub, NULL) != 0) {
DEBUG_ERROR(DEBUG_CATEGORY_CHAT_SYNC, "%s: recv — no channel ch=%s from 0x%016llx, drop",
CA_ID, ch_id, (unsigned long long)from_node);
queue_dgram_free(entry); queue_entry_free(entry); return;
}
uint8_t ed_pub_check[32], x_pub_check[32];
if (ed25519_pub_from_priv(ed_priv, ed_pub_check) != 0
|| memcmp(ed_pub_check, ch_ed_pub, 32) != 0
|| sc_compute_public_key_from_private(x_priv, x_pub_check) != SC_OK
|| memcmp(x_pub_check, ch_x_pub, 32) != 0) {
DEBUG_ERROR(DEBUG_CATEGORY_CHAT_SYNC, "%s: recv — key mismatch for ch=%s from 0x%016llx, drop",
CA_ID, ch_id, (unsigned long long)from_node);
queue_dgram_free(entry); queue_entry_free(entry); return;
}
if (topo_node_sqlite_channel_set_privs(cc->db, ch_id, x_priv, ed_priv) != 0) {
DEBUG_ERROR(DEBUG_CATEGORY_CHAT_SYNC, "%s: recv — set_privs FAILED ch=%s from 0x%016llx",
CA_ID, ch_id, (unsigned long long)from_node);
queue_dgram_free(entry); queue_entry_free(entry); return;
}
DEBUG_INFO(DEBUG_CATEGORY_CHAT_SYNC, "%s: received admin key ch=%s from 0x%016llx — imported, now co-owner",
CA_ID, ch_id, (unsigned long long)from_node);
uint8_t evt[72];
uint8_t cl = (uint8_t)strlen(ch_id);
evt[0] = cl; memcpy(evt + 1, ch_id, cl);
memcpy(evt + 1 + cl, &from_node, 8);
chat_event_post(inst, CHAT_EVT_ADMIN_KEY_RECEIVED, evt, 1 + cl + 8);
queue_dgram_free(entry); queue_entry_free(entry);
}
/* ─── жизненный цикл ─── */
int chat_admin_init(struct UTUN_INSTANCE* inst) {
if (!inst) return -1;
if (etcp_router_bind(inst, ETCP_RT_ID_ADMIN_KEY, admin_key_recv_cb) != 0) {
DEBUG_ERROR(DEBUG_CATEGORY_CHAT_SYNC, "%s: etcp_router_bind failed", CA_ID);
return -1;
}
DEBUG_INFO(DEBUG_CATEGORY_CHAT_SYNC, "%s: initialized", CA_ID);
return 0;
}
void chat_admin_destroy(struct UTUN_INSTANCE* inst) {
if (!inst) return;
etcp_router_unbind(inst, ETCP_RT_ID_ADMIN_KEY);
DEBUG_INFO(DEBUG_CATEGORY_CHAT_SYNC, "%s: destroyed", CA_ID);
}
/* ─── передача прав (вызывается из uasync-потока) ─── */
void chat_core_transfer_admin(struct UTUN_INSTANCE* inst, const char* ch_id, uint64_t target_node_id) {
struct chat_core_ctx* cc = CC(inst);
if (!cc || !cc->initialized || !inst || !cc->db) {
DEBUG_ERROR(DEBUG_CATEGORY_CHAT_SYNC, "%s: transfer — not initialized", CA_ID);
return;
}
if (!ch_id || !ch_id[0] || target_node_id == 0) {
DEBUG_ERROR(DEBUG_CATEGORY_CHAT_SYNC, "%s: transfer — invalid args ch=%s target=0x%016llx",
CA_ID, ch_id ? ch_id : "(null)", (unsigned long long)target_node_id);
return;
}
uint8_t x_priv[32], ed_priv[32];
if (topo_node_sqlite_channel_get_privs(cc->db, ch_id, x_priv, ed_priv) != 0) {
DEBUG_ERROR(DEBUG_CATEGORY_CHAT_SYNC, "%s: transfer — no channel privkey ch=%s (not owner?)", CA_ID, ch_id);
return;
}
uint64_t group_id = strtoull(ch_id, NULL, 10);
if (group_id == 0) {
DEBUG_ERROR(DEBUG_CATEGORY_CHAT_SYNC, "%s: transfer — invalid ch_id=%s", CA_ID, ch_id);
return;
}
uint8_t payload[8 + 32 + 32];
memcpy(payload, &group_id, 8);
memcpy(payload + 8, ed_priv, 32);
memcpy(payload + 8 + 32, x_priv, 32);
struct ll_entry* entry = queue_entry_new(0);
if (!entry) { DEBUG_ERROR(DEBUG_CATEGORY_CHAT_SYNC, "%s: transfer — queue_entry_new failed", CA_ID); return; }
entry->dgram = u_malloc(1 + sizeof(payload));
if (!entry->dgram) { queue_entry_free(entry); return; }
entry->dgram[0] = ETCP_RT_ID_ADMIN_KEY;
memcpy(entry->dgram + 1, payload, sizeof(payload));
entry->len = 1 + (uint16_t)sizeof(payload);
int rc = etcp_route_send(inst, group_id, target_node_id, entry, 1,
ROUTE_CRYPTO_SIGN | ROUTE_CRYPTO_ENCRYPT);
if (rc != 0) {
DEBUG_ERROR(DEBUG_CATEGORY_CHAT_SYNC, "%s: transfer — etcp_route_send FAILED rc=%d ch=%s → 0x%016llx",
CA_ID, rc, ch_id, (unsigned long long)target_node_id);
queue_dgram_free(entry); queue_entry_free(entry);
return;
}
DEBUG_INFO(DEBUG_CATEGORY_CHAT_SYNC, "%s: admin key sent ch=%s → 0x%016llx (encrypted+signed)",
CA_ID, ch_id, (unsigned long long)target_node_id);
}
void chat_core_transfer_admin_trampoline(void* arg) {
struct chat_transfer_admin_req* req = (struct chat_transfer_admin_req*)arg;
if (!req) return;
chat_core_transfer_admin(req->inst, req->ch_id, req->target_node_id);
u_free(req);
}
+28
View File
@@ -0,0 +1,28 @@
/*
* chat_admin.h — передача канального приватного ключа (прав админа) другому узлу
*
* Владелец канала (имеющий ed25519_privkey + x25519_privkey) может передать оба ключа
* выбранному мемберу. Передача идёт через сервисный пакет etcp_router с
* ROUTE_CRYPTO_SIGN | ROUTE_CRYPTO_ENCRYPT: X25519 ECDH + AES-128-CCM и Ed25519-подпись.
* Получатель автоматически импортирует ключи и становится совладельцем канала.
*/
#ifndef CHAT_ADMIN_H
#define CHAT_ADMIN_H
#include <stdint.h>
#ifdef __cplusplus
extern "C" {
#endif
struct UTUN_INSTANCE;
/* Зарегистрировать/снять обработчик сервиса ETCP_RT_ID_ADMIN_KEY. */
int chat_admin_init(struct UTUN_INSTANCE* inst);
void chat_admin_destroy(struct UTUN_INSTANCE* inst);
#ifdef __cplusplus
}
#endif
#endif /* CHAT_ADMIN_H */
+195 -48
View File
@@ -6,6 +6,7 @@
#include "chat_core_priv.h"
#include "chat_event.h"
#include "chat_setting.h"
#include "../utun_instance.h"
#include "../ntp_time.h"
@@ -13,7 +14,10 @@
#include "../routing_layer/topo_group.h"
#include "../routing_layer/topo_group_connect.h"
#include "member_sync.h"
#include "chat_sync.h"
#include "../transport_layer/secure_channel.h"
#include "../media_delivery/media_delivery.h"
#include "../media_delivery/media_index.h"
#include "../../lib/mem.h"
#include "../../lib/ll_queue.h"
#include "../../lib/platform_compat.h"
@@ -52,26 +56,28 @@ static int channel_sign_join(const uint8_t* ch_x25519, const uint8_t* ch_ed25519
/* ─── подготовка инфраструктуры канала (db_sync instance) ─── */
void chat_core_ensure_channel_ready(const char* ch_id) {
if (!g_cc.initialized || !ch_id || !ch_id[0]) return;
if (si_find(ch_id)) return;
void chat_core_ensure_channel_ready(struct UTUN_INSTANCE* inst, const char* ch_id) {
struct chat_core_ctx* cc = CC(inst);
if (!cc || !cc->initialized || !ch_id || !ch_id[0]) return;
if (si_find(inst, ch_id)) return;
char tbl_msg[80]; msg_table_name(ch_id, tbl_msg, sizeof(tbl_msg));
uint64_t ch_hash = 0;
{ const uint8_t* chd = (const uint8_t*)ch_id; size_t chl = strlen(ch_id); uint8_t sh[32]; SHA256(chd, chl, sh); memcpy(&ch_hash, sh, 8); }
struct DB_SYNC_INSTANCE* si = db_sync_instance_add(g_cc.inst, tbl_msg, ch_hash, 1);
uint64_t gid = strtoull(ch_id, NULL, 10);
if (gid != 0 && g_cc.inst->topo_groups && !topo_groups_find(g_cc.inst->topo_groups, gid)) {
struct TOPO_GROUP* g = topo_groups_create_group(g_cc.inst->topo_groups, gid, TOPO_GROUP_TYPE_CHAT, ch_id);
if (g) member_sync_subscribe_group(g_cc.inst, g);
struct DB_SYNC_INSTANCE* si = db_sync_instance_add(inst, tbl_msg, gid, 1);
if (gid != 0 && inst->topo_groups && !topo_groups_find(inst->topo_groups, gid)) {
struct TOPO_GROUP* g = topo_groups_create_group(inst->topo_groups, gid, TOPO_GROUP_TYPE_CHAT, ch_id);
if (g) member_sync_subscribe_group(inst, g);
}
if (si) {
si_register(si, ch_id);
db_sync_set_insert_cb(si, on_msg_inserted, u_strdup(ch_id));
DEBUG_INFO(DEBUG_CATEGORY_CHAT_SYNC, "%s: channel ready ch=%s tbl=%s hash=0x%016llx",
CC_ID, ch_id, tbl_msg, (unsigned long long)ch_hash);
si_register(inst, si, ch_id);
struct msg_insert_arg* ia = u_malloc(sizeof(*ia));
if (ia) { ia->inst = inst; snprintf(ia->ch_id, sizeof(ia->ch_id), "%s", ch_id); }
db_sync_set_insert_cb(si, on_msg_inserted, ia);
db_sync_instance_set_gated(si, 1);
DEBUG_INFO(DEBUG_CATEGORY_CHAT_SYNC, "%s: channel ready ch=%s tbl=%s group=0x%016llx",
CC_ID, ch_id, tbl_msg, (unsigned long long)gid);
} else {
DEBUG_ERROR(DEBUG_CATEGORY_CHAT_SYNC, "%s: db_sync_instance_add failed for ch=%s",
CC_ID, ch_id);
@@ -80,8 +86,9 @@ void chat_core_ensure_channel_ready(const char* ch_id) {
/* ─── создание канала ─── */
void chat_core_create_channel(struct chat_channel_create* req) {
if (!g_cc.initialized) {
void chat_core_create_channel(struct UTUN_INSTANCE* inst, struct chat_channel_create* req) {
struct chat_core_ctx* cc = CC(inst);
if (!cc || !cc->initialized) {
DEBUG_ERROR(DEBUG_CATEGORY_CHAT_SYNC, "%s: create_channel NOT INITIALIZED ch=%s",
CC_ID, req ? req->channel_id : "(null)");
return;
@@ -91,7 +98,7 @@ void chat_core_create_channel(struct chat_channel_create* req) {
DEBUG_INFO(DEBUG_CATEGORY_CHAT_SYNC, "%s: create_channel BEGIN ch=%s name=%s",
CC_ID, req->channel_id, req->name);
int rc = topo_node_sqlite_channel_put(g_cc.db,
int rc = topo_node_sqlite_channel_put(cc->db,
req->channel_id, req->name, req->owner_node_id,
req->x25519_pubkey, req->x25519_privkey,
req->ed25519_pubkey, req->ed25519_privkey,
@@ -104,7 +111,7 @@ void chat_core_create_channel(struct chat_channel_create* req) {
DEBUG_INFO(DEBUG_CATEGORY_CHAT_SYNC, "%s: channel_put OK ch=%s name=%s owner=0x%016llx",
CC_ID, req->channel_id, req->name, (unsigned long long)req->owner_node_id);
chat_core_ensure_channel_ready(req->channel_id);
chat_core_ensure_channel_ready(inst, req->channel_id);
uint8_t ch_id_len = (uint8_t)strlen(req->channel_id);
uint8_t data[65];
@@ -112,46 +119,53 @@ void chat_core_create_channel(struct chat_channel_create* req) {
memcpy(data + 1, req->channel_id, ch_id_len);
{
uint64_t myid = g_cc.inst->node_id;
uint64_t join_ts = (uint64_t)ntp_time_get_seconds(g_cc.inst);
uint64_t myid = inst->node_id;
uint64_t join_ts = (uint64_t)ntp_time_get_seconds(inst);
uint8_t join_sig[64];
channel_sign_join(req->x25519_pubkey, req->ed25519_pubkey,
myid, g_cc.inst->my_keys.public_key,
join_ts, g_cc.inst->my_ed25519_privkey, join_sig);
myid, inst->my_keys.public_key,
join_ts, inst->my_ed25519_privkey, join_sig);
char juser3[256]; snprintf(juser3, sizeof(juser3), "{\"name\":\"%s\"}",
g_cc.inst->name[0] ? g_cc.inst->name : "");
int mrc = member_sync_put(g_cc.inst, req->channel_id, myid,
g_cc.inst->my_keys.public_key, g_cc.inst->my_ed25519_pubkey,
join_sig, join_ts, NULL, 0, juser3, NULL, NULL, 0);
inst->name[0] ? inst->name : "");
/* корень дерева приглашений: подписан ключом группы (канальный ed25519) */
uint8_t root_sig[64]; memset(root_sig, 0, 64);
member_sync_sign_pubkey(req->ed25519_privkey, inst->my_keys.public_key, root_sig);
int mrc = member_sync_put(inst, req->channel_id, myid,
inst->my_keys.public_key, inst->my_ed25519_pubkey,
join_sig, join_ts, NULL, 0, juser3, NULL, NULL, 0, 0, root_sig);
if (mrc < 0) {
DEBUG_ERROR(DEBUG_CATEGORY_CHAT_SYNC, "%s: member_sync_put(self) FAILED ch=%s rc=%d",
CC_ID, req->channel_id, mrc);
} else if (mrc & MS_APPLY_CHANGED) {
member_sync_broadcast_one(g_cc.inst, req->channel_id, myid);
member_sync_broadcast_one(inst, req->channel_id, myid);
}
/* адреса + update_ts + подпись + синк — единой функцией */
chat_core_update_my_member();
chat_core_update_my_member(inst);
}
chat_event_post(CHAT_EVT_CHANNEL_UPDATED, data, 1 + ch_id_len);
chat_event_post(inst, CHAT_EVT_CHANNEL_UPDATED, data, 1 + ch_id_len);
}
}
void chat_core_create_channel_trampoline(void* arg) {
DEBUG_INFO(DEBUG_CATEGORY_CHAT_SYNC, "chat_core: TRAMPOLINE invoked arg=%p", arg);
struct chat_channel_create* req = (struct chat_channel_create*)arg;
chat_core_create_channel(req);
if (!req) return;
chat_core_create_channel(req->inst, req);
u_free(req);
}
void chat_core_create_channel_auto_trampoline(void* arg) {
chat_core_create_channel_auto((const char*)arg);
struct chat_create_auto_arg* a = (struct chat_create_auto_arg*)arg;
if (!a) return;
chat_core_create_channel_auto(a->inst, a->name);
u_free(arg);
}
void chat_core_create_channel_auto(const char* name) {
if (!g_cc.initialized || !name) {
void chat_core_create_channel_auto(struct UTUN_INSTANCE* inst, const char* name) {
struct chat_core_ctx* cc = CC(inst);
if (!cc || !cc->initialized || !name) {
DEBUG_ERROR(DEBUG_CATEGORY_CHAT_SYNC, "%s: create_channel_auto — not initialized or NULL name", CC_ID);
return;
}
@@ -169,6 +183,8 @@ void chat_core_create_channel_auto(const char* name) {
if (ed_pkey) { size_t pkLen = 32, skLen = 32; EVP_PKEY_get_raw_public_key(ed_pkey, ed25519_pub, &pkLen); EVP_PKEY_get_raw_private_key(ed_pkey, ed25519_priv, &skLen); } }
uint64_t chId = 0; RAND_bytes((uint8_t*)&chId, sizeof(chId));
chId &= 0x7FFFFFFFFFFFFFFFULL; /* как node_id — всегда < 2^63 (помещается в signed Long) */
if (chId == 0) chId = 1;
char channelId[64]; snprintf(channelId, sizeof(channelId), "%llu", (unsigned long long)chId);
uint8_t sig[64] = {0};
@@ -181,7 +197,7 @@ void chat_core_create_channel_auto(const char* name) {
uint8_t* p = msg;
memcpy(p, channelId, chIdLen); p += chIdLen; *p++ = '\0';
memcpy(p, name, nameLen); p += nameLen; *p++ = '\0';
memcpy(p, &g_cc.my_node_id, 8); p += 8;
memcpy(p, &cc->my_node_id, 8); p += 8;
memcpy(p, x25519_pub, 32); p += 32;
memcpy(p, ed25519_pub, 32);
EVP_MD_CTX* mdctx = EVP_MD_CTX_new();
@@ -193,31 +209,33 @@ void chat_core_create_channel_auto(const char* name) {
struct chat_channel_create* req = u_calloc(1, sizeof(*req));
if (!req) return;
req->inst = inst;
snprintf(req->channel_id, sizeof(req->channel_id), "%s", channelId);
snprintf(req->name, sizeof(req->name), "%s", name);
req->owner_node_id = g_cc.my_node_id;
req->owner_node_id = cc->my_node_id;
memcpy(req->x25519_pubkey, x25519_pub, 32); memcpy(req->x25519_privkey, x25519_priv, 32);
memcpy(req->ed25519_pubkey, ed25519_pub, 32); memcpy(req->ed25519_privkey, ed25519_priv, 32);
memcpy(req->signature, sig, 64);
chat_core_create_channel(req);
chat_core_create_channel(inst, req);
u_free(req);
}
/* ─── управление подключением к каналу (выбор в GUI) ─── */
void chat_core_connect_channel(const char* ch_id) {
DEBUG_TRACE(DEBUG_CATEGORY_CHAT_SYNC, "%s: connect_channel called ch_id=%s initialized=%d", CC_ID, ch_id ? ch_id : "(null)", g_cc.initialized);
if (!g_cc.initialized || !ch_id || !ch_id[0]) { DEBUG_WARN(DEBUG_CATEGORY_CHAT_SYNC, "%s: connect_channel — skip (not ready)", CC_ID); return; }
if (!g_cc.inst->topo_groups) { DEBUG_ERROR(DEBUG_CATEGORY_CHAT_SYNC, "%s: connect_channel — no topo_groups", CC_ID); return; }
void chat_core_connect_channel(struct UTUN_INSTANCE* inst, const char* ch_id) {
struct chat_core_ctx* cc = CC(inst);
DEBUG_TRACE(DEBUG_CATEGORY_CHAT_SYNC, "%s: connect_channel called ch_id=%s initialized=%d", CC_ID, ch_id ? ch_id : "(null)", cc ? cc->initialized : 0);
if (!cc || !cc->initialized || !ch_id || !ch_id[0]) { DEBUG_WARN(DEBUG_CATEGORY_CHAT_SYNC, "%s: connect_channel — skip (not ready)", CC_ID); return; }
if (!inst->topo_groups) { DEBUG_ERROR(DEBUG_CATEGORY_CHAT_SYNC, "%s: connect_channel — no topo_groups", CC_ID); return; }
uint64_t gid = strtoull(ch_id, NULL, 10);
if (gid == 0) { DEBUG_ERROR(DEBUG_CATEGORY_CHAT_SYNC, "%s: connect_channel — invalid ch_id=%s", CC_ID, ch_id); return; }
chat_core_ensure_channel_ready(ch_id);
chat_core_ensure_channel_ready(inst, ch_id);
struct TOPO_GROUP* group = topo_groups_find(g_cc.inst->topo_groups, gid);
if (!group) { DEBUG_ERROR(DEBUG_CATEGORY_CHAT_SYNC, "%s: connect_channel — group not found ch=%s gid=%016llx total_groups=%d", CC_ID, ch_id, (unsigned long long)gid, queue_entry_count(g_cc.inst->topo_groups->group_list)); return; }
struct TOPO_GROUP* group = topo_groups_find(inst->topo_groups, gid);
if (!group) { DEBUG_ERROR(DEBUG_CATEGORY_CHAT_SYNC, "%s: connect_channel — group not found ch=%s gid=%016llx total_groups=%d", CC_ID, ch_id, (unsigned long long)gid, queue_entry_count(inst->topo_groups->group_list)); return; }
if (group->connect) {
DEBUG_DEBUG(DEBUG_CATEGORY_CHAT_SYNC, "%s: connect_channel — already in progress ch=%s active=%d",
@@ -225,14 +243,143 @@ void chat_core_connect_channel(const char* ch_id) {
return;
}
if (!chat_setting_get_int(inst, "group_autoconnect", 1)) {
DEBUG_INFO(DEBUG_CATEGORY_CHAT_SYNC, "%s: connect_channel ch=%s — group_autoconnect disabled, skip auto-connect loop",
CC_ID, ch_id);
return;
}
DEBUG_INFO(DEBUG_CATEGORY_CHAT_SYNC, "%s: connect_channel ch=%s gid=%016llx group_type=%d — starting connect_init", CC_ID, ch_id, (unsigned long long)gid, group->group_type);
topo_group_connect_init(group);
}
void chat_core_connect_channel_trampoline(void* arg) {
char* ch_id = (char*)arg;
DEBUG_TRACE(DEBUG_CATEGORY_CHAT_SYNC, "%s: connect_channel_trampoline ch_id=%s", CC_ID, ch_id ? ch_id : "(null)");
if (!ch_id) return;
chat_core_connect_channel(ch_id);
u_free(ch_id);
struct chat_create_auto_arg* a = (struct chat_create_auto_arg*)arg;
DEBUG_TRACE(DEBUG_CATEGORY_CHAT_SYNC, "%s: connect_channel_trampoline ch_id=%s", CC_ID, a ? a->name : "(null)");
if (!a) return;
chat_core_connect_channel(a->inst, a->name);
u_free(a);
}
/* ─── однократное подключение к конкретному узлу канала (GUI → uasync) ─── */
void chat_core_connect_node(struct UTUN_INSTANCE* inst, const char* ch_id, uint64_t node_id) {
struct chat_core_ctx* cc = CC(inst);
if (!cc || !cc->initialized || !inst || !ch_id || !ch_id[0] || !node_id) {
DEBUG_WARN(DEBUG_CATEGORY_CHAT_SYNC, "%s: connect_node — skip (not ready or invalid args)", CC_ID);
return;
}
if (node_id == inst->node_id) {
DEBUG_WARN(DEBUG_CATEGORY_CHAT_SYNC, "%s: connect_node — skip self node=0x%016llx", CC_ID, (unsigned long long)node_id);
return;
}
chat_core_ensure_channel_ready(inst, ch_id);
uint64_t gid = strtoull(ch_id, NULL, 10);
struct TOPO_GROUP* group = topo_groups_find(inst->topo_groups, gid);
if (!group) {
DEBUG_ERROR(DEBUG_CATEGORY_CHAT_SYNC, "%s: connect_node — group not found ch=%s", CC_ID, ch_id);
return;
}
topo_group_connect_node_once(group, node_id);
}
void chat_core_connect_node_trampoline(void* arg) {
struct chat_connect_node_req* r = (struct chat_connect_node_req*)arg;
if (!r) return;
chat_core_connect_node(r->inst, r->ch_id, r->node_id);
u_free(r);
}
/* ─── применение настройки group_autoconnect ко всем CHAT-группам ─── */
void chat_core_apply_group_autoconnect(struct UTUN_INSTANCE* inst) {
int enabled = chat_setting_get_int(inst, "group_autoconnect", 1);
if (!inst || !inst->topo_groups || !inst->topo_groups->group_list) return;
struct ll_entry* e = inst->topo_groups->group_list->head;
while (e) {
struct TOPO_GROUP* g = (struct TOPO_GROUP*)e;
if (g->group_type == TOPO_GROUP_TYPE_CHAT) {
if (enabled && !g->connect) {
topo_group_connect_init(g);
} else if (!enabled && g->connect) {
DEBUG_INFO(DEBUG_CATEGORY_CHAT_SYNC, "%s: group_autoconnect OFF — stop auto-connect ch=%s", CC_ID, g->channel_id);
topo_group_connect_destroy(g);
}
}
e = e->next;
}
}
/* ─── локальное удаление канала ─── */
static const char* cc_ui_state_get(struct chat_core_ctx* cc, const char* key) {
static char val[512];
val[0] = '\0';
sqlite3_stmt* st = NULL;
if (sqlite3_prepare_v2(cc->db, "SELECT value FROM ui_state WHERE key=?", -1, &st, NULL) == SQLITE_OK) {
sqlite3_bind_text(st, 1, key, -1, SQLITE_STATIC);
if (sqlite3_step(st) == SQLITE_ROW) {
const char* v = (const char*)sqlite3_column_text(st, 0);
if (v) snprintf(val, sizeof(val), "%s", v);
}
sqlite3_finalize(st);
}
return val;
}
void chat_core_delete_channel(struct UTUN_INSTANCE* inst, const char* ch_id) {
struct chat_core_ctx* cc = CC(inst);
if (!cc || !cc->initialized || !cc->db || !inst) {
DEBUG_ERROR(DEBUG_CATEGORY_CHAT_SYNC, "%s: delete_channel — not initialized", CC_ID);
return;
}
if (!ch_id || !ch_id[0]) { DEBUG_ERROR(DEBUG_CATEGORY_CHAT_SYNC, "%s: delete_channel — empty ch_id", CC_ID); return; }
DEBUG_INFO(DEBUG_CATEGORY_CHAT_SYNC, "%s: delete_channel BEGIN ch=%s", CC_ID, ch_id);
uint64_t gid = strtoull(ch_id, NULL, 10);
/* 1. отмена медиа-загрузок/релэев/суперузлов канала (держат conn_mgr handle группы) */
media_delivery_cancel_channel(inst, gid);
/* 2. снять merkle-сессии этого namespace */
member_sync_cancel_channel(inst, ch_id);
/* 3. удалить группу (conn_mgr → закрытие соединений, connect-цикл, broadcast, recovery) */
if (inst->topo_groups)
topo_groups_remove_group(inst->topo_groups, gid);
/* 4. удалить db_sync инстанс (освобождает память, отменяет таймеры) */
{
struct DB_SYNC_INSTANCE* si = si_find(inst, ch_id);
if (si) db_sync_instance_remove(si);
}
/* 5. немедленно убрать канал из кеша chat_sync */
chat_sync_on_channel_deleted(inst, ch_id);
/* 6. удалить медиафайлы канала на диске + строки media_files */
media_index_delete_channel(cc->db, ch_id, cc_ui_state_get(cc, "media_base"));
/* 7. удалить таблицы и метаданные канала */
topo_node_sqlite_channel_delete(cc->db, ch_id);
/* 8. уведомить GUI */
{
uint8_t evt[65]; uint8_t cl = (uint8_t)strlen(ch_id);
evt[0] = cl; memcpy(evt + 1, ch_id, cl);
chat_event_post(inst, CHAT_EVT_CHANNEL_DELETED, evt, 1 + cl);
}
DEBUG_INFO(DEBUG_CATEGORY_CHAT_SYNC, "%s: delete_channel DONE ch=%s gid=0x%016llx", CC_ID, ch_id, (unsigned long long)gid);
}
void chat_core_delete_channel_trampoline(void* arg) {
struct chat_delete_channel_req* req = (struct chat_delete_channel_req*)arg;
if (!req) return;
chat_core_delete_channel(req->inst, req->ch_id);
u_free(req);
}
+104 -115
View File
@@ -1,5 +1,5 @@
/*
* chat_core.c — жизненный цикл чата: init, destroy, глобальное состояние g_cc
* chat_core.c — жизненный цикл чата: init, destroy, per-instance состояние (inst->chat_core)
*
* Под-модули (каждый в своём .c):
* chat_msg.c — отправка сообщений, DB-операции для chat_sync
@@ -28,10 +28,6 @@
#include <stdio.h>
#include "../../lib/platform_compat.h"
/* ─── глобальное состояние (extern в chat_core_priv.h) ─── */
struct chat_core_ctx g_cc;
/* ─── хелперы ─── */
void sanitize_ch_id(const char* ch_id, char* out, size_t out_sz) {
@@ -56,9 +52,9 @@ void peers_table_name(const char* ch_id, char* buf, size_t sz) {
snprintf(buf, sz, "peers_%s", san);
}
int db_exec(const char* sql) {
static int db_exec(struct chat_core_ctx* cc, const char* sql) {
char* err = NULL;
int rc = sqlite3_exec(g_cc.db, sql, NULL, NULL, &err);
int rc = sqlite3_exec(cc->db, sql, NULL, NULL, &err);
if (rc != SQLITE_OK) {
DEBUG_ERROR(DEBUG_CATEGORY_CHAT_SYNC, "%s: sql error: %s", CC_ID, err);
sqlite3_free(err);
@@ -68,23 +64,12 @@ int db_exec(const char* sql) {
/* ─── db_sync registry ─── */
struct DB_SYNC_INSTANCE* si_find(const char* ch_id) {
for (int i = 0; i < g_cc.si_count; i++)
if (strcmp(g_cc.si_ch_id[i], ch_id) == 0) return g_cc.si[i];
return NULL;
struct DB_SYNC_INSTANCE* si_find(struct UTUN_INSTANCE* inst, const char* ch_id) {
if (!inst || !inst->db_sync || !ch_id || !ch_id[0]) return NULL;
return db_sync_instance_find(inst, strtoull(ch_id, NULL, 10));
}
void si_register(struct DB_SYNC_INSTANCE* si, const char* ch_id) {
if (g_cc.si_count >= g_cc.si_capacity) {
int nc = g_cc.si_capacity ? g_cc.si_capacity * 2 : 8;
g_cc.si = u_realloc(g_cc.si, nc * sizeof(void*));
g_cc.si_ch_id = u_realloc(g_cc.si_ch_id, nc * sizeof(char*));
g_cc.si_capacity = nc;
}
g_cc.si[g_cc.si_count] = si;
g_cc.si_ch_id[g_cc.si_count] = u_strdup(ch_id);
g_cc.si_count++;
void si_register(struct UTUN_INSTANCE* inst, struct DB_SYNC_INSTANCE* si, const char* ch_id) {
uint32_t msg_cnt = db_sync_count(si);
DEBUG_INFO(DEBUG_CATEGORY_CHAT_SYNC, "%s: si_register ch=%s mc=%u", CC_ID, ch_id, msg_cnt);
}
@@ -93,41 +78,47 @@ void si_register(struct DB_SYNC_INSTANCE* si, const char* ch_id) {
int chat_core_init(struct UTUN_INSTANCE* inst, const char* db_path) {
if (!inst || !db_path) { DEBUG_ERROR(DEBUG_CATEGORY_CHAT_SYNC, "%s: invalid args inst=%p db_path=%p", CC_ID, (void*)inst, (void*)db_path); return -1; }
memset(&g_cc, 0, sizeof(g_cc));
snprintf(g_cc.db_path, sizeof(g_cc.db_path), "%s", db_path);
struct chat_core_ctx* cc = u_calloc(1, sizeof(*cc));
if (!cc) { DEBUG_ERROR(DEBUG_CATEGORY_CHAT_SYNC, "%s: OOM for chat_core_ctx", CC_ID); return -1; }
inst->chat_core = cc;
snprintf(cc->db_path, sizeof(cc->db_path), "%s", db_path);
if (inst->topo_sqlite_db) {
g_cc.db = inst->topo_sqlite_db; g_cc.shared_db = 1;
DEBUG_INFO(DEBUG_CATEGORY_CHAT_SYNC, "%s: using shared SQLite db=%p", CC_ID, (void*)g_cc.db);
cc->db = inst->topo_sqlite_db; cc->shared_db = 1;
DEBUG_INFO(DEBUG_CATEGORY_CHAT_SYNC, "%s: using shared SQLite db=%p", CC_ID, (void*)cc->db);
} else {
int rc = sqlite3_open_v2(db_path, &g_cc.db,
int rc = sqlite3_open_v2(db_path, &cc->db,
SQLITE_OPEN_READWRITE | SQLITE_OPEN_CREATE | SQLITE_OPEN_FULLMUTEX, NULL);
if (rc != SQLITE_OK) {
DEBUG_ERROR(DEBUG_CATEGORY_CHAT_SYNC, "%s: cannot open DB %s: %s",
CC_ID, db_path, sqlite3_errmsg(g_cc.db));
sqlite3_close(g_cc.db); g_cc.db = NULL; return -1;
CC_ID, db_path, sqlite3_errmsg(cc->db));
sqlite3_close(cc->db); inst->chat_core = NULL; u_free(cc); return -1;
}
sqlite3_exec(g_cc.db, "PRAGMA journal_mode=WAL", NULL, NULL, NULL);
sqlite3_exec(g_cc.db, "PRAGMA foreign_keys=ON", NULL, NULL, NULL);
sqlite3_exec(cc->db, "PRAGMA journal_mode=WAL", NULL, NULL, NULL);
sqlite3_exec(cc->db, "PRAGMA foreign_keys=ON", NULL, NULL, NULL);
}
g_cc.inst = inst;
g_cc.my_node_id = inst->node_id;
cc->inst = inst;
cc->my_node_id = inst->node_id;
/* перенести chat-настройки из конфига (заполнены парсером) в per-instance состояние */
if (inst->config) memcpy(&inst->chat_settings, &inst->config->global.chat_settings, sizeof(inst->chat_settings));
sqlite3_int64 now_sec = (sqlite3_int64)ntp_time_get_seconds(inst);
/* записать себя в nodes + local_identity */
{
const char* my_name = inst->name[0] ? inst->name : "Me";
topo_node_sqlite_node_update_verified(g_cc.db, g_cc.my_node_id,
topo_node_sqlite_node_update_verified(cc->db, cc->my_node_id,
my_name, inst->my_keys.public_key, inst->my_ed25519_pubkey,
(uint64_t)now_sec, (time_t)now_sec);
topo_node_sqlite_node_set_online(g_cc.db, g_cc.my_node_id, 1);
sqlite3_stmt* st = NULL;
sqlite3_prepare_v2(g_cc.db,
sqlite3_prepare_v2(cc->db,
"INSERT OR REPLACE INTO local_identity(id,node_id,name,x25519_pubkey,ed25519_pubkey,created_at,updated_at)"
" VALUES(1,?,?,?,?,?,?)", -1, &st, NULL);
if (st) {
sqlite3_bind_int64(st, 1, (sqlite3_int64)g_cc.my_node_id);
sqlite3_bind_int64(st, 1, (sqlite3_int64)cc->my_node_id);
sqlite3_bind_text(st, 2, my_name, -1, SQLITE_STATIC);
sqlite3_bind_blob(st, 3, inst->my_keys.public_key, 32, SQLITE_STATIC);
sqlite3_bind_blob(st, 4, inst->my_ed25519_pubkey, 32, SQLITE_STATIC);
@@ -137,15 +128,15 @@ int chat_core_init(struct UTUN_INSTANCE* inst, const char* db_path) {
}
}
g_cc.initialized = 1;
cc->initialized = 1;
/* register adm_tags change callback → CHAT_EVT_MEMBER_UPDATED */
chat_member_init();
chat_member_init(inst);
/* create media_files table */
media_index_init(g_cc.db);
media_index_init(cc->db);
db_exec(
db_exec(cc,
"CREATE TABLE IF NOT EXISTS local_identity ("
" id INTEGER PRIMARY KEY CHECK (id = 1),"
" node_id INTEGER NOT NULL UNIQUE,"
@@ -179,14 +170,14 @@ int chat_core_init(struct UTUN_INSTANCE* inst, const char* db_path) {
else snprintf(mbase, sizeof(mbase), "%s", db_path);
char sql[1024]; snprintf(sql, sizeof(sql),
"INSERT OR REPLACE INTO ui_state(key,value) VALUES('media_base','%s')", mbase);
sqlite3_exec(g_cc.db, sql, NULL, NULL, NULL);
sqlite3_exec(cc->db, sql, NULL, NULL, NULL);
DEBUG_INFO(DEBUG_CATEGORY_CHAT_SYNC, "%s: media_base=%s", CC_ID, mbase);
}
/* seed stub accounts if empty */
{
sqlite3_stmt* cnt_st = NULL;
if (sqlite3_prepare_v2(g_cc.db, "SELECT COUNT(*) FROM accounts",
if (sqlite3_prepare_v2(cc->db, "SELECT COUNT(*) FROM accounts",
-1, &cnt_st, NULL) == SQLITE_OK) {
if (sqlite3_step(cnt_st) == SQLITE_ROW && sqlite3_column_int(cnt_st, 0) == 0) {
DEBUG_INFO(DEBUG_CATEGORY_CHAT_SYNC, "%s: seeding 5 stub accounts", CC_ID);
@@ -197,7 +188,7 @@ int chat_core_init(struct UTUN_INSTANCE* inst, const char* db_path) {
};
for (int i = 0; i < 5; i++) {
sqlite3_stmt* ins = NULL;
sqlite3_prepare_v2(g_cc.db,
sqlite3_prepare_v2(cc->db,
"INSERT OR IGNORE INTO accounts(node_id,display_name,avatar_color,avatar_letter,created_at)"
" VALUES(?,?,?,?,?)", -1, &ins, NULL);
if (ins) {
@@ -217,12 +208,12 @@ int chat_core_init(struct UTUN_INSTANCE* inst, const char* db_path) {
/* ensure infrastructure for all existing channels (survives restart) */
{
sqlite3_stmt* stmt = NULL;
if (sqlite3_prepare_v2(g_cc.db, "SELECT channel_id FROM channels ORDER BY created_at ASC",
if (sqlite3_prepare_v2(cc->db, "SELECT channel_id FROM channels ORDER BY created_at ASC",
-1, &stmt, NULL) == SQLITE_OK) {
int loaded = 0;
while (sqlite3_step(stmt) == SQLITE_ROW) {
const char* ch = (const char*)sqlite3_column_text(stmt, 0);
if (ch && ch[0]) { chat_core_ensure_channel_ready(ch); loaded++; }
if (ch && ch[0]) { chat_core_ensure_channel_ready(inst, ch); loaded++; }
}
sqlite3_finalize(stmt);
DEBUG_INFO(DEBUG_CATEGORY_CHAT_SYNC, "%s: loaded %d channels from DB", CC_ID, loaded);
@@ -230,56 +221,46 @@ int chat_core_init(struct UTUN_INSTANCE* inst, const char* db_path) {
}
DEBUG_INFO(DEBUG_CATEGORY_CHAT_SYNC, "%s: initialized, db=%s node_id=0x%016llx",
CC_ID, db_path, (unsigned long long)g_cc.my_node_id);
CC_ID, db_path, (unsigned long long)cc->my_node_id);
{ uint8_t nid[8]; memcpy(nid, &g_cc.my_node_id, 8); chat_event_post(CHAT_EVT_MY_NODE_ID, nid, 8); }
{ uint8_t nid[8]; memcpy(nid, &cc->my_node_id, 8); chat_event_post(inst, CHAT_EVT_MY_NODE_ID, nid, 8); }
chat_event_post(CHAT_EVT_DB_READY, NULL, 0);
chat_event_post(inst, CHAT_EVT_DB_READY, NULL, 0);
return 0;
}
void chat_core_destroy(struct UTUN_INSTANCE* inst) {
(void)inst;
if (!g_cc.initialized) return;
g_cc.initialized = 0;
struct chat_core_ctx* cc = CC(inst);
if (!cc || !cc->initialized) return;
cc->initialized = 0;
for (int i = 0; i < g_cc.si_count; i++) u_free(g_cc.si_ch_id[i]);
u_free(g_cc.si_ch_id); u_free(g_cc.si);
g_cc.si = NULL; g_cc.si_ch_id = NULL; g_cc.si_count = g_cc.si_capacity = 0;
if (g_cc.db && !g_cc.shared_db) { sqlite3_close(g_cc.db); }
g_cc.db = NULL; g_cc.inst = NULL;
if (cc->db && !cc->shared_db) { sqlite3_close(cc->db); }
inst->chat_core = NULL;
u_free(cc);
DEBUG_INFO(DEBUG_CATEGORY_CHAT_SYNC, "%s: destroyed", CC_ID);
}
sqlite3* chat_core_get_db(void) { return g_cc.db; }
struct UTUN_INSTANCE* chat_core_get_inst(void) { return g_cc.inst; }
int chat_core_is_initialized(void) { return g_cc.initialized; }
sqlite3* chat_core_get_db(struct UTUN_INSTANCE* inst) { return CC(inst) ? CC(inst)->db : NULL; }
int chat_core_is_initialized(struct UTUN_INSTANCE* inst) { return CC(inst) ? CC(inst)->initialized : 0; }
/* ─── chat settings (GUI → uasync) ─── */
void chat_core_set_setting(const char* name, const char* value) {
chat_setting_set(name, value);
void chat_core_set_setting(struct UTUN_INSTANCE* inst, const char* name, const char* value) {
chat_setting_set(inst, name, value);
if (name && strcmp(name, "group_autoconnect") == 0) chat_core_apply_group_autoconnect(inst);
}
struct chat_setting_arg { char data[256]; };
void chat_core_set_setting_trampoline(void* arg) {
struct chat_setting_arg* a = (struct chat_setting_arg*)arg;
const char* name = a->data;
const char* value = name + strlen(name) + 1;
chat_core_set_setting(name, value);
chat_core_set_setting(a->inst, name, value);
u_free(arg);
}
/* ─── headless API ─── */
void chat_core_set_ctx_for_headless(sqlite3* db, struct UTUN_INSTANCE* inst, uint64_t my_node_id) {
if (!g_cc.initialized) g_cc.initialized = 1;
g_cc.db = db; g_cc.inst = inst; g_cc.my_node_id = my_node_id;
g_cc.shared_db = 1;
}
static void json_escape(const char* src, char* dst, size_t dst_sz) {
size_t i = 0; const char* s = src;
while (*s && i < dst_sz - 2) {
@@ -289,12 +270,13 @@ static void json_escape(const char* src, char* dst, size_t dst_sz) {
dst[i] = '\0';
}
int chat_core_get_channels_json(char* buf, size_t buf_size, size_t* out_len) {
if (!g_cc.initialized || !buf || !out_len) return -1;
int chat_core_get_channels_json(struct UTUN_INSTANCE* inst, char* buf, size_t buf_size, size_t* out_len) {
struct chat_core_ctx* cc = CC(inst);
if (!cc || !cc->initialized || !buf || !out_len) return -1;
char* w = buf; char* end = buf + buf_size;
*w++ = '['; const char* sep = "";
sqlite3_stmt* st = NULL;
if (sqlite3_prepare_v2(g_cc.db,
if (sqlite3_prepare_v2(cc->db,
"SELECT c.channel_id, c.name, c.owner_node_id, "
"(SELECT COUNT(*) FROM nodes WHERE node_id=c.owner_node_id AND name IS NOT NULL) AS has_owner_name, "
"(SELECT name FROM nodes WHERE node_id=c.owner_node_id) AS owner_name "
@@ -307,14 +289,17 @@ int chat_core_get_channels_json(char* buf, size_t buf_size, size_t* out_len) {
const char* owner_name = (const char*)sqlite3_column_text(st, 4);
if (!ch_id) continue;
uint32_t msg_count = chat_core_count(ch_id);
uint32_t msg_count = chat_core_count(inst, ch_id);
char peers_tbl[80]; peers_table_name(ch_id, peers_tbl, sizeof(peers_tbl));
int peer_count = 0, online_count = 0;
sqlite3_stmt* ps = NULL;
char psql[200]; snprintf(psql, sizeof(psql),
"SELECT COUNT(*), SUM(CASE WHEN n.online=1 THEN 1 ELSE 0 END) FROM \"%s\" p LEFT JOIN nodes n ON p.node_id=n.node_id", peers_tbl);
if (sqlite3_prepare_v2(g_cc.db, psql, -1, &ps, NULL) == SQLITE_OK) {
if (sqlite3_step(ps) == SQLITE_ROW) { peer_count = sqlite3_column_int(ps, 0); online_count = sqlite3_column_int(ps, 1); }
char psql[200]; snprintf(psql, sizeof(psql), "SELECT node_id FROM \"%s\" WHERE deleted=0", peers_tbl);
if (sqlite3_prepare_v2(cc->db, psql, -1, &ps, NULL) == SQLITE_OK) {
while (sqlite3_step(ps) == SQLITE_ROW) {
uint64_t pid = (uint64_t)sqlite3_column_int64(ps, 0);
peer_count++;
if (chat_core_member_online(inst, ch_id, pid)) online_count++;
}
sqlite3_finalize(ps);
}
@@ -335,9 +320,10 @@ int chat_core_get_channels_json(char* buf, size_t buf_size, size_t* out_len) {
return 0;
}
int chat_core_get_messages_json(const char* ch_id, int count, int offset,
int chat_core_get_messages_json(struct UTUN_INSTANCE* inst, const char* ch_id, int count, int offset,
char* buf, size_t buf_size, size_t* out_len) {
if (!g_cc.initialized || !ch_id || !buf || !out_len) return -1;
struct chat_core_ctx* cc = CC(inst);
if (!cc || !cc->initialized || !ch_id || !buf || !out_len) return -1;
char tbl[80]; msg_table_name(ch_id, tbl, sizeof(tbl));
char* w = buf; char* end = buf + buf_size;
*w++ = '['; const char* sep = "";
@@ -347,7 +333,7 @@ int chat_core_get_messages_json(const char* ch_id, int count, int offset,
"FROM \"%s\" m LEFT JOIN nodes n ON m.node_id=n.node_id "
"ORDER BY m.timestamp DESC LIMIT ? OFFSET ?", tbl);
sqlite3_stmt* st = NULL;
if (sqlite3_prepare_v2(g_cc.db, sql, -1, &st, NULL) != SQLITE_OK) { *out_len = 0; return -1; }
if (sqlite3_prepare_v2(cc->db, sql, -1, &st, NULL) != SQLITE_OK) { *out_len = 0; return -1; }
sqlite3_bind_int(st, 1, count > 0 ? count : 100);
sqlite3_bind_int(st, 2, offset);
while (sqlite3_step(st) == SQLITE_ROW) {
@@ -379,26 +365,27 @@ int chat_core_get_messages_json(const char* ch_id, int count, int offset,
return 0;
}
int chat_core_get_members_json(const char* ch_id, char* buf, size_t buf_size, size_t* out_len) {
if (!g_cc.initialized || !ch_id || !buf || !out_len) return -1;
int chat_core_get_members_json(struct UTUN_INSTANCE* inst, const char* ch_id, char* buf, size_t buf_size, size_t* out_len) {
struct chat_core_ctx* cc = CC(inst);
if (!cc || !cc->initialized || !ch_id || !buf || !out_len) return -1;
char peers_tbl[80]; peers_table_name(ch_id, peers_tbl, sizeof(peers_tbl));
char* w = buf; char* end = buf + buf_size;
*w++ = '['; const char* sep = "";
char sql[256];
snprintf(sql, sizeof(sql),
"SELECT p.node_id, n.name, n.online, n.x25519_pubkey, n.ed25519_pubkey, p.adm_tags "
"SELECT p.node_id, n.name, n.x25519_pubkey, n.ed25519_pubkey, p.adm_tags "
"FROM \"%s\" p LEFT JOIN nodes n ON p.node_id=n.node_id", peers_tbl);
sqlite3_stmt* st = NULL;
if (sqlite3_prepare_v2(g_cc.db, sql, -1, &st, NULL) != SQLITE_OK) { *out_len = 0; return -1; }
if (sqlite3_prepare_v2(cc->db, sql, -1, &st, NULL) != SQLITE_OK) { *out_len = 0; return -1; }
while (sqlite3_step(st) == SQLITE_ROW) {
uint64_t nid = (uint64_t)sqlite3_column_int64(st, 0);
const char* name = (const char*)sqlite3_column_text(st, 1);
int online = sqlite3_column_int(st, 2);
const uint8_t* x25519 = (const uint8_t*)sqlite3_column_blob(st, 3);
const uint8_t* ed25519 = sqlite3_column_blob(st, 4);
int x25519_len = sqlite3_column_bytes(st, 3);
int ed25519_len = sqlite3_column_bytes(st, 4);
const char* adm_tags = (const char*)sqlite3_column_text(st, 5);
int online = chat_core_member_online(inst, ch_id, nid);
const uint8_t* x25519 = (const uint8_t*)sqlite3_column_blob(st, 2);
const uint8_t* ed25519 = sqlite3_column_blob(st, 3);
int x25519_len = sqlite3_column_bytes(st, 2);
int ed25519_len = sqlite3_column_bytes(st, 3);
const char* adm_tags = (const char*)sqlite3_column_text(st, 4);
char esc_name[256], x25519_hex[65], ed25519_hex[65], esc_tags[384];
json_escape(name ? name : "", esc_name, sizeof(esc_name));
x25519_hex[0] = ed25519_hex[0] = '\0';
@@ -407,8 +394,8 @@ int chat_core_get_members_json(const char* ch_id, char* buf, size_t buf_size, si
/* check connected */
int connected = 0;
if (g_cc.inst && g_cc.inst->connections) {
struct ll_entry* e = queue_find_data_by_index(g_cc.inst->connections, (const uint8_t*)&nid);
if (inst && inst->connections) {
struct ll_entry* e = queue_find_data_by_index(inst->connections, (const uint8_t*)&nid);
if (e) connected = 1;
}
@@ -424,7 +411,7 @@ int chat_core_get_members_json(const char* ch_id, char* buf, size_t buf_size, si
sqlite3_stmt* as = NULL;
char asql[200]; snprintf(asql, sizeof(asql),
"SELECT family, protocol, address, port, rtt FROM node_addresses WHERE node_id=? ORDER BY family, protocol");
if (sqlite3_prepare_v2(g_cc.db, asql, -1, &as, NULL) == SQLITE_OK) {
if (sqlite3_prepare_v2(cc->db, asql, -1, &as, NULL) == SQLITE_OK) {
sqlite3_bind_int64(as, 1, (sqlite3_int64)nid);
const char* asep2 = "";
while (sqlite3_step(as) == SQLITE_ROW) {
@@ -458,11 +445,12 @@ int chat_core_get_members_json(const char* ch_id, char* buf, size_t buf_size, si
return 0;
}
int chat_core_get_node_name(uint64_t node_id, char* out, size_t sz) {
if (!g_cc.initialized || !out) return -1;
int chat_core_get_node_name(struct UTUN_INSTANCE* inst, uint64_t node_id, char* out, size_t sz) {
struct chat_core_ctx* cc = CC(inst);
if (!cc || !cc->initialized || !out) return -1;
out[0] = '\0';
sqlite3_stmt* st = NULL;
if (sqlite3_prepare_v2(g_cc.db, "SELECT name FROM nodes WHERE node_id=?", -1, &st, NULL) != SQLITE_OK) return -1;
if (sqlite3_prepare_v2(cc->db, "SELECT name FROM nodes WHERE node_id=?", -1, &st, NULL) != SQLITE_OK) return -1;
sqlite3_bind_int64(st, 1, (sqlite3_int64)node_id);
if (sqlite3_step(st) == SQLITE_ROW) {
const char* n = (const char*)sqlite3_column_text(st, 0);
@@ -509,8 +497,8 @@ static int tags_is_append_key(const char* key) {
static void tags_set_effective_val(struct chat_member_tags_key* k, const char* raw) {
uint64_t ts = 0;
const char* eff = tags_append_effective(raw, &ts);
if (ts == 0) ts = (uint64_t)ntp_time_get_seconds(g_cc.inst);
snprintf(k->val, CHAT_MEMBER_TAGS_VAL_SZ, "%s:%llu", eff, (unsigned long long)ts);
snprintf(k->val, CHAT_MEMBER_TAGS_VAL_SZ, "%s", eff);
(void)ts;
}
static int tags_json_cb(const char* key, const char* json_val, void* arg) {
@@ -520,13 +508,13 @@ static int tags_json_cb(const char* key, const char* json_val, void* arg) {
snprintf(k->key, CHAT_MEMBER_TAGS_KEY_SZ, "%s", key);
snprintf(k->val, CHAT_MEMBER_TAGS_VAL_SZ, "%s", json_val);
k->raw = tags_is_append_key(key) ? u_strdup(json_val) : NULL;
if (k->raw) tags_set_effective_val(k, json_val);
t->key_count++;
return 0;
}
struct chat_member_tags* chat_member_tags_load(const char* ch_id, uint64_t node_id) {
if (!g_cc.initialized || !ch_id || !g_cc.db) { DEBUG_ERROR(DEBUG_CATEGORY_CHAT_SYNC, "%s: tags_load — not initialized", CC_ID); return NULL; }
struct chat_member_tags* chat_member_tags_load(struct UTUN_INSTANCE* inst, const char* ch_id, uint64_t node_id) {
struct chat_core_ctx* cc = CC(inst);
if (!cc || !cc->initialized || !ch_id || !cc->db) { DEBUG_ERROR(DEBUG_CATEGORY_CHAT_SYNC, "%s: tags_load — not initialized", CC_ID); return NULL; }
struct chat_member_tags* t = u_calloc(1, sizeof(*t));
if (!t) { DEBUG_ERROR(DEBUG_CATEGORY_CHAT_SYNC, "%s: tags_load — OOM", CC_ID); return NULL; }
snprintf(t->ch_id, sizeof(t->ch_id), "%s", ch_id);
@@ -535,7 +523,7 @@ struct chat_member_tags* chat_member_tags_load(const char* ch_id, uint64_t node_
char peers_tbl[80]; peers_table_name(ch_id, peers_tbl, sizeof(peers_tbl));
sqlite3_stmt* st = NULL;
char sql[256]; snprintf(sql, sizeof(sql), "SELECT adm_tags FROM \"%s\" WHERE node_id=?", peers_tbl);
if (sqlite3_prepare_v2(g_cc.db, sql, -1, &st, NULL) == SQLITE_OK) {
if (sqlite3_prepare_v2(cc->db, sql, -1, &st, NULL) == SQLITE_OK) {
sqlite3_bind_int64(st, 1, (sqlite3_int64)node_id);
if (sqlite3_step(st) == SQLITE_ROW) {
const char* tags = (const char*)sqlite3_column_text(st, 0);
@@ -588,8 +576,9 @@ int chat_member_tags_replace(struct chat_member_tags* t, const char* json) {
return 0;
}
int chat_member_tags_commit(struct chat_member_tags* t) {
if (!t || !g_cc.initialized || !g_cc.inst || !g_cc.db) {
int chat_member_tags_commit(struct UTUN_INSTANCE* inst, struct chat_member_tags* t) {
struct chat_core_ctx* cc = CC(inst);
if (!t || !cc || !cc->initialized || !cc->inst || !cc->db) {
DEBUG_ERROR(DEBUG_CATEGORY_CHAT_SYNC, "%s: tags_commit — invalid state", CC_ID); return -1;
}
char json[4096]; int off = 0, ver = 1;
@@ -609,7 +598,7 @@ int chat_member_tags_commit(struct chat_member_tags* t) {
int old_yes = (*old_eff == 'y');
int new_yes = (k->val[0] == 'y'); /* val = "yes" or "yes:<ts>" */
if (new_yes != old_yes) {
uint64_t ts = (uint64_t)ntp_time_get_seconds(g_cc.inst);
uint64_t ts = (uint64_t)ntp_time_get_seconds(cc->inst);
size_t oldl = strlen(k->raw);
char* nraw = u_malloc(oldl + 24);
if (!nraw) { DEBUG_ERROR(DEBUG_CATEGORY_CHAT_SYNC, "%s: tags_commit — OOM for raw", CC_ID); return -1; }
@@ -628,7 +617,7 @@ int chat_member_tags_commit(struct chat_member_tags* t) {
#undef TAGS_ADVANCE
uint8_t ch_ed_priv[32];
if (topo_node_sqlite_channel_get_priv(g_cc.db, t->ch_id, ch_ed_priv) != 0) {
if (topo_node_sqlite_channel_get_priv(cc->db, t->ch_id, ch_ed_priv) != 0) {
DEBUG_ERROR(DEBUG_CATEGORY_CHAT_SYNC, "%s: tags_commit — no channel privkey for ch=%s (not owner?)", CC_ID, t->ch_id); return -1;
}
uint8_t sign_msg[8192]; size_t mlen = (size_t)off;
@@ -645,7 +634,7 @@ int chat_member_tags_commit(struct chat_member_tags* t) {
char sql[384]; snprintf(sql, sizeof(sql),
"SELECT x25519_pubkey, ed25519_pubkey, join_sig, join_ts, update_sig, update_ts, userinfo, storage"
" FROM \"%s\" WHERE node_id=?", peers_tbl);
if (sqlite3_prepare_v2(g_cc.db, sql, -1, &st, NULL) != SQLITE_OK) {
if (sqlite3_prepare_v2(cc->db, sql, -1, &st, NULL) != SQLITE_OK) {
DEBUG_ERROR(DEBUG_CATEGORY_CHAT_SYNC, "%s: tags_commit — member query failed", CC_ID); return -1;
}
sqlite3_bind_int64(st, 1, (sqlite3_int64)t->node_id);
@@ -666,11 +655,11 @@ int chat_member_tags_commit(struct chat_member_tags* t) {
const char* stg_val = chat_member_tags_get(t, "storage");
if (stg_val && strcmp(stg_val, "yes") == 0) stg = 1;
int rc = member_sync_put(g_cc.inst, t->ch_id, t->node_id, x25, ed, jsig, jts, usig, uts,
uinfo ? uinfo : "", json, sig, stg);
int rc = member_sync_put(inst, t->ch_id, t->node_id, x25, ed, jsig, jts, usig, uts,
uinfo ? uinfo : "", json, sig, stg, 0, NULL);
sqlite3_finalize(st);
if (rc < 0) { DEBUG_ERROR(DEBUG_CATEGORY_CHAT_SYNC, "%s: tags_commit — member_sync_put failed rc=%d", CC_ID, rc); return -1; }
if (rc & MS_APPLY_CHANGED) member_sync_broadcast_one(g_cc.inst, t->ch_id, t->node_id);
if (rc & MS_APPLY_CHANGED) member_sync_broadcast_one(inst, t->ch_id, t->node_id);
DEBUG_INFO(DEBUG_CATEGORY_CHAT_SYNC, "%s: tags_commit ch=%s nid=0x%016llx ver=%d keys=%d json=%s",
CC_ID, t->ch_id, (unsigned long long)t->node_id, ver, t->key_count, json);
@@ -687,11 +676,11 @@ void chat_member_tags_free(struct chat_member_tags* t) {
void chat_member_tags_commit_trampoline(void* arg) {
struct chat_member_tags_req* req = (struct chat_member_tags_req*)arg;
if (!req || !g_cc.initialized) { u_free(arg); return; }
struct chat_member_tags* t = chat_member_tags_load(req->ch_id, req->node_id);
if (!req || !CC(req->inst) || !CC(req->inst)->initialized) { u_free(arg); return; }
struct chat_member_tags* t = chat_member_tags_load(req->inst, req->ch_id, req->node_id);
if (!t) { u_free(arg); return; }
if (req->json[0]) chat_member_tags_replace(t, req->json);
chat_member_tags_commit(t);
chat_member_tags_commit(req->inst, t);
chat_member_tags_free(t);
u_free(arg);
}
+114 -46
View File
@@ -2,7 +2,9 @@
* chat_core.h — центральный API чата в потоке uasync
*
* Все DB-операции и сетевые функции выполняются в uasync-потоке.
* GUI вызывает chat_core_submit_message() через gui_bridge_post_uasync().
* Все функции per-instance: первым аргументом struct UTUN_INSTANCE*.
* GUI вызывает через gui_bridge_post_uasync() trampoline-функции,
* чей arg-структура несёт inst.
*/
#ifndef CHAT_CORE_H
@@ -20,18 +22,35 @@ struct UTUN_INSTANCE;
int chat_core_init(struct UTUN_INSTANCE* inst, const char* db_path);
void chat_core_destroy(struct UTUN_INSTANCE* inst);
struct sqlite3* chat_core_get_db(void);
struct UTUN_INSTANCE* chat_core_get_inst(void);
int chat_core_is_initialized(void);
/* Backfill: регистрирует уже скачанные медиафайлы (лежат на диске, но отсутствуют
* в media_files) — вызывается после загрузки каналов/сообщений при старте. */
void chat_media_backfill(struct UTUN_INSTANCE* inst);
/* Докачка недостающего медиа за последние storage_backfill_days дней. Проходит по
* медиасообщениям каналов и запускает загрузку для тех, чей файл отсутствует на диске. */
void chat_media_autodownload_backfill(struct UTUN_INSTANCE* inst);
/* Регистрирует триггер: после первого SYNC_DONE (+ дебаунс 2с) анонсирует локальные
* блоки суперузлам и запускает chat_media_autodownload_backfill. Один раз за процесс. */
void chat_media_startup_backfill_init(struct UTUN_INSTANCE* inst);
void chat_media_startup_backfill_destroy(struct UTUN_INSTANCE* inst);
struct sqlite3* chat_core_get_db(struct UTUN_INSTANCE* inst);
int chat_core_is_initialized(struct UTUN_INSTANCE* inst);
/* ── Отправка сообщения (GUI → uasync) ── */
struct chat_msg_submit {
struct UTUN_INSTANCE* inst;
char channel_id[64];
char content_type[32];
char media_src[1024];
char media_dest[1024];
uint8_t media_copy;
uint8_t media_video; /* 1 = подготовить как видео (пробинг+транскод) */
int duration_ms; /* видео: длительность (0 = неизвестно) */
int width; /* видео: ширина */
int height; /* видео: высота */
uint8_t* data;
uint32_t data_len;
uint64_t timestamp;
@@ -39,19 +58,23 @@ struct chat_msg_submit {
uint64_t reply_to_node_id; /* node_id автора исходного сообщения */
};
void chat_core_submit_message(struct chat_msg_submit* req);
void chat_core_submit_message(struct UTUN_INSTANCE* inst, struct chat_msg_submit* req);
/* Отправка медиа-сообщения (вложение): src->dest (copy) или видео (transcode). */
void chat_core_submit_media_message(struct UTUN_INSTANCE* inst, struct chat_msg_submit* req);
/* ── DB-операции (оставлены для интроспекции) ── */
uint32_t chat_core_count(const char* ch_id);
int chat_core_chain_hash_at(const char* ch_id, uint32_t pos, uint8_t* hash_out);
int chat_core_list_channels(uint8_t* buf, size_t buf_size, size_t* out_len);
int chat_core_list_peers(const char* ch_id, uint8_t* buf, size_t buf_size, size_t* out_len);
int chat_core_load_nodeinfo(uint64_t node_id, uint8_t* buf, size_t buf_size, size_t* out_len);
uint32_t chat_core_count(struct UTUN_INSTANCE* inst, const char* ch_id);
int chat_core_chain_hash_at(struct UTUN_INSTANCE* inst, const char* ch_id, uint32_t pos, uint8_t* hash_out);
int chat_core_list_channels(struct UTUN_INSTANCE* inst, uint8_t* buf, size_t buf_size, size_t* out_len);
int chat_core_list_peers(struct UTUN_INSTANCE* inst, const char* ch_id, uint8_t* buf, size_t buf_size, size_t* out_len);
int chat_core_load_nodeinfo(struct UTUN_INSTANCE* inst, uint64_t node_id, uint8_t* buf, size_t buf_size, size_t* out_len);
/* ── Создание канала (GUI → uasync) ── */
struct chat_channel_create {
struct UTUN_INSTANCE* inst;
char channel_id[64];
char name[128];
uint64_t owner_node_id;
@@ -62,90 +85,122 @@ struct chat_channel_create {
uint8_t signature[64];
};
void chat_core_ensure_channel_ready(const char* ch_id);
void chat_core_create_channel(struct chat_channel_create* req);
struct chat_create_auto_arg { struct UTUN_INSTANCE* inst; char name[128]; };
void chat_core_ensure_channel_ready(struct UTUN_INSTANCE* inst, const char* ch_id);
void chat_core_create_channel(struct UTUN_INSTANCE* inst, struct chat_channel_create* req);
void chat_core_create_channel_trampoline(void* arg);
void chat_core_create_channel_auto(const char* name);
void chat_core_create_channel_auto(struct UTUN_INSTANCE* inst, const char* name);
void chat_core_create_channel_auto_trampoline(void* arg);
/* ── Локальное удаление канала (GUI → uasync) ── */
struct chat_delete_channel_req {
struct UTUN_INSTANCE* inst;
char ch_id[64];
};
/* Удалить канал с этого устройства: отмена медиа-загрузок, удаление группы
* (соединения/синк), db_sync инстанса, таблиц msg_ и peers_, медиафайлов и
* merkle-хешей. Работает в uasync-потоке. Другие узлы канал не теряют. */
void chat_core_delete_channel(struct UTUN_INSTANCE* inst, const char* ch_id);
void chat_core_delete_channel_trampoline(void* arg);
/* ── Утилиты ── */
void chat_core_set_my_node_id(uint64_t node_id);
void chat_core_update_my_name(const char* name);
void chat_core_set_my_node_id(struct UTUN_INSTANCE* inst, uint64_t node_id);
void chat_core_update_my_name(struct UTUN_INSTANCE* inst, const char* name);
struct update_my_name_arg { struct UTUN_INSTANCE* inst; char name[128]; };
void chat_core_update_my_name_trampoline(void* arg);
void chat_core_sync_my_addresses(void);
void chat_core_sync_my_addresses(struct UTUN_INSTANCE* inst);
/* Единое обновление своего мембера: адреса + bump update_ts + подпись (с адресами) + синк */
void chat_core_update_my_member(void);
void chat_core_update_my_member(struct UTUN_INSTANCE* inst);
/* Сохранение key-value в ui_state (вызывается из uasync-потока) */
void chat_core_save_ui_state(const char* key, const char* value);
void chat_core_save_ui_state(struct UTUN_INSTANCE* inst, const char* key, const char* value);
struct save_ui_state_arg { struct UTUN_INSTANCE* inst; char data[256]; };
void chat_core_save_ui_state_trampoline(void* arg);
/* Установка chat-настройки (name=value) через chat_setting API (GUI → uasync) */
void chat_core_set_setting(const char* name, const char* value);
void chat_core_set_setting(struct UTUN_INSTANCE* inst, const char* name, const char* value);
struct chat_setting_arg { struct UTUN_INSTANCE* inst; char data[256]; };
void chat_core_set_setting_trampoline(void* arg);
/* Запуск подключения к узлам выбранного канала (GUI → uasync) */
void chat_core_connect_channel(const char* ch_id);
void chat_core_connect_channel(struct UTUN_INSTANCE* inst, const char* ch_id);
void chat_core_connect_channel_trampoline(void* arg);
/* Однократная попытка подключения к конкретному узлу канала (GUI → uasync) */
void chat_core_connect_node(struct UTUN_INSTANCE* inst, const char* ch_id, uint64_t node_id);
void chat_core_connect_node_trampoline(void* arg);
struct chat_connect_node_req {
struct UTUN_INSTANCE* inst;
char ch_id[64];
uint64_t node_id;
};
/* Трамплин для gui_bridge_post_uasync (GUI → uasync) */
void chat_core_submit_trampoline(void* arg);
/* Трамплин для ручного запуска загрузки медиа (JNI → uasync) */
struct attachment_dl_req {
struct UTUN_INSTANCE* inst;
char channel_id[64];
int64_t msg_id;
};
void chat_core_attachment_download_trampoline(void* arg);
/* Обновление local_attrs для записи (вызывается из uasync-потока) */
int chat_core_update_local_attrs(const char* ch_id, uint64_t ts,
int chat_core_update_local_attrs(struct UTUN_INSTANCE* inst, const char* ch_id, uint64_t ts,
const uint8_t* author_sig,
const char* local_attrs_json);
/* Пометить голосовое сообщение как проигранное (добавляет "pl":1 в local_attrs) */
void chat_core_mark_voice_played(const char* channel_id, uint64_t ts,
void chat_core_mark_voice_played(struct UTUN_INSTANCE* inst, const char* channel_id, uint64_t ts,
uint64_t author_node_id);
void chat_core_mark_voice_played_trampoline(void* arg);
struct voice_played_req {
struct UTUN_INSTANCE* inst;
char channel_id[64];
uint64_t ts;
uint64_t author_node_id;
};
/* Сбор статуса (NTP + connections) и отправка в GUI */
/* Сбор статуса (NTP + connections) и отправка в GUI. arg = struct UTUN_INSTANCE* (inst) */
void chat_core_collect_status_trampoline(void* arg);
/* Сбор списка ETCP-соединений и отправка в GUI (binary packet) */
/* Сбор списка ETCP-соединений и отправка в GUI (binary packet). arg = inst */
void chat_core_collect_conn_list_trampoline(void* arg);
/* Сбор полных метрик одного соединения (arg = u_malloc uint64_t peer_node_id) */
struct chat_node_arg { struct UTUN_INSTANCE* inst; uint64_t node_id; };
/* Сбор полных метрик одного соединения (arg = u_malloc struct chat_node_arg) */
void chat_core_collect_conn_metrics_trampoline(void* arg);
/* Сбор live-снапшота для панели деталей участника (arg = u_malloc uint64_t node_id) */
/* Сбор live-снапшота для панели деталей участника (arg = u_malloc struct chat_node_arg) */
void chat_core_collect_member_detail_trampoline(void* arg);
/* ── Headless CLI API ── */
/** Вариант для сборки без UTUN_INSTANCE (использует sqlite3* и node_id) */
void chat_core_set_ctx_for_headless(sqlite3* db, struct UTUN_INSTANCE* inst, uint64_t my_node_id);
/* Список каналов с метаданными в JSON: [{id,name,owner_id,owner_name,peers,msgs,online}] */
int chat_core_get_channels_json(struct UTUN_INSTANCE* inst, char* buf, size_t buf_size, size_t* out_len);
/** Список каналов с метаданными в JSON: [{id,name,owner_id,owner_name,peers,msgs,online}] */
int chat_core_get_channels_json(char* buf, size_t buf_size, size_t* out_len);
/** Сообщения канала в JSON: [{id,ts,author_id,author_name,content_type,data}]
/* Сообщения канала в JSON: [{id,ts,author_id,author_name,content_type,data}]
* count=0 → все, offset=0 → сначала */
int chat_core_get_messages_json(const char* ch_id, int count, int offset,
int chat_core_get_messages_json(struct UTUN_INSTANCE* inst, const char* ch_id, int count, int offset,
char* buf, size_t buf_size, size_t* out_len);
/** Мемберы канала с полным состоянием в JSON:
/* Мемберы канала с полным состоянием в JSON:
* [{node_id,name,online,connected,x25519,ed25519,addrs:[{ip,port,proto,rtt}]}] */
int chat_core_get_members_json(const char* ch_id, char* buf, size_t buf_size, size_t* out_len);
int chat_core_get_members_json(struct UTUN_INSTANCE* inst, const char* ch_id, char* buf, size_t buf_size, size_t* out_len);
/** Имя узла из таблицы nodes */
int chat_core_get_node_name(uint64_t node_id, char* out, size_t sz);
/* Имя узла из таблицы nodes */
int chat_core_get_node_name(struct UTUN_INSTANCE* inst, uint64_t node_id, char* out, size_t sz);
/* ── Редактирование adm_tags (владелец канала) ── */
@@ -155,31 +210,44 @@ int chat_core_get_node_name(uint64_t node_id, char* out, size_t sz);
struct chat_member_tags;
/** Загрузить текущие adm_tags мембера в память. Возвращает NULL если мембера нет. */
struct chat_member_tags* chat_member_tags_load(const char* ch_id, uint64_t node_id);
/* Загрузить текущие adm_tags мембера в память. Возвращает NULL если мембера нет. */
struct chat_member_tags* chat_member_tags_load(struct UTUN_INSTANCE* inst, const char* ch_id, uint64_t node_id);
/** Установить ключ. val=NULL → удалить ключ. ver — особый, автоинкремент при commit. */
/* Установить ключ. val=NULL → удалить ключ. ver — особый, автоинкремент при commit. */
int chat_member_tags_set(struct chat_member_tags* t, const char* key, const char* val);
/** Получить значение ключа. Возвращает NULL если ключ не найден. */
/* Получить значение ключа. Возвращает NULL если ключ не найден. */
const char* chat_member_tags_get(struct chat_member_tags* t, const char* key);
/** Заменить все теги данными из JSON-строки (плоский, строковые значения). */
/* Заменить все теги данными из JSON-строки (плоский, строковые значения). */
int chat_member_tags_replace(struct chat_member_tags* t, const char* json);
/** Сохранить: сериализовать в JSON, инкрементировать ver, подписать канальным Ed25519,
/* Сохранить: сериализовать в JSON, инкрементировать ver, подписать канальным Ed25519,
* записать через member_sync_put() → синхронизируется всем. */
int chat_member_tags_commit(struct chat_member_tags* t);
int chat_member_tags_commit(struct UTUN_INSTANCE* inst, struct chat_member_tags* t);
/** Освободить без сохранения. */
/* Освободить без сохранения. */
void chat_member_tags_free(struct chat_member_tags* t);
/** Трамплин для gui_bridge: arg = struct chat_member_tags_req* (доверенная память). */
/* Трамплин для gui_bridge: arg = struct chat_member_tags_req* (доверенная память). */
struct chat_member_tags_req {
struct UTUN_INSTANCE* inst;
char ch_id[64];
uint64_t node_id;
char json[512];
};
void chat_member_tags_commit_trampoline(void* arg);
/* ── Передача прав админа (владелец → другому узлу) ── */
/* Передать оба канальных privkey выбранному узлу (encrypted+signed через etcp_router). */
void chat_core_transfer_admin(struct UTUN_INSTANCE* inst, const char* ch_id, uint64_t target_node_id);
void chat_core_transfer_admin_trampoline(void* arg);
struct chat_transfer_admin_req {
struct UTUN_INSTANCE* inst;
char ch_id[64];
uint64_t target_node_id;
};
#endif /* CHAT_CORE_H */
+20 -8
View File
@@ -59,7 +59,7 @@ static inline size_t b64_decode(const char* src, size_t src_len, uint8_t* dst, s
if (vi == 4) {
dst[out++] = (uint8_t)((vals[0] << 2) | (vals[1] >> 4));
if (out < dst_cap) dst[out++] = (uint8_t)((vals[1] << 4) | (vals[2] >> 2));
if (out < dst_cap && i + 1 < src_len && src[i + 1] != '=')
if (out < dst_cap)
dst[out++] = (uint8_t)((vals[2] << 6) | vals[3]);
vi = 0;
}
@@ -75,34 +75,46 @@ static inline size_t b64_decode(const char* src, size_t src_len, uint8_t* dst, s
/* ── Глобальное состояние (определено в chat_core.c) ── */
struct ms_props_cbk; /* member_sync.c */
struct ms_apply_cbk; /* member_sync.c */
struct chat_core_ctx {
struct UTUN_INSTANCE* inst;
sqlite3* db;
char db_path[512];
uint8_t shared_db;
uint64_t my_node_id;
struct DB_SYNC_INSTANCE** si;
char** si_ch_id;
int si_count, si_capacity;
uint8_t initialized;
/* per-instance sub-module state (was file-scope statics) */
struct ms_props_cbk* props_cbks; /* was member_sync.c g_props_cbks */
struct ms_apply_cbk* apply_cbks; /* was member_sync.c g_apply_cbks */
uint64_t media_cache_bytes; /* was chat_msg.c g_media_cache_bytes */
void* media_backfill_timer;/* was chat_msg.c g_media_startup_backfill_timer */
int media_backfill_ran; /* was chat_msg.c g_media_startup_backfill_ran */
struct UTUN_INSTANCE* media_backfill_inst; /* was chat_msg.c g_media_startup_backfill_inst */
};
extern struct chat_core_ctx g_cc;
/* Per-instance chat context. inst->chat_core обязателен после chat_core_init. */
#define CC(inst) ((inst)->chat_core)
/* ── Хелперы ── */
void sanitize_ch_id(const char* ch_id, char* out, size_t out_sz);
void msg_table_name(const char* ch_id, char* buf, size_t sz);
void peers_table_name(const char* ch_id, char* buf, size_t sz);
int db_exec(const char* sql);
/* ── db_sync registry (chat_core.c) ── */
struct DB_SYNC_INSTANCE* si_find(const char* ch_id);
void si_register(struct DB_SYNC_INSTANCE* si, const char* ch_id);
struct DB_SYNC_INSTANCE* si_find(struct UTUN_INSTANCE* inst, const char* ch_id);
void si_register(struct UTUN_INSTANCE* inst, struct DB_SYNC_INSTANCE* si, const char* ch_id);
/* ── db_sync callback (chat_msg.c) ── */
struct msg_insert_arg { struct UTUN_INSTANCE* inst; char ch_id[64]; };
void on_msg_inserted(struct DB_SYNC_INSTANCE* si, uint64_t record_ts, const char* data, size_t len, uint64_t author, void* arg);
/* ── применить настройку group_autoconnect ко всем CHAT-группам (chat_channel.c) ── */
void chat_core_apply_group_autoconnect(struct UTUN_INSTANCE* inst);
#endif /* CHAT_CORE_PRIV_H */
+8 -7
View File
@@ -1,29 +1,30 @@
/*
* chat_event.c — реализация системы нотификаций
*
* Обработчик — per-instance (хранится в UTUN_INSTANCE.chat_event_handler).
* Пока обработчик не зарегистрирован — события логируются.
* При регистрации обработчика — форвардятся ему синхронно.
*/
#include "chat_event.h"
#include "../utun_instance.h"
#include "../lib/debug_config.h"
#ifdef __ANDROID__
#include <android/log.h>
#endif
static chat_event_handler_fn g_handler = NULL;
void chat_event_set_handler(chat_event_handler_fn handler) {
void chat_event_set_handler(struct UTUN_INSTANCE* inst, chat_event_handler_fn handler) {
if (!inst) return;
#ifdef __ANDROID__
__android_log_print(ANDROID_LOG_INFO, "utun-evt", "chat_event_set_handler: handler=%p old=%p",
(void*)handler, (void*)g_handler);
(void*)handler, (void*)inst->chat_event_handler);
#endif
g_handler = handler;
inst->chat_event_handler = handler;
}
void chat_event_post(int type, const uint8_t* data, int len) {
if (g_handler) { g_handler(type, data, len); return; }
void chat_event_post(struct UTUN_INSTANCE* inst, int type, const uint8_t* data, int len) {
if (inst && inst->chat_event_handler) { inst->chat_event_handler(inst, type, data, len); return; }
static const char* names[] = {
[1]="MSG_RECEIVED", [2]="CONNECT_RESULT", [3]="NEW_PEER",
+11 -3
View File
@@ -14,6 +14,8 @@
#include <stdint.h>
#include <stddef.h>
struct UTUN_INSTANCE;
#ifdef __cplusplus
extern "C" {
#endif
@@ -46,11 +48,17 @@ extern "C" {
#define CHAT_EVT_MEMBER_REMOVED 25 /* [ch_id_len:1][ch_id:var][node_id:8] — мембер удалён из канала */
#define CHAT_EVT_BGP_INFO 26 /* текст: "rtt:XXms hop:X via bgp: [XXXX] name1 [XXXX] name2" */
#define CHAT_EVT_CONNECTING_NODES 27 /* [ch_id_len:1][ch_id:var][count:2][node_id:8]* */
#define CHAT_EVT_ADMIN_KEY_RECEIVED 28 /* [ch_id_len:1][ch_id:var][from_node_id:8] — получен канальный ключ (стал совладельцем) */
#define CHAT_EVT_INVITE_LINK_READY 29 /* [ch_id_len:1][ch_id:var][link_len:2][link:var] — пустой link = ошибка */
#define CHAT_EVT_INVITE_CANDIDATES 30 /* [ch_id_len:1][ch_id:var][auto_node_id:8][count:2][node_id:8]* — достижимые узлы для invite */
#define CHAT_EVT_CHANNEL_DELETED 31 /* [ch_id_len:1][ch_id:var] — канал удалён локально */
#define CHAT_EVT_DM_MSG_RECEIVED 32 /* [conv_id_len:1][conv_id:var][author_node_id:8] */
#define CHAT_EVT_DM_CONV_UPDATED 33 /* [conv_id_len:1][conv_id:var] — беседа создана/изменена */
typedef void (*chat_event_handler_fn)(int type, const uint8_t* data, int len);
typedef void (*chat_event_handler_fn)(struct UTUN_INSTANCE* inst, int type, const uint8_t* data, int len);
void chat_event_set_handler(chat_event_handler_fn handler);
void chat_event_post(int type, const uint8_t* data, int len);
void chat_event_set_handler(struct UTUN_INSTANCE* inst, chat_event_handler_fn handler);
void chat_event_post(struct UTUN_INSTANCE* inst, int type, const uint8_t* data, int len);
#ifdef __cplusplus
}
+344 -159
View File
@@ -1,37 +1,54 @@
/*
* chat_headless_control.c — TCP control socket for headless chat CLI
*
* JSON line protocol over TCP. Integrated into uasync event loop.
* Default: localhost-only, port from config.
* JSON line protocol:
* Request: {"id":N,"cmd":"<command>",...params}
* Response: {"id":N,"ok":true,"data":{...}} | {"id":N,"ok":false,"error":"..."}
* Event: {"event":"<type>",...}
*
* Commands: ping, status, channels, members, messages, send, invite,
* invite_nodes, connect, create_channel, subscribe, quit
*/
#include "chat_headless_control.h"
#include "invite_link.h"
#include "chat_core.h"
#include "chat_sync.h"
#include "chat_core_priv.h"
#include "chat_event.h"
#include "../utun_instance.h"
#include "../ntp_time.h"
#include "chat_sync.h"
#include "chat_join.h"
#include "invite_link.h"
#include "invite_build.h"
#include "../dm/dm_core.h"
#include "../dm/dm_crypto.h"
#include "../transport_layer/secure_channel.h"
#include "../transport_layer/etcp.h"
#include "../transport_layer/etcp_connections.h"
#include "../transport_layer/secure_channel.h"
#include "../../lib/u_async.h"
#include "../../lib/debug_config.h"
#include "../../lib/mem.h"
#include "../utun_instance.h"
#include "../ntp_time.h"
#include "../../lib/ll_queue.h"
#include "../routing_layer/topo_node_sqlite.h"
#include "../../lib/u_async.h"
#include "../../lib/mem.h"
#include "../../lib/socket_compat.h"
#include "../../lib/debug_config.h"
#include "../../lib/platform_compat.h"
#include <stdlib.h>
#include <string.h>
#include <stdio.h>
#include "../../lib/platform_compat.h"
#include "../../lib/socket_compat.h"
#include <stdlib.h>
#include <stdarg.h>
#include <errno.h>
#include <sys/stat.h>
#include <time.h>
#include <sqlite3.h>
#define HC_ID "headless"
#define MAX_CLIENTS 16
#define RECV_BUF_SIZE 16384
#define SEND_BUF_SIZE 16384
#define SEND_BUF_SIZE 32768
#define MAX_CLIENTS 64
#ifndef DEBUG_CATEGORY_HEADLESS
#define DEBUG_CATEGORY_HEADLESS 27
#define DEBUG_CATEGORY_HEADLESS DEBUG_CATEGORY_GENERAL
#endif
struct headless_client {
@@ -44,10 +61,11 @@ struct headless_client {
int send_offset;
int subscribed;
int closing;
struct headless_control* hc;
struct headless_client* next;
};
static struct {
struct headless_control {
struct UASYNC* ua;
struct UTUN_INSTANCE* inst;
struct headless_client* clients;
@@ -55,7 +73,11 @@ static struct {
socket_t listen_fd;
void* listen_sock_id;
int running;
} g_hc;
};
static struct headless_control* hc_of(struct UTUN_INSTANCE* inst) {
return inst ? (struct headless_control*)inst->headless : NULL;
}
/* ── Forward declarations ── */
@@ -101,6 +123,7 @@ static const char* json_get_cmd_end(const char* cmd_start) {
/* ── Send helpers ── */
static void cli_send(struct headless_client* cli, const char* data, size_t len) {
struct headless_control* hc = cli->hc;
if (!cli || cli->closing || cli->fd == SOCKET_INVALID || !data || len == 0) return;
if (cli->send_len > 0) {
size_t space = sizeof(cli->send_buf) - (size_t)cli->send_len;
@@ -112,13 +135,13 @@ static void cli_send(struct headless_client* cli, const char* data, size_t len)
ssize_t r = send(cli->fd, data, (int)len, 0);
if (r < 0 && (socket_get_error() == ERR_AGAIN || socket_get_error() == ERR_WOULDBLOCK)) {
if (len <= sizeof(cli->send_buf)) { memcpy(cli->send_buf, data, len); cli->send_len = (int)len; cli->send_offset = 0; }
if (g_hc.ua) uasync_set_socket_write(g_hc.ua, cli->socket_id, 1);
if (hc && hc->ua) uasync_set_socket_write(hc->ua, cli->socket_id, 1);
return;
}
if (r >= 0 && (size_t)r < len) {
size_t remain = len - (size_t)r;
if (remain <= sizeof(cli->send_buf)) { memcpy(cli->send_buf, data + r, remain); cli->send_len = (int)remain; cli->send_offset = 0; }
if (g_hc.ua) uasync_set_socket_write(g_hc.ua, cli->socket_id, 1);
if (hc && hc->ua) uasync_set_socket_write(hc->ua, cli->socket_id, 1);
}
}
@@ -136,17 +159,19 @@ static void send_response(struct headless_client* cli, int id, const char* ok_da
/* ── Broadcast event ── */
static void hc_broadcast_event(const char* json) {
static void hc_broadcast_event(struct headless_control* hc, const char* json) {
size_t jlen = strlen(json);
char buf[4096]; int blen = snprintf(buf, sizeof(buf), "%s\n", json);
if (blen <= 0) return;
for (struct headless_client* c = g_hc.clients; c; c = c->next)
for (struct headless_client* c = hc->clients; c; c = c->next)
if (c->subscribed) cli_send(c, buf, (size_t)blen);
}
/* ── chat_event callback ── */
static void hc_on_chat_event(int type, const uint8_t* data, int len) {
static void hc_on_chat_event(struct UTUN_INSTANCE* inst, int type, const uint8_t* data, int len) {
struct headless_control* hc = hc_of(inst);
if (!hc) return;
if (type == 10) return; /* skip status text */
char json[1024]; json[0] = '\0';
if (type == 1 && data && len >= 10) { /* MSG_RECEIVED */
@@ -169,29 +194,38 @@ static void hc_on_chat_event(int type, const uint8_t* data, int len) {
(int)cl, data + 1, (int)nl, data + 2 + cl, (unsigned long long)inviter);
}
}
} else if (type == 32 && data && len >= 10) { /* DM_MSG_RECEIVED */
uint8_t cl = data[0];
uint64_t author; memcpy(&author, data + 1 + cl, 8);
snprintf(json, sizeof(json), "{\"event\":\"dm_msg\",\"conv\":\"%.*s\",\"author_id\":\"0x%016llx\"}", (int)cl, data + 1, (unsigned long long)author);
} else if (type == 33 && data && len >= 2) { /* DM_CONV_UPDATED */
uint8_t cl = data[0];
snprintf(json, sizeof(json), "{\"event\":\"dm_conv\",\"conv\":\"%.*s\"}", (int)cl, data + 1);
}
if (json[0]) hc_broadcast_event(json);
if (json[0]) hc_broadcast_event(hc, json);
}
/* ── Write callback ── */
static void client_write_callback(socket_t fd, void* arg) {
struct headless_client* cli = (struct headless_client*)arg;
struct headless_control* hc = cli ? cli->hc : NULL;
if (!cli || cli->closing || fd != cli->fd) return;
if (cli->send_len <= 0) { if (g_hc.ua) uasync_set_socket_write(g_hc.ua, cli->socket_id, 0); return; }
if (cli->send_len <= 0) { if (hc && hc->ua) uasync_set_socket_write(hc->ua, cli->socket_id, 0); return; }
ssize_t r = send(cli->fd, cli->send_buf + cli->send_offset, (int)(cli->send_len - cli->send_offset), 0);
if (r < 0 && (socket_get_error() == ERR_AGAIN || socket_get_error() == ERR_WOULDBLOCK)) return;
if (r <= 0) { cli->send_len = 0; cli->send_offset = 0; if (g_hc.ua) uasync_set_socket_write(g_hc.ua, cli->socket_id, 0); return; }
if (r <= 0) { cli->send_len = 0; cli->send_offset = 0; if (hc && hc->ua) uasync_set_socket_write(hc->ua, cli->socket_id, 0); return; }
cli->send_offset += (int)r;
if (cli->send_offset >= cli->send_len) { cli->send_len = 0; cli->send_offset = 0; if (g_hc.ua) uasync_set_socket_write(g_hc.ua, cli->socket_id, 0); }
if (cli->send_offset >= cli->send_len) { cli->send_len = 0; cli->send_offset = 0; if (hc && hc->ua) uasync_set_socket_write(hc->ua, cli->socket_id, 0); }
}
/* ── Close client ── */
static void hc_close_client(struct headless_client* cli) {
struct headless_control* hc = cli ? cli->hc : NULL;
if (!cli || cli->closing) return;
cli->closing = 1;
if (g_hc.ua && cli->socket_id) uasync_remove_socket_t(g_hc.ua, cli->fd);
if (hc && hc->ua && cli->socket_id) uasync_remove_socket_t(hc->ua, cli->fd);
if (cli->fd != SOCKET_INVALID) { socket_close_wrapper(cli->fd); cli->fd = SOCKET_INVALID; }
}
@@ -216,30 +250,29 @@ static void client_read_callback(socket_t fd, void* arg) {
static void hc_handle_ping(struct headless_client* cli, int id, const char* json) {
(void)json;
uint64_t uptime = 0;
(void)uptime;
char data[128]; snprintf(data, sizeof(data), "{\"pong\":true,\"uptime_sec\":%llu}", (unsigned long long)uptime);
char data[128]; snprintf(data, sizeof(data), "{\"pong\":true,\"uptime_sec\":0}");
send_response(cli, id, data, NULL);
}
static void hc_handle_status(struct headless_client* cli, int id, const char* json) {
(void)json;
struct headless_control* hc = cli->hc;
char buf[8192]; int off = 0;
off += snprintf(buf + off, sizeof(buf) - off, "\"");
if (!g_hc.inst) {
if (!hc->inst) {
off += snprintf(buf + off, sizeof(buf) - off, "ERROR: no instance");
} else {
int conn_count = g_hc.inst->connections ? queue_entry_count(g_hc.inst->connections) : 0;
struct NTP_TIME* ntp = &g_hc.inst->ntp;
int conn_count = hc->inst->connections ? queue_entry_count(hc->inst->connections) : 0;
struct NTP_TIME* ntp = &hc->inst->ntp;
off += snprintf(buf + off, sizeof(buf) - off, "NTP: enabled=%s synced=%s offset=%.1fs\\n",
ntp->enabled ? "yes" : "no", ntp->synced ? "yes" : "no", ntp->offset_us / 1000000.0);
off += snprintf(buf + off, sizeof(buf) - off, "Connections: %d active\\n", conn_count);
uint64_t my_id = g_hc.inst->node_id;
struct ll_entry* entry = g_hc.inst->connections ? g_hc.inst->connections->head : NULL;
uint64_t my_id = hc->inst->node_id;
struct ll_entry* entry = hc->inst->connections ? hc->inst->connections->head : NULL;
while (entry) {
struct conn_queue_entry* ce = (struct conn_queue_entry*)entry->data;
if (ce) {
char name[64]; chat_core_get_node_name(ce->peer_node_id, name, sizeof(name));
char name[64]; chat_core_get_node_name(hc->inst, ce->peer_node_id, name, sizeof(name));
off += snprintf(buf + off, sizeof(buf) - off, " [%04llX]->[%04llX] %s ETCP:%s\\n",
(unsigned long long)(my_id & 0xFFFF), (unsigned long long)(ce->peer_node_id & 0xFFFF),
name[0] ? name : "?", ce->conn && ce->conn->links_up ? "UP" : "DOWN");
@@ -255,7 +288,7 @@ static void hc_handle_status(struct headless_client* cli, int id, const char* js
static void hc_handle_channels(struct headless_client* cli, int id, const char* json) {
(void)json;
char buf[16384]; size_t len = 0;
if (chat_core_get_channels_json(buf, sizeof(buf), &len) < 0)
if (chat_core_get_channels_json(cli->hc->inst, buf, sizeof(buf), &len) < 0)
send_response(cli, id, NULL, "failed to query channels");
else
send_response(cli, id, buf, NULL);
@@ -265,7 +298,7 @@ static void hc_handle_members(struct headless_client* cli, int id, const char* j
char ch[64];
if (json_get_str(json, "ch", ch, sizeof(ch)) < 0) { send_response(cli, id, NULL, "missing 'ch' param"); return; }
char buf[16384]; size_t len = 0;
if (chat_core_get_members_json(ch, buf, sizeof(buf), &len) < 0)
if (chat_core_get_members_json(cli->hc->inst, ch, buf, sizeof(buf), &len) < 0)
send_response(cli, id, NULL, "failed to query members");
else
send_response(cli, id, buf, NULL);
@@ -277,7 +310,7 @@ static void hc_handle_messages(struct headless_client* cli, int id, const char*
count = json_get_int(json, "count", 20);
offset = json_get_int(json, "offset", 0);
char buf[16384]; size_t len = 0;
if (chat_core_get_messages_json(ch, count, offset, buf, sizeof(buf), &len) < 0)
if (chat_core_get_messages_json(cli->hc->inst, ch, count, offset, buf, sizeof(buf), &len) < 0)
send_response(cli, id, NULL, "failed to query messages");
else
send_response(cli, id, buf, NULL);
@@ -290,96 +323,168 @@ static void hc_handle_send(struct headless_client* cli, int id, const char* json
if (json_get_str(json, "data", data, sizeof(data)) < 0) { send_response(cli, id, NULL, "missing 'data' param"); return; }
struct chat_msg_submit* req = u_calloc(1, sizeof(*req));
if (!req) { send_response(cli, id, NULL, "out of memory"); return; }
req->inst = cli->hc->inst;
snprintf(req->channel_id, sizeof(req->channel_id), "%s", ch);
snprintf(req->content_type, sizeof(req->content_type), "%s", ct);
req->data = (uint8_t*)u_strdup(data);
req->data_len = (uint32_t)strlen(data);
chat_core_submit_message(req);
chat_core_submit_message(cli->hc->inst, req);
if (req->data) u_free(req->data);
u_free(req);
send_response(cli, id, "{\"sent\":true}", NULL);
}
static void hc_handle_invite(struct headless_client* cli, int id, const char* json) {
char ch[64];
static int hc_mkdirs(const char* path) {
char tmp[1024]; snprintf(tmp, sizeof(tmp), "%s", path);
size_t len = strlen(tmp);
for (size_t i = 1; i < len; i++) {
if (tmp[i] != '/') continue;
tmp[i] = '\0';
if (utun_mkdir(tmp, 0755) != 0 && errno != EEXIST) { tmp[i] = '/'; return -1; }
tmp[i] = '/';
}
if (utun_mkdir(tmp, 0755) != 0 && errno != EEXIST) return -1;
return 0;
}
static void hc_handle_attach(struct headless_client* cli, int id, const char* json) {
struct headless_control* hc = cli->hc;
char ch[64], path[1024], ct[32];
if (json_get_str(json, "ch", ch, sizeof(ch)) < 0) { send_response(cli, id, NULL, "missing 'ch' param"); return; }
if (!g_hc.inst) { send_response(cli, id, NULL, "no instance"); return; }
if (json_get_str(json, "path", path, sizeof(path)) < 0) { send_response(cli, id, NULL, "missing 'path' param"); return; }
if (json_get_str(json, "content_type", ct, sizeof(ct)) < 0)
snprintf(ct, sizeof(ct), "application/octet-stream");
int as_video = json_get_int(json, "as_video", 0);
uint64_t my_node_id = g_hc.inst->node_id;
uint8_t my_pubkey[32]; memcpy(my_pubkey, g_hc.inst->my_keys.public_key, 32);
struct stat st;
if (stat(path, &st) != 0 || st.st_size <= 0) { send_response(cli, id, NULL, "file not found or empty"); return; }
sqlite3* db = chat_core_get_db();
if (!db) { send_response(cli, id, NULL, "no database"); return; }
if (!hc->inst || !hc->inst->media_async) { send_response(cli, id, NULL, "media engine not initialized"); return; }
struct InviteData inv; memset(&inv, 0, sizeof(inv));
inv.channelId = strtoull(ch, NULL, 10);
if (inv.channelId == 0) { send_response(cli, id, NULL, "invalid channel_id"); return; }
memcpy(inv.pubkey, my_pubkey, 32);
inv.nodeId = my_node_id;
sqlite3_stmt* as = NULL;
sqlite3_prepare_v2(db, "SELECT family, protocol, address, port, rtt, socket_id FROM node_addresses WHERE node_id=? AND protocol IN (1,2) ORDER BY family, protocol",
-1, &as, NULL);
if (as) {
sqlite3_bind_int64(as, 1, (sqlite3_int64)my_node_id);
while (sqlite3_step(as) == SQLITE_ROW && inv.addrCount < INVITE_ADDR_MAX) {
struct InviteAddr* a = &inv.addrs[inv.addrCount++];
int fam = sqlite3_column_int(as, 0);
a->proto = (uint8_t)sqlite3_column_int(as, 1);
const uint8_t* addr = (const uint8_t*)sqlite3_column_blob(as, 2);
int addr_len = sqlite3_column_bytes(as, 2);
a->port = (uint16_t)sqlite3_column_int(as, 3);
a->socketId = (uint8_t)sqlite3_column_int(as, 5);
a->family = fam;
memset(a->address, 0, sizeof(a->address));
if (addr && addr_len > 0) memcpy(a->address, addr, addr_len < 16 ? (size_t)addr_len : 16);
}
sqlite3_finalize(as);
}
if (inv.addrCount == 0) {
/* fallback: collect from sockets directly */
struct ETCP_SOCKET* s = g_hc.inst->etcp_sockets;
while (s && inv.addrCount < INVITE_ADDR_MAX) {
if (s->is_tcp) { s = s->next; continue; }
if (s->local_addr.ss_family == AF_INET) {
struct sockaddr_in* sin = (struct sockaddr_in*)&s->local_addr;
struct InviteAddr* a = &inv.addrs[inv.addrCount++];
a->family = 4; a->proto = 1; a->socketId = s->sock_id;
a->port = ntohs(sin->sin_port); memcpy(a->address, &sin->sin_addr, 4);
/* media_base из ui_state (как в блок-сервере media_delivery) */
sqlite3* db = chat_core_get_db(hc->inst);
char media_base[512] = "/tmp/utun_media";
if (db) {
sqlite3_stmt* ms = NULL;
if (sqlite3_prepare_v2(db, "SELECT value FROM ui_state WHERE key='media_base'", -1, &ms, NULL) == SQLITE_OK) {
if (sqlite3_step(ms) == SQLITE_ROW) {
const char* mb = (const char*)sqlite3_column_text(ms, 0);
if (mb) snprintf(media_base, sizeof(media_base), "%s", mb);
}
s = s->next;
}
struct ETCP_SOCKET* ts = g_hc.inst->etcp_sockets;
while (ts && inv.addrCount < INVITE_ADDR_MAX) {
if (!ts->is_tcp) { ts = ts->next; continue; }
if (ts->interface_addr.ss_family == AF_INET) {
struct sockaddr_in* sin = (struct sockaddr_in*)&ts->interface_addr;
struct InviteAddr* a = &inv.addrs[inv.addrCount++];
a->family = 4; a->proto = 2; a->socketId = ts->sock_id;
a->port = ntohs(sin->sin_port); memcpy(a->address, &sin->sin_addr, 4);
}
ts = ts->next;
sqlite3_finalize(ms);
}
}
if (inv.addrCount == 0) { send_response(cli, id, NULL, "no addresses available to build invite link"); return; }
const char* base = strrchr(path, '/'); base = base ? base + 1 : path;
char name[256];
{ size_t blen = strlen(base); if (blen >= sizeof(name)) blen = sizeof(name) - 1;
memcpy(name, base, blen); name[blen] = '\0'; }
char dir[640]; snprintf(dir, sizeof(dir), "%s/%s", media_base, ch);
if (hc_mkdirs(dir) != 0) { send_response(cli, id, NULL, "failed to create media dir"); return; }
char dest[1024];
uint64_t ts = (uint64_t)time(NULL);
if (as_video) {
char* dot = strrchr(name, '.'); if (dot) *dot = '\0';
snprintf(dest, sizeof(dest), "%s/%llu_%s.mp4", dir, (unsigned long long)ts, name);
} else {
snprintf(dest, sizeof(dest), "%s/%llu_%s", dir, (unsigned long long)ts, name);
}
struct chat_msg_submit* req = u_calloc(1, sizeof(*req));
if (!req) { send_response(cli, id, NULL, "out of memory"); return; }
req->inst = hc->inst;
snprintf(req->channel_id, sizeof(req->channel_id), "%s", ch);
snprintf(req->content_type, sizeof(req->content_type), "%s", as_video ? "video/mp4" : ct);
snprintf(req->media_src, sizeof(req->media_src), "%s", path);
snprintf(req->media_dest, sizeof(req->media_dest), "%s", dest);
req->media_copy = as_video ? 0 : 1;
req->media_video = as_video ? 1 : 0;
req->data = (uint8_t*)name;
req->data_len = (uint32_t)strlen(name);
DEBUG_INFO((int)DEBUG_CATEGORY_HEADLESS, "headless: attach ch=%s src=%s dst=%s video=%d",
ch, path, dest, as_video);
chat_core_submit_media_message(hc->inst, req);
u_free(req);
char resp[1400]; snprintf(resp, sizeof(resp), "{\"attached\":true,\"dest\":\"%s\",\"video\":%s}",
dest, as_video ? "true" : "false");
send_response(cli, id, resp, NULL);
}
static void hc_handle_invite(struct headless_client* cli, int id, const char* json) {
struct headless_control* hc = cli->hc;
char ch[64], node_id_str[32];
if (json_get_str(json, "ch", ch, sizeof(ch)) < 0) { send_response(cli, id, NULL, "missing 'ch' param"); return; }
if (!hc->inst) { send_response(cli, id, NULL, "no instance"); return; }
uint64_t channel_id = strtoull(ch, NULL, 10);
if (channel_id == 0) { send_response(cli, id, NULL, "invalid channel_id"); return; }
/* опциональный выбор узла: "0" или отсутствует → auto (лучший узел) */
uint64_t target_node_id = 0;
if (json_get_str(json, "node_id", node_id_str, sizeof(node_id_str)) == 0)
target_node_id = strtoull(node_id_str, NULL, 16);
char link[1024];
int r = invite_link_encode(&inv, NULL, link, sizeof(link));
if (r < 0) { send_response(cli, id, NULL, "failed to encode invite link"); return; }
if (chat_invite_build_link(hc->inst, channel_id, target_node_id, NULL, link, sizeof(link)) < 0) {
send_response(cli, id, NULL, "failed to build invite link (see log)");
return;
}
char resp[1280]; snprintf(resp, sizeof(resp), "{\"link\":\"%s\",\"channel_id\":%llu,\"addrs\":%d}", link,
(unsigned long long)inv.channelId, (int)inv.addrCount);
struct InviteData d; char err[128];
if (invite_link_decode(link, strlen(link), &d, err, sizeof(err)) < 0) {
send_response(cli, id, NULL, "failed to decode built link");
return;
}
DEBUG_INFO((int)DEBUG_CATEGORY_HEADLESS, "headless: invite ch=%s target=0x%016llx node=0x%016llx addrs=%d",
ch, (unsigned long long)target_node_id, (unsigned long long)d.nodeId, (int)d.addrCount);
char resp[1280];
snprintf(resp, sizeof(resp),
"{\"link\":\"%s\",\"channel_id\":%llu,\"node_id\":\"0x%016llx\",\"addrs\":%d}",
link, (unsigned long long)d.channelId, (unsigned long long)d.nodeId, (int)d.addrCount);
send_response(cli, id, resp, NULL);
}
static void hc_handle_invite_nodes(struct headless_client* cli, int id, const char* json) {
struct headless_control* hc = cli->hc;
char ch[64];
if (json_get_str(json, "ch", ch, sizeof(ch)) < 0) { send_response(cli, id, NULL, "missing 'ch' param"); return; }
uint64_t channel_id = strtoull(ch, NULL, 10);
if (channel_id == 0) { send_response(cli, id, NULL, "invalid channel_id"); return; }
uint64_t ids[INVITE_ADDR_MAX];
int cnt = chat_invite_candidate_nodes(hc->inst, channel_id, ids, INVITE_ADDR_MAX);
if (cnt < 0) { send_response(cli, id, NULL, "no candidates (see log)"); return; }
uint64_t auto_id = 0;
chat_invite_best_node(hc->inst, channel_id, &auto_id);
char resp[4096]; int off = 0;
off += snprintf(resp + off, sizeof(resp) - off, "{\"ch\":\"%s\",\"auto_node_id\":\"0x%016llx\",\"nodes\":[",
ch, (unsigned long long)auto_id);
for (int i = 0; i < cnt && off < (int)sizeof(resp) - 256; i++) {
char name[64] = "";
chat_core_get_node_name(hc->inst, ids[i], name, sizeof(name));
off += snprintf(resp + off, sizeof(resp) - off, "%s{\"node_id\":\"0x%016llx\",\"name\":\"%s\"}",
i ? "," : "", (unsigned long long)ids[i], name);
}
off += snprintf(resp + off, sizeof(resp) - off, "]}");
send_response(cli, id, resp, NULL);
}
static void hc_handle_connect(struct headless_client* cli, int id, const char* json) {
struct headless_control* hc = cli->hc;
char link[1024];
if (json_get_str(json, "link", link, sizeof(link)) < 0) { send_response(cli, id, NULL, "missing 'link' param"); return; }
struct InviteData d; char err[256];
if (invite_link_decode(link, strlen(link), &d, err, sizeof(err)) < 0) { send_response(cli, id, NULL, err); return; }
if (!g_hc.inst) { send_response(cli, id, NULL, "no instance"); return; }
if (!hc->inst) { send_response(cli, id, NULL, "no instance"); return; }
DEBUG_INFO((int)DEBUG_CATEGORY_HEADLESS, "headless: connect ch=%llu node=0x%016llx addrs=%d",
(unsigned long long)d.channelId, (unsigned long long)d.nodeId, (int)d.addrCount);
@@ -388,14 +493,13 @@ static void hc_handle_connect(struct headless_client* cli, int id, const char* j
int addrs_len = invite_serialize_addrs(&d, addrs_buf, sizeof(addrs_buf));
if (addrs_len <= 0) { send_response(cli, id, NULL, "serialize addrs failed"); return; }
chat_core_ensure_channel_ready((const char*)(&(char[64]){0}));
{ char chstr[64]; snprintf(chstr, sizeof(chstr), "%llu", (unsigned long long)d.channelId);
chat_core_ensure_channel_ready(chstr); }
chat_core_ensure_channel_ready(hc->inst, chstr); }
chat_core_set_my_node_id(g_hc.inst->node_id);
chat_core_set_my_node_id(hc->inst, hc->inst->node_id);
chat_sync_connect_from_invite(d.channelId, d.nodeId, d.pubkey,
addrs_buf, d.addrCount, addrs_len, d.password_len ? d.password : NULL);
chat_sync_connect_from_invite(hc->inst, d.channelId, d.nodeId, d.pubkey,
addrs_buf, d.addrCount, addrs_len, d.password_len ? d.password : NULL, d.join_key);
char resp[512]; snprintf(resp, sizeof(resp),
"{\"channel_id\":%llu,\"node_id\":\"0x%016llx\",\"addrs\":%d,\"connecting\":true}",
@@ -406,21 +510,22 @@ static void hc_handle_connect(struct headless_client* cli, int id, const char* j
static void hc_handle_create_channel(struct headless_client* cli, int id, const char* json) {
char name[256];
if (json_get_str(json, "name", name, sizeof(name)) < 0) { send_response(cli, id, NULL, "missing 'name' param"); return; }
chat_core_create_channel_auto(name);
chat_core_create_channel_auto(cli->hc->inst, name);
send_response(cli, id, "{\"created\":true}", NULL);
}
static void hc_handle_invite_to(struct headless_client* cli, int id, const char* json) {
struct headless_control* hc = cli->hc;
char ch[64], node_id_str[32], pubkey_hex[128], addr[128];
if (json_get_str(json, "ch", ch, sizeof(ch)) < 0) { send_response(cli, id, NULL, "missing 'ch' param"); return; }
if (json_get_str(json, "node_id", node_id_str, sizeof(node_id_str)) < 0) { send_response(cli, id, NULL, "missing 'node_id' param"); return; }
if (json_get_str(json, "pubkey", pubkey_hex, sizeof(pubkey_hex)) < 0) { send_response(cli, id, NULL, "missing 'pubkey' param"); return; }
if (json_get_str(json, "addr", addr, sizeof(addr)) < 0) { send_response(cli, id, NULL, "missing 'addr' param"); return; }
int proto = json_get_int(json, "proto", 1);
if (!g_hc.inst) { send_response(cli, id, NULL, "no instance"); return; }
if (!hc->inst) { send_response(cli, id, NULL, "no instance"); return; }
uint64_t target_node_id = strtoull(node_id_str, NULL, 16);
if (target_node_id == 0 || target_node_id == g_hc.inst->node_id) { send_response(cli, id, NULL, "invalid node_id"); return; }
if (target_node_id == 0 || target_node_id == hc->inst->node_id) { send_response(cli, id, NULL, "invalid node_id"); return; }
uint8_t pubkey_bin[SC_PUBKEY_SIZE];
if (sc_hex_to_binary(pubkey_hex, pubkey_bin, SC_PUBKEY_SIZE) != 0) { send_response(cli, id, NULL, "invalid pubkey hex"); return; }
@@ -432,24 +537,26 @@ static void hc_handle_invite_to(struct headless_client* cli, int id, const char*
int port = (int)strtol(colon + 1, NULL, 10);
if (port <= 0 || port > 65535) { send_response(cli, id, NULL, "invalid port"); return; }
uint8_t addrs_data[21]; int addrs_len = 0;
uint8_t addrs_data[22]; int addrs_len = 0;
struct in_addr a4; struct in6_addr a6;
if (inet_pton(AF_INET, ip, &a4) == 1) {
addrs_data[0] = 4; addrs_data[1] = 0; addrs_data[2] = (uint8_t)proto;
memcpy(addrs_data + 3, &a4.s_addr, 4);
addrs_data[7] = (uint8_t)(port >> 8); addrs_data[8] = (uint8_t)(port & 0xFF);
addrs_len = 9;
addrs_data[0] = 0; /* reality-префикс: has=0 */
addrs_data[1] = 4; addrs_data[2] = 0; addrs_data[3] = (uint8_t)proto;
memcpy(addrs_data + 4, &a4.s_addr, 4);
addrs_data[8] = (uint8_t)(port >> 8); addrs_data[9] = (uint8_t)(port & 0xFF);
addrs_len = 10;
} else if (inet_pton(AF_INET6, ip, &a6) == 1) {
addrs_data[0] = 6; addrs_data[1] = 0; addrs_data[2] = (uint8_t)proto;
memcpy(addrs_data + 3, &a6, 16);
addrs_data[19] = (uint8_t)(port >> 8); addrs_data[20] = (uint8_t)(port & 0xFF);
addrs_len = 21;
addrs_data[0] = 0; /* reality-префикс: has=0 */
addrs_data[1] = 6; addrs_data[2] = 0; addrs_data[3] = (uint8_t)proto;
memcpy(addrs_data + 4, &a6, 16);
addrs_data[20] = (uint8_t)(port >> 8); addrs_data[21] = (uint8_t)(port & 0xFF);
addrs_len = 22;
} else { send_response(cli, id, NULL, "invalid addr ip"); return; }
DEBUG_INFO((int)DEBUG_CATEGORY_HEADLESS, "headless: invite_to ch=%s node=0x%016llx addr=%s:%d proto=%d",
ch, (unsigned long long)target_node_id, ip, port, proto);
chat_sync_invite_to_channel_with_addrs(g_hc.inst, ch, target_node_id,
chat_sync_invite_to_channel_with_addrs(hc->inst, ch, target_node_id,
pubkey_bin, addrs_data, 1, addrs_len);
char resp[256]; snprintf(resp, sizeof(resp),
@@ -472,14 +579,85 @@ static void hc_handle_quit(struct headless_client* cli, int id, const char* json
hc_close_client(cli);
}
/* ── DM (прямой чат) ── */
static void hc_handle_dm_start(struct headless_client* cli, int id, const char* json) {
struct headless_control* hc = cli->hc;
char node_id_str[64], ch[64];
if (json_get_str(json, "node_id", node_id_str, sizeof(node_id_str)) < 0) { send_response(cli, id, NULL, "missing 'node_id' param"); return; }
if (json_get_str(json, "ch", ch, sizeof(ch)) < 0) ch[0] = '\0';
uint64_t target = strtoull(node_id_str, NULL, 16);
if (!target || !hc->inst) { send_response(cli, id, NULL, "invalid node_id"); return; }
sqlite3* db = chat_core_get_db(hc->inst);
uint8_t x25519[32], ed25519[32];
int ok = 0;
sqlite3_stmt* st = NULL;
if (db && sqlite3_prepare_v2(db,
"SELECT x25519_pubkey, ed25519_pubkey, COALESCE(name,'') FROM nodes WHERE node_id=?",
-1, &st, NULL) == SQLITE_OK) {
sqlite3_bind_int64(st, 1, (sqlite3_int64)target);
if (sqlite3_step(st) == SQLITE_ROW) {
const uint8_t* x = (const uint8_t*)sqlite3_column_blob(st, 0);
const uint8_t* e = (const uint8_t*)sqlite3_column_blob(st, 1);
if (x && e && sqlite3_column_bytes(st, 0) >= 32 && sqlite3_column_bytes(st, 1) >= 32) {
memcpy(x25519, x, 32); memcpy(ed25519, e, 32);
const char* name = (const char*)sqlite3_column_text(st, 2);
if (dm_start(hc->inst, target, x25519, ed25519, name, ch[0] ? ch : NULL) == 0) ok = 1;
}
}
}
if (st) sqlite3_finalize(st);
if (!ok) { send_response(cli, id, NULL, "failed to start DM (node not found?)"); return; }
char conv[64]; uint64_t conv_num = dm_derive_conv_id(hc->inst->node_id, target);
snprintf(conv, sizeof(conv), "%llu", (unsigned long long)conv_num);
char resp[256]; snprintf(resp, sizeof(resp), "{\"conv_id\":\"%s\",\"started\":true}", conv);
send_response(cli, id, resp, NULL);
}
/* Список DM-бесед. */
static void hc_handle_dm_list(struct headless_client* cli, int id, const char* json) {
(void)json;
char buf[16384]; size_t len = 0;
if (dm_list_conversations_json(cli->hc->inst, buf, sizeof(buf), &len) < 0)
send_response(cli, id, NULL, "failed to query conversations");
else
send_response(cli, id, buf, NULL);
}
/* Сообщения DM-беседы (расшифрованные). */
static void hc_handle_dm_messages(struct headless_client* cli, int id, const char* json) {
char conv[64]; int count = 20, offset = 0;
if (json_get_str(json, "conv_id", conv, sizeof(conv)) < 0) { send_response(cli, id, NULL, "missing 'conv_id' param"); return; }
count = json_get_int(json, "count", 20);
offset = json_get_int(json, "offset", 0);
char buf[16384]; size_t len = 0;
if (dm_list_messages_json(cli->hc->inst, conv, count, offset, buf, sizeof(buf), &len) < 0)
send_response(cli, id, NULL, "failed to query DM messages");
else
send_response(cli, id, buf, NULL);
}
/* Отправить сообщение в DM-беседу. */
static void hc_handle_dm_send(struct headless_client* cli, int id, const char* json) {
char conv[64], data[4096];
if (json_get_str(json, "conv_id", conv, sizeof(conv)) < 0) { send_response(cli, id, NULL, "missing 'conv_id' param"); return; }
if (json_get_str(json, "data", data, sizeof(data)) < 0) { send_response(cli, id, NULL, "missing 'data' param"); return; }
if (dm_send(cli->hc->inst, conv, "text", (const uint8_t*)data, (uint32_t)strlen(data)) != 0)
send_response(cli, id, NULL, "failed to send DM");
else
send_response(cli, id, "{\"sent\":true}", NULL);
}
/* ── Command dispatch ── */
typedef void (*hc_cmd_fn)(struct headless_client* cli, int id, const char* json);
static const char* cmd_names[] = { "ping", "status", "channels", "members", "messages", "send", "invite", "connect", "create_channel", "invite_to", "subscribe", "quit", NULL };
static hc_cmd_fn cmd_handlers[] = { hc_handle_ping, hc_handle_status, hc_handle_channels, hc_handle_members, hc_handle_messages, hc_handle_send, hc_handle_invite, hc_handle_connect, hc_handle_create_channel, hc_handle_invite_to, hc_handle_subscribe, hc_handle_quit };
static const char* cmd_names[] = { "ping", "status", "channels", "members", "messages", "send", "attach", "invite", "invite_nodes", "connect", "create_channel", "invite_to", "subscribe", "dm_start", "dm_list", "dm_messages", "dm_send", "quit", NULL };
static hc_cmd_fn cmd_handlers[] = { hc_handle_ping, hc_handle_status, hc_handle_channels, hc_handle_members, hc_handle_messages, hc_handle_send, hc_handle_attach, hc_handle_invite, hc_handle_invite_nodes, hc_handle_connect, hc_handle_create_channel, hc_handle_invite_to, hc_handle_subscribe, hc_handle_dm_start, hc_handle_dm_list, hc_handle_dm_messages, hc_handle_dm_send, hc_handle_quit };
void hc_handle_command(struct headless_client* cli, const char* json) {
static void hc_handle_command(struct headless_client* cli, const char* json) {
if (!cli || !json) return;
int id = json_get_int(json, "id", 0);
const char* cs = json_get_cmd(json);
@@ -497,87 +675,94 @@ void hc_handle_command(struct headless_client* cli, const char* json) {
/* ── Accept callback ── */
static void accept_callback(socket_t fd, void* arg) {
(void)arg;
struct headless_control* hc = (struct headless_control*)arg;
struct sockaddr_storage addr; socklen_t alen = sizeof(addr);
socket_t cfd = accept(fd, (struct sockaddr*)&addr, &alen);
if (cfd == SOCKET_INVALID) { int e = socket_get_error(); if (e != ERR_AGAIN && e != ERR_WOULDBLOCK) DEBUG_ERROR((int)DEBUG_CATEGORY_HEADLESS, "headless: accept failed: %s", socket_strerror(e)); return; }
socket_set_nonblocking(cfd);
if (g_hc.client_count >= MAX_CLIENTS) { socket_close_wrapper(cfd); DEBUG_WARN((int)DEBUG_CATEGORY_HEADLESS, "headless: max clients reached"); return; }
if (hc->client_count >= MAX_CLIENTS) { socket_close_wrapper(cfd); DEBUG_WARN((int)DEBUG_CATEGORY_HEADLESS, "headless: max clients reached"); return; }
struct headless_client* cli = u_calloc(1, sizeof(*cli));
if (!cli) { socket_close_wrapper(cfd); DEBUG_ERROR((int)DEBUG_CATEGORY_HEADLESS, "headless: failed to allocate client"); return; }
cli->fd = cfd;
cli->hc = hc;
cli->socket_id = uasync_add_socket_t(g_hc.ua, cfd, client_read_callback, client_write_callback, NULL, cli);
cli->socket_id = uasync_add_socket_t(hc->ua, cfd, client_read_callback, client_write_callback, NULL, cli);
if (!cli->socket_id) { u_free(cli); socket_close_wrapper(cfd); DEBUG_ERROR((int)DEBUG_CATEGORY_HEADLESS, "headless: failed to register client fd"); return; }
cli->next = g_hc.clients; g_hc.clients = cli; g_hc.client_count++;
DEBUG_INFO((int)DEBUG_CATEGORY_HEADLESS, "headless: client connected fd=%d total=%d", (int)cfd, g_hc.client_count);
cli->next = hc->clients; hc->clients = cli; hc->client_count++;
DEBUG_INFO((int)DEBUG_CATEGORY_HEADLESS, "headless: client connected fd=%d total=%d", (int)cfd, hc->client_count);
}
/* ── Periodic cleanup ── */
static void hc_cleanup(void* arg) {
(void)arg;
struct headless_client** prev = &g_hc.clients;
struct headless_control* hc = (struct headless_control*)arg;
struct headless_client** prev = &hc->clients;
while (*prev) {
if ((*prev)->closing) {
struct headless_client* dead = *prev; *prev = dead->next;
if (dead->socket_id && g_hc.ua) uasync_remove_socket_t(g_hc.ua, dead->fd);
if (dead->socket_id && hc->ua) uasync_remove_socket_t(hc->ua, dead->fd);
if (dead->fd != SOCKET_INVALID) socket_close_wrapper(dead->fd);
u_free(dead); g_hc.client_count--;
u_free(dead); hc->client_count--;
} else prev = &(*prev)->next;
}
if (g_hc.ua) uasync_set_timeout(g_hc.ua, 5000, NULL, hc_cleanup, "hc_cleanup");
if (hc->ua) uasync_set_timeout(hc->ua, 5000, hc, hc_cleanup, "hc_cleanup");
}
/* ── Init / Destroy ── */
int chat_headless_control_init(struct UASYNC* ua, struct UTUN_INSTANCE* inst,
const char* bind_ip, int port) {
if (!ua || !bind_ip || port <= 0 || port > 65535) return -1;
if (g_hc.running) return 0;
if (!ua || !inst || !bind_ip || port <= 0 || port > 65535) return -1;
struct headless_control* hc = hc_of(inst);
if (hc && hc->running) return 0;
memset(&g_hc, 0, sizeof(g_hc));
g_hc.ua = ua; g_hc.inst = inst;
if (!hc) {
hc = u_calloc(1, sizeof(*hc));
if (!hc) return -1;
inst->headless = hc;
}
hc->ua = ua; hc->inst = inst;
g_hc.listen_fd = socket(AF_INET, SOCK_STREAM, 0);
if (g_hc.listen_fd == SOCKET_INVALID) { DEBUG_ERROR((int)DEBUG_CATEGORY_HEADLESS, "headless: socket() failed: %s", socket_strerror(socket_get_error())); return -1; }
hc->listen_fd = socket(AF_INET, SOCK_STREAM, 0);
if (hc->listen_fd == SOCKET_INVALID) { DEBUG_ERROR((int)DEBUG_CATEGORY_HEADLESS, "headless: socket() failed: %s", socket_strerror(socket_get_error())); return -1; }
socket_set_reuseaddr(g_hc.listen_fd, 1);
socket_set_nonblocking(g_hc.listen_fd);
socket_set_reuseaddr(hc->listen_fd, 1);
socket_set_nonblocking(hc->listen_fd);
struct sockaddr_in sin; memset(&sin, 0, sizeof(sin));
sin.sin_family = AF_INET; sin.sin_port = htons((uint16_t)port);
if (inet_pton(AF_INET, bind_ip, &sin.sin_addr) != 1) { socket_close_wrapper(g_hc.listen_fd); g_hc.listen_fd = SOCKET_INVALID; return -1; }
if (inet_pton(AF_INET, bind_ip, &sin.sin_addr) != 1) { socket_close_wrapper(hc->listen_fd); hc->listen_fd = SOCKET_INVALID; return -1; }
if (bind(g_hc.listen_fd, (struct sockaddr*)&sin, sizeof(sin)) < 0) {
if (bind(hc->listen_fd, (struct sockaddr*)&sin, sizeof(sin)) < 0) {
DEBUG_ERROR((int)DEBUG_CATEGORY_HEADLESS, "headless: bind(%s:%d) failed: %s", bind_ip, port, socket_strerror(socket_get_error()));
socket_close_wrapper(g_hc.listen_fd); g_hc.listen_fd = SOCKET_INVALID; return -1;
socket_close_wrapper(hc->listen_fd); hc->listen_fd = SOCKET_INVALID; return -1;
}
if (listen(g_hc.listen_fd, 5) < 0) { socket_close_wrapper(g_hc.listen_fd); g_hc.listen_fd = SOCKET_INVALID; return -1; }
if (listen(hc->listen_fd, 5) < 0) { socket_close_wrapper(hc->listen_fd); hc->listen_fd = SOCKET_INVALID; return -1; }
g_hc.listen_sock_id = uasync_add_socket_t(ua, g_hc.listen_fd, accept_callback, NULL, NULL, NULL);
if (!g_hc.listen_sock_id) { socket_close_wrapper(g_hc.listen_fd); g_hc.listen_fd = SOCKET_INVALID; return -1; }
hc->listen_sock_id = uasync_add_socket_t(ua, hc->listen_fd, accept_callback, NULL, NULL, hc);
if (!hc->listen_sock_id) { socket_close_wrapper(hc->listen_fd); hc->listen_fd = SOCKET_INVALID; return -1; }
chat_event_set_handler(hc_on_chat_event);
uasync_set_timeout(ua, 5000, NULL, hc_cleanup, "hc_cleanup");
chat_event_set_handler(inst, hc_on_chat_event);
uasync_set_timeout(ua, 5000, hc, hc_cleanup, "hc_cleanup");
g_hc.running = 1;
hc->running = 1;
DEBUG_INFO((int)DEBUG_CATEGORY_HEADLESS, "headless: listening on %s:%d", bind_ip, port);
return 0;
}
void chat_headless_control_destroy(void) {
if (!g_hc.running) return;
g_hc.running = 0;
chat_event_set_handler(NULL);
if (g_hc.listen_sock_id && g_hc.ua) uasync_remove_socket_t(g_hc.ua, g_hc.listen_fd);
if (g_hc.listen_fd != SOCKET_INVALID) { socket_close_wrapper(g_hc.listen_fd); g_hc.listen_fd = SOCKET_INVALID; }
struct headless_client* c = g_hc.clients;
void chat_headless_control_destroy(struct UTUN_INSTANCE* inst) {
struct headless_control* hc = hc_of(inst);
if (!hc || !hc->running) return;
hc->running = 0;
chat_event_set_handler(inst, NULL);
if (hc->listen_sock_id && hc->ua) uasync_remove_socket_t(hc->ua, hc->listen_fd);
if (hc->listen_fd != SOCKET_INVALID) { socket_close_wrapper(hc->listen_fd); hc->listen_fd = SOCKET_INVALID; }
struct headless_client* c = hc->clients;
while (c) { struct headless_client* n = c->next; if (c->fd != SOCKET_INVALID) socket_close_wrapper(c->fd); u_free(c); c = n; }
g_hc.clients = NULL; g_hc.client_count = 0;
hc->clients = NULL; hc->client_count = 0;
DEBUG_INFO((int)DEBUG_CATEGORY_HEADLESS, "headless: destroyed");
}
+3 -3
View File
@@ -6,8 +6,8 @@
* Response: {"id":N,"ok":true,"data":{...}} | {"id":N,"ok":false,"error":"..."}
* Event: {"event":"<type>",...}
*
* Commands: ping, status, channels, members, messages, send, invite, connect,
* create_channel, subscribe, quit
* Commands: ping, status, channels, members, messages, send, invite,
* invite_nodes, connect, create_channel, subscribe, quit
*/
#ifndef CHAT_HEADLESS_CONTROL_H
#define CHAT_HEADLESS_CONTROL_H
@@ -20,6 +20,6 @@ struct UTUN_INSTANCE;
int chat_headless_control_init(struct UASYNC* ua, struct UTUN_INSTANCE* inst,
const char* bind_ip, int port);
void chat_headless_control_destroy(void);
void chat_headless_control_destroy(struct UTUN_INSTANCE* inst);
#endif /* CHAT_HEADLESS_CONTROL_H */
+339
View File
@@ -0,0 +1,339 @@
/*
* chat_join.c — авторизация входа в канал через connection-узел и инвайтера.
* Маршрутизируемая (etcp_router) часть: регистрация ключа, форвард запроса,
* обработка на инвайтере (верификация + добавление мембера + подпись дерева).
* См. описание flow в chat_join.h.
*/
#include "chat_join.h"
#include "member_sync.h"
#include "../utun_instance.h"
#include "../routing_layer/topo_node_sqlite.h"
#include "../routing_layer/topo_group.h"
#include "../routing_layer/etcp_router.h"
#include "../routing_layer/route_crypto.h"
#include "../transport_layer/etcp_api.h"
#include "../transport_layer/etcp.h"
#include "../transport_layer/secure_channel.h"
#include "../ntp_time.h"
#include "../../lib/mem.h"
#include "../../lib/ll_queue.h"
#include "../../lib/u_async.h"
#include "../../lib/debug_config.h"
#include "../../lib/platform_compat.h"
#include <string.h>
#include <stdlib.h>
#define CJ_ID "chat_join"
/* ── Хранилище ключей на connection-узле ── */
struct join_key_entry {
struct join_key_entry* next;
uint64_t join_key;
uint64_t channel_id;
uint64_t inviter_id; /* кто пригласил (src KEY_REGISTER) */
uint64_t expiry_tb; /* timebase (0.1ms) истечения */
};
static uint64_t _now_tb(void) { return get_time_tb(); }
static void _keys_purge_expired(struct UTUN_INSTANCE* inst) {
uint64_t now = _now_tb();
struct join_key_entry** p = &inst->join_keys;
while (*p) {
if ((*p)->expiry_tb <= now) {
struct join_key_entry* r = *p;
*p = r->next;
DEBUG_DEBUG(DEBUG_CATEGORY_CHAT_SYNC, "%s: join-key expired key=%016llx ch=%llu inviter=%016llx",
CJ_ID, (unsigned long long)r->join_key, (unsigned long long)r->channel_id,
(unsigned long long)r->inviter_id);
u_free(r);
} else {
p = &(*p)->next;
}
}
}
static void _keys_put(struct UTUN_INSTANCE* inst, uint64_t channel_id, uint64_t join_key, uint64_t inviter_id) {
_keys_purge_expired(inst);
struct join_key_entry* e = u_malloc(sizeof(*e));
if (!e) { DEBUG_ERROR(DEBUG_CATEGORY_CHAT_SYNC, "%s: join-key alloc failed", CJ_ID); return; }
e->join_key = join_key; e->channel_id = channel_id; e->inviter_id = inviter_id;
e->expiry_tb = _now_tb() + (uint64_t)JOIN_KEY_TTL_SECONDS * 10000ULL;
e->next = inst->join_keys;
inst->join_keys = e;
DEBUG_INFO(DEBUG_CATEGORY_CHAT_SYNC, "%s: join-key stored key=%016llx ch=%llu inviter=%016llx ttl=%ds",
CJ_ID, (unsigned long long)join_key, (unsigned long long)channel_id,
(unsigned long long)inviter_id, JOIN_KEY_TTL_SECONDS);
}
static struct join_key_entry* _keys_find(struct UTUN_INSTANCE* inst, uint64_t channel_id, uint64_t join_key) {
_keys_purge_expired(inst);
for (struct join_key_entry* e = inst->join_keys; e; e = e->next)
if (e->channel_id == channel_id && e->join_key == join_key) return e;
return NULL;
}
static void _keys_free_all(struct UTUN_INSTANCE* inst) {
struct join_key_entry* e = inst->join_keys;
while (e) { struct join_key_entry* n = e->next; u_free(e); e = n; }
inst->join_keys = NULL;
}
/* ── Сериализация member-блока ── */
int chat_join_serialize_member(const struct join_member_data* m,
uint8_t* out, size_t out_sz, size_t* out_len) {
if (!m || !out || !out_len || !m->x25519 || !m->ed25519) return -1;
size_t ul = strnlen(m->userinfo, 255);
size_t need = 8 + 32 + 32 + 64 + 8 + 64 + 8 + 1 + ul;
if (need > out_sz) return -1;
size_t off = 0;
memcpy(out + off, &m->node_id, 8); off += 8;
memcpy(out + off, m->x25519, 32); off += 32;
memcpy(out + off, m->ed25519, 32); off += 32;
if (m->join_sig) memcpy(out + off, m->join_sig, 64); else memset(out + off, 0, 64); off += 64;
memcpy(out + off, &m->join_ts, 8); off += 8;
if (m->update_sig) memcpy(out + off, m->update_sig, 64); else memset(out + off, 0, 64); off += 64;
memcpy(out + off, &m->update_ts, 8); off += 8;
out[off++] = (uint8_t)ul;
if (ul) memcpy(out + off, m->userinfo, ul);
off += ul;
*out_len = off;
return 0;
}
int chat_join_parse_member(const uint8_t* data, size_t len, struct join_member_data* m) {
if (!data || !m || len < 8 + 32 + 32 + 64 + 8 + 64 + 8 + 1) return -1;
memset(m, 0, sizeof(*m));
size_t off = 0;
memcpy(&m->node_id, data + off, 8); off += 8;
m->x25519 = data + off; off += 32;
m->ed25519 = data + off; off += 32;
m->join_sig = data + off; off += 64;
memcpy(&m->join_ts, data + off, 8); off += 8;
m->update_sig = data + off; off += 64;
memcpy(&m->update_ts, data + off, 8); off += 8;
uint8_t ul = data[off++];
if (off + ul > len) return -1;
if (ul > 255) ul = 255;
memcpy(m->userinfo, data + off, ul);
m->userinfo[ul] = '\0';
return 0;
}
/* ── Обработка на инвайтере: верификация + добавление мембера + подпись дерева ── */
void chat_join_process_request(struct UTUN_INSTANCE* inst, uint64_t group_id,
uint64_t from_node, uint64_t join_key,
const uint8_t* member_data, size_t member_len) {
struct join_member_data m;
if (chat_join_parse_member(member_data, member_len, &m) != 0) {
DEBUG_WARN(DEBUG_CATEGORY_CHAT_SYNC, "%s: process_request parse failed ch=%llu from=%016llx", CJ_ID, (unsigned long long)group_id, (unsigned long long)from_node);
return;
}
char ch_id[32]; snprintf(ch_id, sizeof(ch_id), "%llu", (unsigned long long)group_id);
sqlite3* db = inst->topo_sqlite_db;
/* 0. верификация join-ключа: принимаем только выданные нами (инвайтером) ключи */
struct join_key_entry* ke = _keys_find(inst, group_id, join_key);
if (!ke || ke->inviter_id != inst->node_id) {
DEBUG_WARN(DEBUG_CATEGORY_CHAT_SYNC, "%s: process_request invalid join key ch=%s from=%016llx key=%016llx — drop",
CJ_ID, ch_id, (unsigned long long)from_node, (unsigned long long)join_key);
return;
}
/* 1. привязка идентичности: node_id = derive(x25519) */
if (sc_derive_node_id_from_pubkey(m.x25519) != m.node_id) {
DEBUG_ERROR(DEBUG_CATEGORY_CHAT_SYNC, "%s: process_request node_id/x25519 mismatch (forgery) nid=0x%016llx ch=%s",
CJ_ID, (unsigned long long)m.node_id, ch_id);
return;
}
/* 2. верификация join_sig: sign(ch_x||ch_ed||node_id||x25519||join_ts) ключом ed25519 */
uint8_t ch_x[32], ch_ed[32];
if (!db || topo_node_sqlite_channel_get(db, ch_id, NULL, 0, NULL, ch_x, ch_ed, NULL) != 0) {
DEBUG_WARN(DEBUG_CATEGORY_CHAT_SYNC, "%s: process_request no channel ch=%s", CJ_ID, ch_id);
return;
}
uint8_t vmsg[128];
int vlen = member_sync_build_join_msg(ch_x, ch_ed, m.node_id, m.x25519, m.join_ts, vmsg, (int)sizeof(vmsg));
if (vlen < 0 || sc_ed25519_verify(m.ed25519, vmsg, (size_t)vlen, m.join_sig) != SC_OK) {
DEBUG_ERROR(DEBUG_CATEGORY_CHAT_SYNC, "%s: process_request invalid join_sig nid=0x%016llx ch=%s",
CJ_ID, (unsigned long long)m.node_id, ch_id);
return;
}
/* 3. добавить мембера + подписать его pubkey (дерево приглашений) */
uint8_t tree_sig[64];
if (member_sync_sign_pubkey(inst->my_ed25519_privkey, m.x25519, tree_sig) != 0) {
DEBUG_ERROR(DEBUG_CATEGORY_CHAT_SYNC, "%s: process_request sign_pubkey FAILED nid=0x%016llx", CJ_ID, (unsigned long long)m.node_id);
return;
}
int rc = member_sync_put(inst, ch_id, m.node_id, m.x25519, m.ed25519,
m.join_sig, m.join_ts, m.update_sig, m.update_ts,
m.userinfo, NULL, NULL, 0, inst->node_id, tree_sig);
if (rc < 0) {
DEBUG_ERROR(DEBUG_CATEGORY_CHAT_SYNC, "%s: process_request member_sync_put FAILED ch=%s nid=0x%016llx rc=%d",
CJ_ID, ch_id, (unsigned long long)m.node_id, rc);
return;
}
if (rc & MS_APPLY_CHANGED) {
member_sync_broadcast_one(inst, ch_id, m.node_id);
/* джойнер авторизован — добавляем в CHAT-группу сразу, чтобы BGP подключился */
uint64_t gid = strtoull(ch_id, NULL, 10);
struct TOPO_GROUP* g = topo_groups_find(inst->topo_groups, gid);
if (g && g->group_type == TOPO_GROUP_TYPE_CHAT) {
struct ETCP_CONN* jc = instance_find_conn(inst, m.node_id);
if (jc) {
topo_group_new_conn(g, jc);
DEBUG_INFO(DEBUG_CATEGORY_CHAT_SYNC, "%s: joiner 0x%016llx added to group ch=%s for BGP",
CJ_ID, (unsigned long long)m.node_id, ch_id);
}
}
}
DEBUG_INFO(DEBUG_CATEGORY_CHAT_SYNC, "%s: process_request accepted joiner=0x%016llx ch=%s via=0x%016llx key=%016llx — signed by me",
CJ_ID, (unsigned long long)m.node_id, ch_id, (unsigned long long)from_node, (unsigned long long)join_key);
}
/* ── etcp_router recv ── */
static void join_recv_cb(struct ETCP_CONN* conn, struct ll_entry* entry) {
if (!entry || !conn || !entry->dgram || entry->len < ROUTER_SVC_PAYLOAD_OFF + 1) {
if (entry) { queue_dgram_free(entry); queue_entry_free(entry); }
return;
}
struct UTUN_INSTANCE* inst = conn->instance;
if (!inst) { queue_dgram_free(entry); queue_entry_free(entry); return; }
const uint8_t* d = entry->dgram;
uint64_t from_node; memcpy(&from_node, d + ROUTER_SVC_SRC_OFF, 8);
uint64_t group_id; memcpy(&group_id, d + ROUTER_SVC_GROUP_OFF, 8);
const uint8_t* payload = d + ROUTER_SVC_PAYLOAD_OFF;
size_t plen = entry->len - ROUTER_SVC_PAYLOAD_OFF;
uint8_t subcmd = payload[0];
DEBUG_DEBUG(DEBUG_CATEGORY_CHAT_SYNC, "%s: recv subcmd=%02x from=%016llx ch=%llu len=%zu",
CJ_ID, subcmd, (unsigned long long)from_node, (unsigned long long)group_id, plen);
if (subcmd == JOIN_SUBCMD_KEY_REGISTER) {
/* payload: [subcmd:1][join_key:8] */
if (plen < 1 + 8) { queue_dgram_free(entry); queue_entry_free(entry); return; }
uint64_t join_key; memcpy(&join_key, payload + 1, 8);
_keys_put(inst, group_id, join_key, from_node);
} else if (subcmd == JOIN_SUBCMD_REQUEST_FWD) {
/* payload: [subcmd:1][join_key:8][member...] */
if (plen < 1 + 8) { queue_dgram_free(entry); queue_entry_free(entry); return; }
uint64_t join_key; memcpy(&join_key, payload + 1, 8);
chat_join_process_request(inst, group_id, from_node, join_key, payload + 1 + 8, plen - 1 - 8);
} else {
DEBUG_WARN(DEBUG_CATEGORY_CHAT_SYNC, "%s: unknown subcmd=%02x from=%016llx", CJ_ID, subcmd, (unsigned long long)from_node);
}
queue_dgram_free(entry); queue_entry_free(entry);
}
/* ── Жизненный цикл ── */
int chat_join_init(struct UTUN_INSTANCE* inst) {
if (!inst) return -1;
if (etcp_router_bind(inst, ETCP_RT_ID_JOIN, join_recv_cb) != 0) {
DEBUG_ERROR(DEBUG_CATEGORY_CHAT_SYNC, "%s: etcp_router_bind failed", CJ_ID);
return -1;
}
DEBUG_INFO(DEBUG_CATEGORY_CHAT_SYNC, "%s: initialized", CJ_ID);
return 0;
}
void chat_join_destroy(struct UTUN_INSTANCE* inst) {
if (!inst) return;
etcp_router_unbind(inst, ETCP_RT_ID_JOIN);
_keys_free_all(inst);
DEBUG_INFO(DEBUG_CATEGORY_CHAT_SYNC, "%s: destroyed", CJ_ID);
}
/* ── Публичные операции ── */
void chat_join_register_key(struct UTUN_INSTANCE* inst, uint64_t channel_id,
uint64_t target_node_id, uint64_t join_key) {
if (!inst || channel_id == 0 || join_key == 0) return;
/* инвайтер всегда хранит выданный ключ локально — для верификации REQUEST_FWD */
_keys_put(inst, channel_id, join_key, inst->node_id);
if (target_node_id == inst->node_id || target_node_id == 0) {
return;
}
uint8_t payload[1 + 8];
payload[0] = JOIN_SUBCMD_KEY_REGISTER;
memcpy(payload + 1, &join_key, 8);
struct ll_entry* entry = queue_entry_new(0);
if (!entry) { DEBUG_ERROR(DEBUG_CATEGORY_CHAT_SYNC, "%s: register_key queue_entry_new failed", CJ_ID); return; }
entry->dgram = u_malloc(1 + sizeof(payload));
if (!entry->dgram) { queue_entry_free(entry); return; }
entry->dgram[0] = ETCP_RT_ID_JOIN;
memcpy(entry->dgram + 1, payload, sizeof(payload));
entry->len = 1 + (uint16_t)sizeof(payload);
int rc = etcp_route_send(inst, channel_id, target_node_id, entry, 1, ROUTE_CRYPTO_SIGN | ROUTE_CRYPTO_ENCRYPT);
if (rc != 0) {
DEBUG_ERROR(DEBUG_CATEGORY_CHAT_SYNC, "%s: register_key etcp_route_send FAILED rc=%d ch=%llu → 0x%016llx",
CJ_ID, rc, (unsigned long long)channel_id, (unsigned long long)target_node_id);
queue_dgram_free(entry); queue_entry_free(entry);
return;
}
DEBUG_INFO(DEBUG_CATEGORY_CHAT_SYNC, "%s: join-key registered ch=%llu → connection=0x%016llx key=%016llx",
CJ_ID, (unsigned long long)channel_id, (unsigned long long)target_node_id, (unsigned long long)join_key);
}
uint64_t chat_join_lookup_inviter(struct UTUN_INSTANCE* inst, uint64_t channel_id, uint64_t join_key) {
struct join_key_entry* e = _keys_find(inst, channel_id, join_key);
return e ? e->inviter_id : 0;
}
int chat_join_forward_request(struct UTUN_INSTANCE* inst, uint64_t channel_id,
uint64_t inviter_id, uint64_t join_key,
const struct join_member_data* m) {
if (!inst || !m || inviter_id == 0) return -1;
uint8_t mbuf[512]; size_t mlen = 0;
if (chat_join_serialize_member(m, mbuf, sizeof(mbuf), &mlen) != 0) {
DEBUG_ERROR(DEBUG_CATEGORY_CHAT_SYNC, "%s: forward serialize failed", CJ_ID);
return -1;
}
size_t plen = 1 + 8 + mlen;
uint8_t* payload = u_malloc(plen);
if (!payload) return -1;
payload[0] = JOIN_SUBCMD_REQUEST_FWD;
memcpy(payload + 1, &join_key, 8);
memcpy(payload + 1 + 8, mbuf, mlen);
struct ll_entry* entry = queue_entry_new(0);
if (!entry) { u_free(payload); return -1; }
entry->dgram = u_malloc(1 + plen);
if (!entry->dgram) { u_free(payload); queue_entry_free(entry); return -1; }
entry->dgram[0] = ETCP_RT_ID_JOIN;
memcpy(entry->dgram + 1, payload, plen);
entry->len = 1 + (uint16_t)plen;
u_free(payload);
int rc = etcp_route_send(inst, channel_id, inviter_id, entry, 1, ROUTE_CRYPTO_SIGN | ROUTE_CRYPTO_ENCRYPT);
if (rc != 0) {
DEBUG_ERROR(DEBUG_CATEGORY_CHAT_SYNC, "%s: forward etcp_route_send FAILED rc=%d ch=%llu → 0x%016llx",
CJ_ID, rc, (unsigned long long)channel_id, (unsigned long long)inviter_id);
queue_dgram_free(entry); queue_entry_free(entry);
return -1;
}
DEBUG_INFO(DEBUG_CATEGORY_CHAT_SYNC, "%s: join request forwarded ch=%llu → inviter=0x%016llx joiner=0x%016llx",
CJ_ID, (unsigned long long)channel_id, (unsigned long long)inviter_id, (unsigned long long)m->node_id);
return 0;
}
+94
View File
@@ -0,0 +1,94 @@
/*
* chat_join.h — авторизация входа в канал (join) через connection-узел и инвайтера
*
* Flow (замена старого CHANNEL_INFO_REQ/CHANNEL_JOIN/WELCOME):
*
* инвайтер A connection-узел C джойнер J
* │ KEY_REGISTER(key,ch) │ │
* ├──────etcp_router─────────▶│ хранит {key→A, 10 мин} │
* │ │◀─── JOIN_INFO_REQ(key) ──────┤ (J подключился к C по ссылке)
* │ │── JOIN_INFO_RESP ───────────▶│ (ch_x25519, ch_ed25519, ch_sig, C-мембер)
* │ │◀─── JOIN_REQUEST ────────────┤ (key + полный набор мембера J)
* │ │ (валиден ключ → BGP new_conn, pending status=sent)
* │◀── REQUEST_FWD ───────────┤ │
* │ verif join_sig, node_id=derive(x25519) │
* │ member_sync_put(J, signed_by=A, signature=sign(A, J_x25519||"group_join"))
* │ ── merkle broadcast ──▶ (рекорд J дошёл по merkle) │
* │ │── JOIN_READY(ch) ────────────▶│ → topo_group_new_conn + ensure_channel_ready
*
* Модуль отвечает только за маршрутизируемую (etcp_router) часть:
* - регистрация ключа (инвайтер → connection),
* - форвард join-запроса (connection → инвайтер),
* - обработка на инвайтере: верификация + добавление мембера + подпись дерева.
* Прямой обмен (джойнер ↔ connection, JOIN_INFO_REQ/RESP, JOIN_REQUEST, JOIN_READY)
* и pending-трекинг готовности — в chat_sync.c.
*/
#ifndef CHAT_JOIN_H
#define CHAT_JOIN_H
#include <stdint.h>
#include <stddef.h>
struct UTUN_INSTANCE;
#ifdef __cplusplus
extern "C" {
#endif
/* etcp_router service id (connection ↔ инвайтер) */
#define ETCP_RT_ID_JOIN 0x33
/* subcommands (первый байт payload etcp_router) */
#define JOIN_SUBCMD_KEY_REGISTER 0x01 /* инвайтер → connection: {join_key:8} */
#define JOIN_SUBCMD_REQUEST_FWD 0x02 /* connection → инвайтер: {join_key:8, member...} */
#define JOIN_KEY_TTL_SECONDS 600 /* 10 минут хранения ключа на connection-узле */
/* Полный набор мембера джойнера (без подписи дерева — её ставит инвайтер). */
struct join_member_data {
uint64_t node_id;
const uint8_t* x25519; /* 32 */
const uint8_t* ed25519; /* 32 */
const uint8_t* join_sig; /* 64, может быть NULL */
uint64_t join_ts;
const uint8_t* update_sig; /* 64, может быть NULL */
uint64_t update_ts;
char userinfo[256]; /* JSON, null-terminated */
};
/* Сериализовать member-блок (без join_key). Возвращает 0 и *out_len, -1 если не влез. */
int chat_join_serialize_member(const struct join_member_data* m,
uint8_t* out, size_t out_sz, size_t* out_len);
/* Разобрать member-блок. Указатели ссылаются на data (не копируют). 0=ок, -1=ошибка. */
int chat_join_parse_member(const uint8_t* data, size_t len, struct join_member_data* m);
/* Инициализация: etcp_router_bind(ETCP_RT_ID_JOIN). */
int chat_join_init(struct UTUN_INSTANCE* inst);
/* Деинициализация. */
void chat_join_destroy(struct UTUN_INSTANCE* inst);
/* Инвайтер: зарегистрировать join-ключ на connection-узле.
* target==self → храним локально (инвайтер и есть connection-узел). */
void chat_join_register_key(struct UTUN_INSTANCE* inst, uint64_t channel_id,
uint64_t target_node_id, uint64_t join_key);
/* Connection-узел: найти инвайтера по ключу. 0 = нет/истёк. */
uint64_t chat_join_lookup_inviter(struct UTUN_INSTANCE* inst, uint64_t channel_id, uint64_t join_key);
/* Connection-узел: форварднуть join-запрос инвайтеру по etcp_router. 0=ок, -1=ошибка. */
int chat_join_forward_request(struct UTUN_INSTANCE* inst, uint64_t channel_id,
uint64_t inviter_id, uint64_t join_key,
const struct join_member_data* m);
/* Инвайтер (локально, когда connection==инвайтер, или из recv REQUEST_FWD):
* верификация join_sig + добавление мембера + подпись дерева. */
void chat_join_process_request(struct UTUN_INSTANCE* inst, uint64_t group_id,
uint64_t from_node, uint64_t join_key,
const uint8_t* member_data, size_t member_len);
#ifdef __cplusplus
}
#endif
#endif /* CHAT_JOIN_H */
+110 -80
View File
@@ -11,11 +11,27 @@
#include "member_sync.h"
#include "../utun_instance.h"
#include "../routing_layer/topo_group.h"
#include "../../lib/mem.h"
#include "../../lib/ll_queue.h"
#include "../../lib/json_flat.h"
#include "../../lib/platform_compat.h"
#include <stdlib.h>
/* ─── live-онлайн: узел присутствует в BGP/topo_group канала ─── */
int chat_core_member_online(struct UTUN_INSTANCE* inst, const char* ch_id, uint64_t node_id) {
struct chat_core_ctx* cc = CC(inst);
if (!cc || !cc->inst || !ch_id || !ch_id[0]) return 0;
if (node_id == cc->my_node_id) return 1; /* self всегда онлайн (local_node вне group->nodes) */
if (!cc->inst->topo_groups) return 0;
uint64_t group_id = strtoull(ch_id, NULL, 10);
struct TOPO_GROUP* g = topo_groups_find(cc->inst->topo_groups, group_id);
if (!g || g->group_type != TOPO_GROUP_TYPE_CHAT) return 0;
return topo_node_find_by_id(g, node_id) != NULL;
}
/* ─── парсинг adm_tags в flags ─── */
static uint8_t parse_adm_tags_flags(const char* adm_tags) {
@@ -27,6 +43,10 @@ static uint8_t parse_adm_tags_flags(const char* adm_tags) {
if (json_flat_get(adm_tags, "supernode", buf, sizeof(buf)) == 0
&& strcmp(buf, "yes") == 0) flags |= CHAT_MEMBER_FLAG_SUPERNODE;
/* dm_storage: узел хранит offline-сообщения прямого чата (DM) */
if (json_flat_get(adm_tags, "dm_storage", buf, sizeof(buf)) == 0
&& strcmp(buf, "yes") == 0) flags |= CHAT_MEMBER_FLAG_DM_STORAGE;
/* admin: значение — накопительная строка ed...ed..., последний символ e/d */
if (json_flat_get(adm_tags, "admin", buf, sizeof(buf)) == 0) {
const char* last = buf;
@@ -76,30 +96,31 @@ int chat_core_deserialize_member(const uint8_t* src, struct chat_member_display*
/* ─── запрос списка мемберов ─── */
int chat_core_get_member_list(const char* ch_id, uint8_t** out, int* count) {
int chat_core_get_member_list(struct UTUN_INSTANCE* inst, const char* ch_id, uint8_t** out, int* count) {
struct chat_core_ctx* cc = CC(inst);
if (!out || !count) return -1;
*out = NULL; *count = 0;
if (!g_cc.initialized || !ch_id || !g_cc.db || !g_cc.inst) {
if (!cc || !cc->initialized || !ch_id || !cc->db || !cc->inst) {
DEBUG_ERROR(DEBUG_CATEGORY_CHAT_SYNC, "chat_member: get_list — not initialized");
return -1;
}
char peers_tbl[80]; peers_table_name(ch_id, peers_tbl, sizeof(peers_tbl));
uint64_t my_id = g_cc.my_node_id;
uint64_t my_id = cc->my_node_id;
/* 1. запрашиваем всех мемберов */
sqlite3_stmt* st = NULL;
char sql[512];
snprintf(sql, sizeof(sql),
"SELECT p.node_id, COALESCE(n.online,0), COALESCE(n.name,''),"
"SELECT p.node_id, COALESCE(n.name,''),"
" COALESCE(p.local_nick,''), COALESCE(p.adm_tags,''),"
" COALESCE(p.storage,0)"
" COALESCE(p.storage,0), COALESCE(p.deleted,0)"
" FROM \"%s\" p LEFT JOIN nodes n ON p.node_id=n.node_id"
" ORDER BY n.online DESC, p.node_id ASC", peers_tbl);
" ORDER BY p.node_id ASC", peers_tbl);
if (sqlite3_prepare_v2(g_cc.db, sql, -1, &st, NULL) != SQLITE_OK) {
if (sqlite3_prepare_v2(cc->db, sql, -1, &st, NULL) != SQLITE_OK) {
DEBUG_ERROR(DEBUG_CATEGORY_CHAT_SYNC, "chat_member: get_list — query failed ch=%s err=%s",
ch_id, sqlite3_errmsg(g_cc.db));
ch_id, sqlite3_errmsg(cc->db));
return -1;
}
@@ -115,13 +136,14 @@ int chat_core_get_member_list(const char* ch_id, uint8_t** out, int* count) {
}
struct chat_member_display* m = &members[cnt];
m->node_id = (uint64_t)sqlite3_column_int64(st, 0);
m->online = (uint8_t)sqlite3_column_int(st, 1);
const char* node_name = (const char*)sqlite3_column_text(st, 2);
const char* local_nick = (const char*)sqlite3_column_text(st, 3);
const char* adm_tags = (const char*)sqlite3_column_text(st, 4);
m->storage = (uint8_t)sqlite3_column_int(st, 5);
m->online = (uint8_t)chat_core_member_online(inst, ch_id, m->node_id);
const char* node_name = (const char*)sqlite3_column_text(st, 1);
const char* local_nick = (const char*)sqlite3_column_text(st, 2);
const char* adm_tags = (const char*)sqlite3_column_text(st, 3);
m->storage = (uint8_t)sqlite3_column_int(st, 4);
m->is_self = (uint8_t)(m->node_id == my_id ? 1 : 0);
m->flags = parse_adm_tags_flags(adm_tags);
if (sqlite3_column_int(st, 5)) m->flags |= CHAT_MEMBER_FLAG_DELETED;
m->rtt = 0xFFFF;
m->nat_type = 0xFF;
@@ -154,11 +176,16 @@ int chat_core_get_member_list(const char* ch_id, uint8_t** out, int* count) {
}
snprintf(sql, sizeof(sql),
"SELECT node_id, MIN(rtt), MIN(addr_type) FROM node_addresses"
" WHERE node_id IN (%s) AND rtt > 0 GROUP BY node_id", id_list);
"SELECT node_id, MIN(rtt),"
" CASE WHEN MAX(addr_type=1) THEN 1"
" WHEN MAX(addr_type=2) THEN 2"
" WHEN MAX(addr_type=4) THEN 4"
" WHEN MAX(addr_type=3) THEN 3"
" ELSE 0 END"
" FROM node_addresses WHERE node_id IN (%s) AND rtt > 0 GROUP BY node_id", id_list);
u_free(id_list);
if (sqlite3_prepare_v2(g_cc.db, sql, -1, &st, NULL) == SQLITE_OK) {
if (sqlite3_prepare_v2(cc->db, sql, -1, &st, NULL) == SQLITE_OK) {
while (sqlite3_step(st) == SQLITE_ROW) {
uint64_t nid = (uint64_t)sqlite3_column_int64(st, 0);
int rtt_val = sqlite3_column_int(st, 1);
@@ -174,12 +201,16 @@ int chat_core_get_member_list(const char* ch_id, uint8_t** out, int* count) {
sqlite3_finalize(st);
}
/* 3. проверяем connected (активное ETCP-соединение) */
for (int i = 0; i < cnt; i++) {
if (!members[i].online) continue;
if (g_cc.inst->connections
&& queue_find_data_by_index(g_cc.inst->connections, (const uint8_t*)&members[i].node_id))
members[i].online = 2; /* 2 = connected (online + active conn) */
/* 3. сортируем: онлайн первыми, затем по node_id (стабильно) */
for (int i = 1; i < cnt; i++) {
struct chat_member_display key = members[i];
int j = i - 1;
while (j >= 0 && ((members[j].online < key.online)
|| (members[j].online == key.online && members[j].node_id > key.node_id))) {
members[j + 1] = members[j];
j--;
}
members[j + 1] = key;
}
/* 4. сериализуем в wire-формат */
@@ -197,35 +228,37 @@ int chat_core_get_member_list(const char* ch_id, uint8_t** out, int* count) {
/* ─── один мембер ─── */
int chat_core_get_single_member(const char* ch_id, uint64_t node_id, uint8_t* out) {
int chat_core_get_single_member(struct UTUN_INSTANCE* inst, const char* ch_id, uint64_t node_id, uint8_t* out) {
struct chat_core_ctx* cc = CC(inst);
if (!out) return -1;
if (!g_cc.initialized || !ch_id || !g_cc.db || !g_cc.inst) return -1;
if (!cc || !cc->initialized || !ch_id || !cc->db || !cc->inst) return -1;
char peers_tbl[80]; peers_table_name(ch_id, peers_tbl, sizeof(peers_tbl));
uint64_t my_id = g_cc.my_node_id;
uint64_t my_id = cc->my_node_id;
sqlite3_stmt* st = NULL;
char sql[384];
snprintf(sql, sizeof(sql),
"SELECT COALESCE(n.online,0), COALESCE(n.name,''),"
"SELECT COALESCE(n.name,''),"
" COALESCE(p.local_nick,''), COALESCE(p.adm_tags,''),"
" COALESCE(p.storage,0)"
" COALESCE(p.storage,0), COALESCE(p.deleted,0)"
" FROM \"%s\" p LEFT JOIN nodes n ON p.node_id=n.node_id"
" WHERE p.node_id=?", peers_tbl);
if (sqlite3_prepare_v2(g_cc.db, sql, -1, &st, NULL) != SQLITE_OK) return -1;
if (sqlite3_prepare_v2(cc->db, sql, -1, &st, NULL) != SQLITE_OK) return -1;
sqlite3_bind_int64(st, 1, (sqlite3_int64)node_id);
if (sqlite3_step(st) != SQLITE_ROW) { sqlite3_finalize(st); return -1; }
struct chat_member_display m;
m.node_id = node_id;
m.online = (uint8_t)sqlite3_column_int(st, 0);
const char* node_name = (const char*)sqlite3_column_text(st, 1);
const char* local_nick = (const char*)sqlite3_column_text(st, 2);
const char* adm_tags = (const char*)sqlite3_column_text(st, 3);
m.storage = (uint8_t)sqlite3_column_int(st, 4);
m.online = (uint8_t)chat_core_member_online(inst, ch_id, node_id);
const char* node_name = (const char*)sqlite3_column_text(st, 0);
const char* local_nick = (const char*)sqlite3_column_text(st, 1);
const char* adm_tags = (const char*)sqlite3_column_text(st, 2);
m.storage = (uint8_t)sqlite3_column_int(st, 3);
m.is_self = (uint8_t)(node_id == my_id ? 1 : 0);
m.flags = parse_adm_tags_flags(adm_tags);
if (sqlite3_column_int(st, 4)) m.flags |= CHAT_MEMBER_FLAG_DELETED;
m.rtt = 0xFFFF;
m.nat_type = 0xFF;
@@ -241,9 +274,14 @@ int chat_core_get_single_member(const char* ch_id, uint64_t node_id, uint8_t* ou
/* RTT / NAT */
snprintf(sql, sizeof(sql),
"SELECT MIN(rtt), MIN(addr_type) FROM node_addresses"
" WHERE node_id=? AND rtt > 0");
if (sqlite3_prepare_v2(g_cc.db, sql, -1, &st, NULL) == SQLITE_OK) {
"SELECT MIN(rtt),"
" CASE WHEN MAX(addr_type=1) THEN 1"
" WHEN MAX(addr_type=2) THEN 2"
" WHEN MAX(addr_type=4) THEN 4"
" WHEN MAX(addr_type=3) THEN 3"
" ELSE 0 END"
" FROM node_addresses WHERE node_id=? AND rtt > 0");
if (sqlite3_prepare_v2(cc->db, sql, -1, &st, NULL) == SQLITE_OK) {
sqlite3_bind_int64(st, 1, (sqlite3_int64)node_id);
if (sqlite3_step(st) == SQLITE_ROW) {
int rtt_val = sqlite3_column_int(st, 0);
@@ -253,11 +291,6 @@ int chat_core_get_single_member(const char* ch_id, uint64_t node_id, uint8_t* ou
sqlite3_finalize(st);
}
/* connected */
if (m.online && g_cc.inst->connections
&& queue_find_data_by_index(g_cc.inst->connections, (const uint8_t*)&node_id))
m.online = 2;
chat_core_serialize_member(&m, out);
return 0;
}
@@ -265,13 +298,15 @@ int chat_core_get_single_member(const char* ch_id, uint64_t node_id, uint8_t* ou
/* ─── трамплины для gui_bridge ─── */
void chat_core_request_member_list_trampoline(void* arg) {
if (!arg || !g_cc.initialized) { u_free(arg); return; }
char ch_id[64]; strncpy(ch_id, (const char*)arg, sizeof(ch_id) - 1); ch_id[sizeof(ch_id) - 1] = '\0';
struct chat_member_list_req* req = (struct chat_member_list_req*)arg;
if (!req || !CC(req->inst) || !CC(req->inst)->initialized) { u_free(arg); return; }
struct UTUN_INSTANCE* inst = req->inst;
char ch_id[64]; strncpy(ch_id, req->ch_id, sizeof(ch_id) - 1); ch_id[sizeof(ch_id) - 1] = '\0';
u_free(arg);
uint8_t* members = NULL;
int count = 0;
if (chat_core_get_member_list(ch_id, &members, &count) != 0 || !members) return;
if (chat_core_get_member_list(inst, ch_id, &members, &count) != 0 || !members) return;
size_t ch_len = strlen(ch_id);
size_t evt_sz = 1 + ch_len + 2 + (size_t)count * CHAT_MEMBER_DISPLAY_SIZE;
@@ -283,20 +318,22 @@ void chat_core_request_member_list_trampoline(void* arg) {
memcpy(evt + 1, ch_id, ch_len);
memcpy(evt + 1 + ch_len, &u16cnt, 2);
memcpy(evt + 1 + ch_len + 2, members, (size_t)count * CHAT_MEMBER_DISPLAY_SIZE);
chat_event_post(CHAT_EVT_MEMBER_LIST, evt, (int)evt_sz);
chat_event_post(inst, CHAT_EVT_MEMBER_LIST, evt, (int)evt_sz);
u_free(evt);
u_free(members);
}
void chat_core_request_member_rtt_trampoline(void* arg) {
if (!arg || !g_cc.initialized) { u_free(arg); return; }
char ch_id[64]; strncpy(ch_id, (const char*)arg, sizeof(ch_id) - 1); ch_id[sizeof(ch_id) - 1] = '\0';
struct chat_member_list_req* req = (struct chat_member_list_req*)arg;
if (!req || !CC(req->inst) || !CC(req->inst)->initialized) { u_free(arg); return; }
struct UTUN_INSTANCE* inst = req->inst;
char ch_id[64]; strncpy(ch_id, req->ch_id, sizeof(ch_id) - 1); ch_id[sizeof(ch_id) - 1] = '\0';
u_free(arg);
uint8_t* members = NULL;
int count = 0;
if (chat_core_get_member_list(ch_id, &members, &count) != 0 || !members) return;
if (chat_core_get_member_list(inst, ch_id, &members, &count) != 0 || !members) return;
size_t ch_len = strlen(ch_id);
size_t evt_sz = 1 + ch_len + 1 + CHAT_MEMBER_DISPLAY_SIZE;
@@ -313,50 +350,43 @@ void chat_core_request_member_rtt_trampoline(void* arg) {
memcpy(evt + 1, ch_id, ch_len);
evt[1 + ch_len] = 1;
memcpy(evt + 1 + ch_len + 1, members + i * CHAT_MEMBER_DISPLAY_SIZE, CHAT_MEMBER_DISPLAY_SIZE);
chat_event_post(CHAT_EVT_MEMBER_UPDATED, evt, (int)evt_sz);
chat_event_post(inst, CHAT_EVT_MEMBER_UPDATED, evt, (int)evt_sz);
}
u_free(evt);
u_free(members);
}
/* ─── node_props_changed callback (member_sync → chat_event) ─── */
/* ─── node_props_changed callback (BGP-события узла → MEMBER_UPDATED в GUI) ─── */
static void on_adm_tags_changed(uint64_t node_id, const char* adm_tags, void* arg) {
(void)adm_tags; (void)arg;
if (!g_cc.initialized || !g_cc.inst || !g_cc.db) return;
static void on_member_props_changed(uint64_t node_id, const char* adm_tags, const char* channel_id, void* arg) {
struct UTUN_INSTANCE* inst = (struct UTUN_INSTANCE*)arg;
struct chat_core_ctx* cc = CC(inst);
(void)adm_tags;
if (!cc || !cc->initialized || !cc->inst || !cc->db || !channel_id || !channel_id[0]) return;
struct chat_core_ctx* cc = &g_cc;
for (int i = 0; i < cc->si_count; i++) {
const char* ch_id = cc->si_ch_id[i];
if (!ch_id) continue;
char peers_tbl[80]; peers_table_name(channel_id, peers_tbl, sizeof(peers_tbl));
char sql[256]; snprintf(sql, sizeof(sql), "SELECT 1 FROM \"%s\" WHERE node_id=?", peers_tbl);
char peers_tbl[80]; peers_table_name(ch_id, peers_tbl, sizeof(peers_tbl));
char sql[256]; snprintf(sql, sizeof(sql),
"SELECT 1 FROM \"%s\" WHERE node_id=?", peers_tbl);
sqlite3_stmt* st = NULL;
if (sqlite3_prepare_v2(cc->db, sql, -1, &st, NULL) == SQLITE_OK) {
sqlite3_bind_int64(st, 1, (sqlite3_int64)node_id);
if (sqlite3_step(st) == SQLITE_ROW) {
sqlite3_finalize(st);
size_t cl = strlen(ch_id);
uint8_t evt[1 + 64 + 1 + CHAT_MEMBER_DISPLAY_SIZE];
evt[0] = (uint8_t)cl;
memcpy(evt + 1, ch_id, cl);
evt[1 + cl] = 1;
if (chat_core_get_single_member(ch_id, node_id,
evt + 1 + cl + 1) == 0)
chat_event_post(CHAT_EVT_MEMBER_UPDATED, evt,
1 + (int)cl + 1 + CHAT_MEMBER_DISPLAY_SIZE);
return;
}
sqlite3_stmt* st = NULL;
if (sqlite3_prepare_v2(cc->db, sql, -1, &st, NULL) == SQLITE_OK) {
sqlite3_bind_int64(st, 1, (sqlite3_int64)node_id);
if (sqlite3_step(st) == SQLITE_ROW) {
sqlite3_finalize(st);
size_t cl = strlen(channel_id);
uint8_t evt[1 + 64 + 1 + CHAT_MEMBER_DISPLAY_SIZE];
evt[0] = (uint8_t)cl;
memcpy(evt + 1, channel_id, cl);
evt[1 + cl] = 1;
if (chat_core_get_single_member(inst, channel_id, node_id, evt + 1 + cl + 1) == 0)
chat_event_post(inst, CHAT_EVT_MEMBER_UPDATED, evt, 1 + (int)cl + 1 + CHAT_MEMBER_DISPLAY_SIZE);
return;
}
sqlite3_finalize(st);
}
}
void chat_member_init(void) {
member_sync_add_props_cbk(on_adm_tags_changed, NULL);
DEBUG_INFO(DEBUG_CATEGORY_CHAT_SYNC, "chat_member: initialized (adm_tags → MEMBER_UPDATED)");
void chat_member_init(struct UTUN_INSTANCE* inst) {
member_sync_add_props_cbk(inst, on_member_props_changed, inst);
DEBUG_INFO(DEBUG_CATEGORY_CHAT_SYNC, "chat_member: initialized (BGP node events → MEMBER_UPDATED)");
}
+12 -5
View File
@@ -29,14 +29,20 @@ extern "C" {
#define CHAT_MEMBER_FLAG_SUPERNODE 0x01
#define CHAT_MEMBER_FLAG_ADMIN 0x02
#define CHAT_MEMBER_FLAG_MODER 0x04
#define CHAT_MEMBER_FLAG_DELETED 0x08
#define CHAT_MEMBER_FLAG_DM_STORAGE 0x10
#define CHAT_MEMBER_DISPLAY_SIZE 79
/* Получить всех мемберов канала. out = u_malloc-массив, count = количество. */
int chat_core_get_member_list(const char* ch_id, uint8_t** out, int* count);
int chat_core_get_member_list(struct UTUN_INSTANCE* inst, const char* ch_id, uint8_t** out, int* count);
/* Получить одного мембера по node_id. out = буфер размером CHAT_MEMBER_DISPLAY_SIZE. */
int chat_core_get_single_member(const char* ch_id, uint64_t node_id, uint8_t* out);
int chat_core_get_single_member(struct UTUN_INSTANCE* inst, const char* ch_id, uint64_t node_id, uint8_t* out);
/* Live-онлайн: узел присутствует в BGP/topo_group канала (self всегда онлайн).
Не читает nodes.online — статус не хранится в БД. */
int chat_core_member_online(struct UTUN_INSTANCE* inst, const char* ch_id, uint64_t node_id);
/* Сериализовать из внутренней структуры в wire-формат */
struct chat_member_display {
@@ -54,15 +60,16 @@ void chat_core_serialize_member(const struct chat_member_display* src, uint8_t*
int chat_core_deserialize_member(const uint8_t* src, struct chat_member_display* dst);
/* Трамплин для gui_bridge: GUI запрашивает полный список мемберов канала.
* arg = вызов u_strdup(ch_id). Результат: CHAT_EVT_MEMBER_LIST. */
* arg = struct chat_member_list_req*. Результат: CHAT_EVT_MEMBER_LIST. */
struct chat_member_list_req { struct UTUN_INSTANCE* inst; char ch_id[64]; };
void chat_core_request_member_list_trampoline(void* arg);
/* Трамплин для gui_bridge: GUI запрашивает RTT мемберов канала.
* arg = вызов u_strdup(ch_id). Результат: CHAT_EVT_MEMBER_UPDATED с RTT-обновлениями. */
* arg = struct chat_member_list_req*. Результат: CHAT_EVT_MEMBER_UPDATED с RTT-обновлениями. */
void chat_core_request_member_rtt_trampoline(void* arg);
/* Инициализация: регистрирует node_props_changed callback для рассылки MEMBER_UPDATED. */
void chat_member_init(void);
void chat_member_init(struct UTUN_INSTANCE* inst);
#ifdef __cplusplus
}
+939 -165
View File
File diff suppressed because it is too large. Load diff
+39 -32
View File
@@ -9,6 +9,7 @@
#include "chat_member.h"
#include "../../lib/json_flat.h"
#include "../routing_layer/topo_node_sqlite.h"
#include "../routing_layer/topo_node.h"
#include "member_sync.h"
#include "../utun_instance.h"
@@ -22,36 +23,41 @@
#include <openssl/evp.h>
void chat_core_set_my_node_id(uint64_t node_id) {
g_cc.my_node_id = node_id;
void chat_core_set_my_node_id(struct UTUN_INSTANCE* inst, uint64_t node_id) {
struct chat_core_ctx* cc = CC(inst);
if (cc) cc->my_node_id = node_id;
}
void chat_core_update_my_name(const char* name) {
if (!g_cc.initialized || !name) return;
topo_node_sqlite_node_update_verified(g_cc.db, g_cc.my_node_id,
name, g_cc.inst->my_keys.public_key, g_cc.inst->my_ed25519_pubkey,
(uint64_t)ntp_time_get_seconds(g_cc.inst), 0);
snprintf(g_cc.inst->name, sizeof(g_cc.inst->name), "%s", name);
chat_core_update_my_member();
void chat_core_update_my_name(struct UTUN_INSTANCE* inst, const char* name) {
struct chat_core_ctx* cc = CC(inst);
if (!cc || !cc->initialized || !name) return;
topo_node_sqlite_node_update_verified(cc->db, cc->my_node_id,
name, inst->my_keys.public_key, inst->my_ed25519_pubkey,
(uint64_t)ntp_time_get_seconds(inst), 0);
snprintf(inst->name, sizeof(inst->name), "%s", name);
chat_core_update_my_member(inst);
}
void chat_core_update_my_name_trampoline(void* arg) {
chat_core_update_my_name((const char*)arg);
struct update_my_name_arg* a = (struct update_my_name_arg*)arg;
if (!a) return;
chat_core_update_my_name(a->inst, a->name);
u_free(arg);
}
/* ── Обновление своего мембера (userinfo + update_ts + подпись + синк).
Адреса больше не здесь — их синхронизирует BGP/topo_group. ── */
void chat_core_update_my_member(void) {
if (!g_cc.initialized || !g_cc.db || !g_cc.inst) return;
void chat_core_update_my_member(struct UTUN_INSTANCE* inst) {
struct chat_core_ctx* cc = CC(inst);
if (!cc || !cc->initialized || !cc->db || !inst) return;
uint64_t myid = g_cc.my_node_id;
const char* my_name = g_cc.inst->name[0] ? g_cc.inst->name : "";
uint64_t myid = cc->my_node_id;
const char* my_name = inst->name[0] ? inst->name : "";
char juser[256]; snprintf(juser, sizeof(juser), "{\"name\":\"%s\"}", my_name);
sqlite3_stmt* cs = NULL;
sqlite3_prepare_v2(g_cc.db, "SELECT channel_id FROM channels", -1, &cs, NULL);
sqlite3_prepare_v2(cc->db, "SELECT channel_id FROM channels", -1, &cs, NULL);
if (!cs) return;
while (sqlite3_step(cs) == SQLITE_ROW) {
const char* ch = (const char*)sqlite3_column_text(cs, 0);
@@ -59,26 +65,26 @@ void chat_core_update_my_member(void) {
char tbl[80]; peers_table_name(ch, tbl, sizeof(tbl));
uint8_t join_sig[64]; uint64_t join_ts = 0;
if (topo_node_sqlite_member_get_join(g_cc.db, ch, myid, join_sig, &join_ts) != 0) continue;
if (topo_node_sqlite_member_get_join(cc->db, ch, myid, join_sig, &join_ts) != 0) continue;
uint64_t local_uts = 0;
{ char q[256]; snprintf(q, sizeof(q), "SELECT update_ts FROM \"%s\" WHERE node_id=?", tbl);
sqlite3_stmt* ps = NULL;
if (sqlite3_prepare_v2(g_cc.db, q, -1, &ps, NULL) == SQLITE_OK) {
if (sqlite3_prepare_v2(cc->db, q, -1, &ps, NULL) == SQLITE_OK) {
sqlite3_bind_int64(ps, 1, (sqlite3_int64)myid);
if (sqlite3_step(ps) == SQLITE_ROW) local_uts = (uint64_t)sqlite3_column_int64(ps, 0);
sqlite3_finalize(ps);
}
}
uint64_t now = (uint64_t)ntp_time_get_seconds(g_cc.inst);
uint64_t now = (uint64_t)ntp_time_get_seconds(inst);
uint64_t update_ts = (now > local_uts) ? now : (local_uts + 1); /* строго больше локального */
uint8_t umsg[8192];
int ulen = member_sync_build_update_msg(join_sig, update_ts, juser, umsg, (int)sizeof(umsg));
if (ulen < 0) { DEBUG_ERROR(DEBUG_CATEGORY_CHAT_SYNC, "%s: build_update_msg failed ch=%s", CC_ID, ch); continue; }
uint8_t update_sig[64]; memset(update_sig, 0, 64);
EVP_PKEY* pkey = EVP_PKEY_new_raw_private_key(EVP_PKEY_ED25519, NULL, g_cc.inst->my_ed25519_privkey, 32);
EVP_PKEY* pkey = EVP_PKEY_new_raw_private_key(EVP_PKEY_ED25519, NULL, inst->my_ed25519_privkey, 32);
if (pkey) {
EVP_MD_CTX* mdctx = EVP_MD_CTX_new();
if (mdctx) {
@@ -89,11 +95,11 @@ void chat_core_update_my_member(void) {
EVP_PKEY_free(pkey);
}
int r2 = member_sync_put(g_cc.inst, ch, myid, g_cc.inst->my_keys.public_key,
g_cc.inst->my_ed25519_pubkey, join_sig, join_ts,
update_sig, update_ts, juser, NULL, NULL, 0);
int r2 = member_sync_put(inst, ch, myid, inst->my_keys.public_key,
inst->my_ed25519_pubkey, join_sig, join_ts,
update_sig, update_ts, juser, NULL, NULL, 0, 0, NULL);
if (r2 < 0) DEBUG_ERROR(DEBUG_CATEGORY_CHAT_SYNC, "%s: member_sync_put(self) FAILED ch=%s rc=%d", CC_ID, ch, r2);
else if (r2 & MS_APPLY_CHANGED) member_sync_broadcast_one(g_cc.inst, ch, myid);
else if (r2 & MS_APPLY_CHANGED) member_sync_broadcast_one(inst, ch, myid);
DEBUG_INFO(DEBUG_CATEGORY_MEMBER_SYNC, "%s: update my member ch=%s ts=%llu", CC_ID, ch,
(unsigned long long)update_ts);
@@ -101,16 +107,17 @@ void chat_core_update_my_member(void) {
sqlite3_finalize(cs);
}
void chat_core_sync_my_addresses(void) {
/* Адреса теперь синхронизируются BGP/topo_group (topo_node_update_my_addresses).
Здесь оставлено только обновление userinfo/подписи своего мембера. */
chat_core_update_my_member();
void chat_core_sync_my_addresses(struct UTUN_INSTANCE* inst) {
if (inst)
topo_node_update_my_addresses(inst);
chat_core_update_my_member(inst);
}
void chat_core_save_ui_state(const char* key, const char* value) {
if (!g_cc.initialized || !key || !value) return;
void chat_core_save_ui_state(struct UTUN_INSTANCE* inst, const char* key, const char* value) {
struct chat_core_ctx* cc = CC(inst);
if (!cc || !cc->initialized || !key || !value) return;
sqlite3_stmt* st = NULL;
sqlite3_prepare_v2(g_cc.db,
sqlite3_prepare_v2(cc->db,
"INSERT OR REPLACE INTO ui_state(key, value) VALUES(?, ?)", -1, &st, NULL);
if (st) {
sqlite3_bind_text(st, 1, key, -1, SQLITE_STATIC);
@@ -119,11 +126,11 @@ void chat_core_save_ui_state(const char* key, const char* value) {
}
}
struct save_ui_state_arg { char data[256]; };
void chat_core_save_ui_state_trampoline(void* arg) {
struct save_ui_state_arg* a = (struct save_ui_state_arg*)arg;
if (!a) return;
const char* key = a->data;
const char* value = key + strlen(key) + 1;
chat_core_save_ui_state(key, value);
chat_core_save_ui_state(a->inst, key, value);
u_free(arg);
}
+60 -53
View File
@@ -2,22 +2,19 @@
* chat_setting.c — chat-related settings: validation, storage, on-the-fly apply
*
* Используется и из парсера конфига (headless), и из GUI (через bridge).
* Никаких зависимостей от g_cc или instance — работает автономно.
* Состояние per-instance (struct chat_setting_state).
*/
#include "chat_setting.h"
#include "../utun_instance.h"
#include "../../lib/debug_config.h"
#ifdef DEBUG_CATEGORY_CHAT_SYNC
#define CS_ID DEBUG_CATEGORY_CHAT_SYNC
#else
#define CS_ID DEBUG_CATEGORY_GENERAL
#endif
#include <string.h>
#include <stdio.h>
#include <stdlib.h>
#define CS_ID DEBUG_CATEGORY_CHAT_SYNC
/* ─── типы настроек ─── */
#define CHAT_SETTING_INT 0
@@ -32,12 +29,6 @@ struct chat_setting_def {
int max_val;
};
/* хранилище для строковых значений */
#define CHAT_SETTING_STRING_MAX 16
static int g_chat_string_idx[CHAT_SETTING_STRING_MAX];
static char g_chat_string_val[CHAT_SETTING_STRING_MAX][256];
static int g_chat_string_count = 0;
/* ─── реестр всех известных настроек ─── */
static const struct chat_setting_def g_setting_defs[] = {
@@ -47,48 +38,44 @@ static const struct chat_setting_def g_setting_defs[] = {
{"compressor_max_gain_db", CHAT_SETTING_INT, 25, 1, 100},
{"compressor_rise_rate", CHAT_SETTING_INT, 10, 1, 100},
{"media_download_max_peers", CHAT_SETTING_INT, 3, 1, 16},
{"storage_backfill_days", CHAT_SETTING_INT, 7, 1, 365},
{"standby_active_sec", CHAT_SETTING_INT, 2, 1, 3600},
{"standby_sleep_sec", CHAT_SETTING_INT, 60, 1, 86400},
{"standby_min_sleep_sec", CHAT_SETTING_INT, 15, 1, 86400},
{"join_policy", CHAT_SETTING_INT, 1, 0, 2}, /* 0=autojoin, 1=ask, 2=deny */
{"whisper_enabled", CHAT_SETTING_BOOL, 0, 0, 1},
{"whisper_model_path", CHAT_SETTING_STRING, 0, 0, 0},
{"group_autoconnect", CHAT_SETTING_BOOL, 1, 0, 1},
};
#define CHAT_SETTING_COUNT (sizeof(g_setting_defs) / sizeof(g_setting_defs[0]))
/* ─── состояние (текущие значения) ─── */
/* ─── state helpers ─── */
static int g_chat_values[CHAT_SETTING_COUNT];
static int g_chat_initialized = 0;
static void chat_setting_reset_defaults(void) {
static void chat_setting_reset_defaults(struct chat_setting_state* s) {
for (size_t i = 0; i < CHAT_SETTING_COUNT; i++)
g_chat_values[i] = g_setting_defs[i].default_val;
g_chat_initialized = 1;
s->values[i] = g_setting_defs[i].default_val;
s->initialized = 1;
}
/* ─── поиск ─── */
static int chat_setting_find(const char* name) {
for (size_t i = 0; i < CHAT_SETTING_COUNT; i++)
if (strcmp(g_setting_defs[i].name, name) == 0) return (int)i;
return -1;
}
/* ─── apply: вызов специфичного для настройки кода ─── */
static void chat_setting_apply(void) {
static void chat_setting_apply(struct chat_setting_state* s) {
(void)s;
/* будет заполняться по мере появления runtime-setter'ов в модулях:
if (idx == ...) some_module_set_xxx(value); */
}
/* ─── public API ─── */
/* ─── state-level API ─── */
int chat_setting_set(const char* name, const char* value) {
if (!name || !value || !*name || !*value) return -2;
int chat_setting_set_state(struct chat_setting_state* s, const char* name, const char* value) {
if (!s || !name || !value || !*name || !*value) return -2;
if (!g_chat_initialized) chat_setting_reset_defaults();
if (!s->initialized) chat_setting_reset_defaults(s);
int idx = chat_setting_find(name);
if (idx < 0) {
@@ -100,25 +87,24 @@ int chat_setting_set(const char* name, const char* value) {
int v;
if (def->type == CHAT_SETTING_STRING) {
/* сохраняем строку в отдельном хранилище */
for (int i = 0; i < g_chat_string_count; i++) {
if (g_chat_string_idx[i] == idx) {
if (strcmp(g_chat_string_val[i], value) == 0) return 0;
snprintf(g_chat_string_val[i], sizeof(g_chat_string_val[i]), "%s", value);
for (int i = 0; i < s->string_count; i++) {
if (s->string_idx[i] == idx) {
if (strcmp(s->string_val[i], value) == 0) return 0;
snprintf(s->string_val[i], sizeof(s->string_val[i]), "%s", value);
DEBUG_INFO(CS_ID, "chat_setting: %s = %s", name, value);
chat_setting_apply();
chat_setting_apply(s);
return 0;
}
}
if (g_chat_string_count >= CHAT_SETTING_STRING_MAX) {
if (s->string_count >= CHAT_SETTING_STRING_MAX) {
DEBUG_ERROR(CS_ID, "chat_setting: string storage full for '%s'", name);
return -2;
}
g_chat_string_idx[g_chat_string_count] = idx;
snprintf(g_chat_string_val[g_chat_string_count], sizeof(g_chat_string_val[0]), "%s", value);
g_chat_string_count++;
s->string_idx[s->string_count] = idx;
snprintf(s->string_val[s->string_count], sizeof(s->string_val[0]), "%s", value);
s->string_count++;
DEBUG_INFO(CS_ID, "chat_setting: %s = %s", name, value);
chat_setting_apply();
chat_setting_apply(s);
return 0;
}
@@ -136,34 +122,55 @@ int chat_setting_set(const char* name, const char* value) {
v = (int)lv;
}
if (g_chat_values[idx] == v) return 0;
if (s->values[idx] == v) return 0;
g_chat_values[idx] = v;
s->values[idx] = v;
DEBUG_INFO(CS_ID, "chat_setting: %s = %s", name, value);
chat_setting_apply();
chat_setting_apply(s);
return 0;
}
const char* chat_setting_get(const char* name) {
if (!g_chat_initialized) chat_setting_reset_defaults();
const char* chat_setting_get_state(const struct chat_setting_state* s, const char* name) {
if (!s || !s->initialized) return NULL;
int idx = chat_setting_find(name);
if (idx < 0) return NULL;
const struct chat_setting_def* def = &g_setting_defs[idx];
if (def->type == CHAT_SETTING_STRING) {
for (int i = 0; i < g_chat_string_count; i++) {
if (g_chat_string_idx[i] == idx) return g_chat_string_val[i];
for (int i = 0; i < s->string_count; i++) {
if (s->string_idx[i] == idx) return s->string_val[i];
}
return "";
}
static char buf[16];
snprintf(buf, sizeof(buf), "%d", g_chat_values[idx]);
return buf;
snprintf((char*)s->int_buf, sizeof(s->int_buf), "%d", s->values[idx]);
return s->int_buf;
}
int chat_setting_get_int(const char* name, int def) {
if (!g_chat_initialized) chat_setting_reset_defaults();
int chat_setting_get_int_state(const struct chat_setting_state* s, const char* name, int def) {
if (!s || !s->initialized) return def;
int idx = chat_setting_find(name);
return idx >= 0 ? g_chat_values[idx] : def;
return idx >= 0 ? s->values[idx] : def;
}
/* ─── instance-level API ─── */
static struct chat_setting_state* cs_state(struct UTUN_INSTANCE* inst) {
return inst ? &inst->chat_settings : NULL;
}
int chat_setting_set(struct UTUN_INSTANCE* inst, const char* name, const char* value) {
struct chat_setting_state* s = cs_state(inst);
if (!s) return -2;
return chat_setting_set_state(s, name, value);
}
const char* chat_setting_get(struct UTUN_INSTANCE* inst, const char* name) {
struct chat_setting_state* s = cs_state(inst);
return s ? chat_setting_get_state(s, name) : NULL;
}
int chat_setting_get_int(struct UTUN_INSTANCE* inst, const char* name, int def) {
struct chat_setting_state* s = cs_state(inst);
return s ? chat_setting_get_int_state(s, name, def) : def;
}
+31 -6
View File
@@ -4,23 +4,48 @@
* Один API для всех потребителей: парсер конфига (headless) и GUI (через bridge).
* chat_setting_set(name, value) — валидация + хранение + apply к модулям.
* Возвращает 0=OK, -1=неизвестная настройка, -2=невалидное значение.
*
* Состояние — per-instance. Конфиг-парсер заполняет struct chat_setting_state
* из [chatserver] ещё до создания instance (state-уровень), затем runtime-модули
* читают через instance-уровень (chat_setting_get_int(inst, ...)).
*/
#ifndef CHAT_SETTING_H
#define CHAT_SETTING_H
#include <stdint.h>
struct UTUN_INSTANCE;
#ifdef __cplusplus
extern "C" {
#endif
/* Установить настройку. Применяет на лету если модуль уже инициализирован. */
int chat_setting_set(const char* name, const char* value);
/* ─── per-instance хранилище ─── */
/* Прочитать сохранённое значение. NULL если такой настройки нет. */
const char* chat_setting_get(const char* name);
#define CHAT_SETTING_MAX_SETTINGS 32
#define CHAT_SETTING_STRING_MAX 16
/* Прочитать как int. Возвращает def если настройка не найдена. */
int chat_setting_get_int(const char* name, int def);
struct chat_setting_state {
int values[CHAT_SETTING_MAX_SETTINGS];
int string_idx[CHAT_SETTING_STRING_MAX];
char string_val[CHAT_SETTING_STRING_MAX][256];
int string_count;
int initialized;
char int_buf[16]; /* буфер для chat_setting_get_state у int-настроек */
};
/* ─── state-уровень (конфиг-парсер, до создания instance) ─── */
int chat_setting_set_state(struct chat_setting_state* s, const char* name, const char* value);
const char* chat_setting_get_state(const struct chat_setting_state* s, const char* name);
int chat_setting_get_int_state(const struct chat_setting_state* s, const char* name, int def);
/* ─── instance-уровень (runtime) ─── */
int chat_setting_set(struct UTUN_INSTANCE* inst, const char* name, const char* value);
const char* chat_setting_get(struct UTUN_INSTANCE* inst, const char* name);
int chat_setting_get_int(struct UTUN_INSTANCE* inst, const char* name, int def);
#ifdef __cplusplus
}
+60 -47
View File
@@ -24,11 +24,12 @@ static const char* nat_type_str(uint8_t t) {
switch (t) { case 0: return "UNKNOWN"; case 1: return "EIM"; case 2: return "STRICT"; case 3: return "DIRECT"; default: return "?"; }
}
static void get_node_name(uint64_t node_id, char* out, size_t sz) {
static void get_node_name(struct UTUN_INSTANCE* inst, uint64_t node_id, char* out, size_t sz) {
struct chat_core_ctx* cc = CC(inst);
out[0] = '\0';
if (!g_cc.db || node_id == 0) return;
if (!cc || !cc->db || node_id == 0) return;
sqlite3_stmt* st = NULL;
if (sqlite3_prepare_v2(g_cc.db, "SELECT name FROM nodes WHERE node_id=?", -1, &st, NULL) != SQLITE_OK) return;
if (sqlite3_prepare_v2(cc->db, "SELECT name FROM nodes WHERE node_id=?", -1, &st, NULL) != SQLITE_OK) return;
sqlite3_bind_int64(st, 1, (sqlite3_int64)node_id);
if (sqlite3_step(st) == SQLITE_ROW) {
const char* n = (const char*)sqlite3_column_text(st, 0);
@@ -37,16 +38,16 @@ static void get_node_name(uint64_t node_id, char* out, size_t sz) {
sqlite3_finalize(st);
}
static void chat_core_collect_status(void) {
static void chat_core_collect_status(struct UTUN_INSTANCE* inst) {
char buf[8192]; int off = 0;
if (!g_cc.initialized || !g_cc.inst) {
if (!CC(inst) || !CC(inst)->initialized || !inst) {
off = snprintf(buf, sizeof(buf), "uTun not initialized\n");
chat_event_post(CHAT_EVT_STATUS_REFRESH, (const uint8_t*)buf, off);
chat_event_post(inst, CHAT_EVT_STATUS_REFRESH, (const uint8_t*)buf, off);
return;
}
struct NTP_TIME* ntp = &g_cc.inst->ntp;
struct NTP_TIME* ntp = &inst->ntp;
off += snprintf(buf + off, sizeof(buf) - off, "=== NTP ===\n");
off += snprintf(buf + off, sizeof(buf) - off, "Enabled: %s\n", ntp->enabled ? "yes" : "no");
off += snprintf(buf + off, sizeof(buf) - off, "Synced: %s\n", ntp->synced ? "yes" : "no");
@@ -64,12 +65,12 @@ static void chat_core_collect_status(void) {
}
int conn_count = 0;
struct ll_entry* entry = g_cc.inst->connections->head;
struct ll_entry* entry = inst->connections ? inst->connections->head : NULL;
while (entry) { conn_count++; entry = entry->next; }
off += snprintf(buf + off, sizeof(buf) - off, "=== Connections (%d) ===\n", conn_count);
uint64_t my_id = g_cc.inst->node_id;
entry = g_cc.inst->connections->head;
uint64_t my_id = inst->node_id;
entry = inst->connections ? inst->connections->head : NULL;
while (entry) {
struct conn_queue_entry* ce = (struct conn_queue_entry*)entry->data;
if (!ce || !ce->conn) { entry = entry->next; continue; }
@@ -88,7 +89,7 @@ static void chat_core_collect_status(void) {
snprintf(peerhex, sizeof(peerhex), "%04llX", (unsigned long long)(pid & 0xFFFF));
char peername[64];
get_node_name(pid, peername, sizeof(peername));
get_node_name(inst, pid, peername, sizeof(peername));
if (peername[0])
off += snprintf(buf + off, sizeof(buf) - off, "[%s]→[%s] %s ETCP:%s%s(%dL)\n",
@@ -117,18 +118,18 @@ static void chat_core_collect_status(void) {
entry = entry->next;
}
chat_event_post(CHAT_EVT_STATUS_REFRESH, (const uint8_t*)buf, off);
chat_event_post(inst, CHAT_EVT_STATUS_REFRESH, (const uint8_t*)buf, off);
}
/* ── Бинарный список соединений: [count:2][entry:46B]* ── */
/* entry: peer_node_id:8 flags:1 link_count:1 rtt:2 inflight_kb:2 name:32 */
#define CONN_LIST_ENTRY_SIZE 46
static void collect_conn_list(void) {
if (!g_cc.initialized || !g_cc.inst || !g_cc.inst->connections) return;
static void collect_conn_list(struct UTUN_INSTANCE* inst) {
if (!CC(inst) || !CC(inst)->initialized || !inst || !inst->connections) return;
uint16_t count = 0;
struct ll_entry* entry = g_cc.inst->connections->head;
struct ll_entry* entry = inst->connections->head;
while (entry) { count++; entry = entry->next; }
if (count > 250) count = 250;
@@ -140,7 +141,7 @@ static void collect_conn_list(void) {
*hdr = count;
uint8_t* p = buf + 2;
entry = g_cc.inst->connections->head;
entry = inst->connections->head;
for (uint16_t i = 0; i < count && entry; i++, entry = entry->next) {
struct conn_queue_entry* ce = (struct conn_queue_entry*)entry->data;
if (!ce || !ce->conn) { memset(p, 0, CONN_LIST_ENTRY_SIZE); p += CONN_LIST_ENTRY_SIZE; continue; }
@@ -169,41 +170,41 @@ static void collect_conn_list(void) {
memcpy(p, &inflight_kb, 2); p += 2;
char name[32]; memset(name, 0, 32);
get_node_name(ce->peer_node_id, name, 31);
get_node_name(inst, ce->peer_node_id, name, 31);
if (!name[0]) snprintf(name, 32, "%016llX", (unsigned long long)ce->peer_node_id);
memcpy(p, name, 32); p += 32;
}
chat_event_post(CHAT_EVT_CONN_LIST, buf, (int)buf_sz);
chat_event_post(inst, CHAT_EVT_CONN_LIST, buf, (int)buf_sz);
u_free(buf);
}
/* ── Полный текстовый дамп одного соединения ── */
static void collect_conn_metrics(uint64_t peer_node_id) {
static void collect_conn_metrics(struct UTUN_INSTANCE* inst, uint64_t peer_node_id) {
char buf[8192]; int off = 0;
if (!g_cc.initialized || !g_cc.inst || !g_cc.inst->connections) {
if (!CC(inst) || !CC(inst)->initialized || !inst || !inst->connections) {
off = snprintf(buf, sizeof(buf), "uTun not initialized\n");
chat_event_post(CHAT_EVT_CONN_METRICS, (const uint8_t*)buf, off);
chat_event_post(inst, CHAT_EVT_CONN_METRICS, (const uint8_t*)buf, off);
return;
}
struct ll_entry* e = queue_find_data_by_index(g_cc.inst->connections, (const uint8_t*)&peer_node_id);
struct ll_entry* e = queue_find_data_by_index(inst->connections, (const uint8_t*)&peer_node_id);
if (!e) {
off = snprintf(buf, sizeof(buf), "Connection %016llX not found\n", (unsigned long long)peer_node_id);
chat_event_post(CHAT_EVT_CONN_METRICS, (const uint8_t*)buf, off);
chat_event_post(inst, CHAT_EVT_CONN_METRICS, (const uint8_t*)buf, off);
return;
}
struct conn_queue_entry* ce = (struct conn_queue_entry*)e->data;
if (!ce || !ce->conn) {
off = snprintf(buf, sizeof(buf), "Connection %016llX has null data\n", (unsigned long long)peer_node_id);
chat_event_post(CHAT_EVT_CONN_METRICS, (const uint8_t*)buf, off);
chat_event_post(inst, CHAT_EVT_CONN_METRICS, (const uint8_t*)buf, off);
return;
}
struct ETCP_CONN* conn = ce->conn;
uint64_t my_id = g_cc.inst->node_id;
uint64_t my_id = inst->node_id;
char peername[64]; get_node_name(peer_node_id, peername, sizeof(peername));
char peername[64]; get_node_name(inst, peer_node_id, peername, sizeof(peername));
if (!peername[0]) snprintf(peername, sizeof(peername), "%016llX", (unsigned long long)peer_node_id);
int is_server = conn->links ? conn->links->is_server : -1;
@@ -216,7 +217,7 @@ static void collect_conn_metrics(uint64_t peer_node_id) {
off += snprintf(buf + off, sizeof(buf) - off,
"=== PEER: %04llX\u2192%04llX [%s] ===\n"
"Role: %s\n"
"Status: %s Links: %d/%d Initialized: %s Session: 0x%X MTU: %d\n"
"Status: %s Links: %d/%d Initialized: %s MTU: %d\n"
"Reinit: %u Reset: %u Tx_state: %d Routing_exchange: %d\n"
"\n--- ETCP ---\n"
"RTT last/avg10: %.1f/%.1f ms Jitter: %.1f ms\n"
@@ -228,7 +229,7 @@ static void collect_conn_metrics(uint64_t peer_node_id) {
(unsigned long long)(my_id & 0xFFFF), (unsigned long long)(peer_node_id & 0xFFFF), peername,
role_str,
conn->links_up ? "UP" : "DOWN", links_up_count, link_count,
conn->initialized ? "yes" : "no", conn->session_id, conn->mtu,
conn->initialized ? "yes" : "no", conn->mtu,
conn->reinit_count, conn->reset_count, conn->tx_state, conn->routing_exchange_active,
conn->rtt_last / 10.0f, conn->rtt_avg_10 / 10.0f, conn->jitter / 10.0f,
conn->bytes_sent_total, conn->retransmissions_count, conn->ack_packets_count,
@@ -284,24 +285,25 @@ static void collect_conn_metrics(uint64_t peer_node_id) {
link = link->next; link_idx++;
}
chat_event_post(CHAT_EVT_CONN_METRICS, (const uint8_t*)buf, off);
chat_event_post(inst, CHAT_EVT_CONN_METRICS, (const uint8_t*)buf, off);
}
void chat_core_collect_status_trampoline(void* arg) {
(void)arg;
chat_core_collect_status();
struct UTUN_INSTANCE* inst = (struct UTUN_INSTANCE*)arg;
chat_core_collect_status(inst);
}
void chat_core_collect_conn_list_trampoline(void* arg) {
(void)arg;
collect_conn_list();
struct UTUN_INSTANCE* inst = (struct UTUN_INSTANCE*)arg;
collect_conn_list(inst);
}
void chat_core_collect_conn_metrics_trampoline(void* arg) {
uint64_t peer_id;
memcpy(&peer_id, arg, sizeof(peer_id));
struct chat_node_arg* a = (struct chat_node_arg*)arg;
if (!a) return;
uint64_t peer_id = a->node_id;
collect_conn_metrics(a->inst, peer_id);
u_free(arg);
collect_conn_metrics(peer_id);
}
/* ── Live-снапшот для панели деталей участника ──
@@ -316,7 +318,7 @@ void chat_core_collect_conn_metrics_trampoline(void* arg) {
#define MEMBER_DETAIL_MAX_LINKS 16
#define MEMBER_DETAIL_MAX_SOCKS 16
#define MEMBER_DETAIL_LINK_SIZE 63
#define MEMBER_DETAIL_LINK_SIZE 64
#define MEMBER_DETAIL_SOCK_SIZE 48
/* Повторяет Qt: name.lastIndexOf("_v") → name.mid(3, idx - 3) */
@@ -334,9 +336,8 @@ static void sock_ifname(const char* name, char* out, size_t out_sz) {
out[len] = '\0';
}
static void collect_member_detail(uint64_t node_id) {
if (!g_cc.initialized || !g_cc.inst) return;
struct UTUN_INSTANCE* inst = g_cc.inst;
static void collect_member_detail(struct UTUN_INSTANCE* inst, uint64_t node_id) {
if (!CC(inst) || !CC(inst)->initialized || !inst) return;
uint8_t flags = 0;
struct ETCP_CONN* conn = instance_find_conn(inst, node_id);
@@ -345,6 +346,16 @@ static void collect_member_detail(uint64_t node_id) {
if (conn->links_up) flags |= 2;
if (conn->initialized) flags |= 4;
}
/* bit 3 (0x08): узел присутствует в BGP/topo_group — live-онлайн (self всегда онлайн) */
{
int bgp_present = (node_id == inst->node_id) ? 1 : 0;
struct ll_queue* gl = inst->topo_groups ? inst->topo_groups->group_list : NULL;
for (struct ll_entry* ge = gl ? gl->head : NULL; !bgp_present && ge; ge = ge->next) {
struct TOPO_GROUP* group = (struct TOPO_GROUP*)ge->data;
if (group && topo_node_find_by_id(group, node_id)) bgp_present = 1;
}
if (bgp_present) flags |= 8;
}
uint16_t own_tcp_active = 0;
if (node_id == inst->node_id) {
@@ -424,6 +435,7 @@ static void collect_member_detail(uint64_t node_id) {
if (lk->conn) strncpy(sn, lk->conn->name, 19);
memcpy(p, sn, 20); p += 20;
}
*p++ = lk->reality_set ? 1 : 0;
}
*p++ = sock_count;
@@ -458,7 +470,7 @@ static void collect_member_detail(uint64_t node_id) {
memcpy(p, ifname, 38); p += 38;
}
chat_event_post(CHAT_EVT_MEMBER_DETAIL, buf, (int)(p - buf));
chat_event_post(inst, CHAT_EVT_MEMBER_DETAIL, buf, (int)(p - buf));
u_free(buf);
/* summary text: direct RTT or BGP */
@@ -467,7 +479,7 @@ static void collect_member_detail(uint64_t node_id) {
if (rtt > 0) {
char tbuf[32];
int off = snprintf(tbuf, sizeof(tbuf), "direct, rtt:%ums", rtt / 10);
chat_event_post(CHAT_EVT_BGP_INFO, (const uint8_t*)tbuf, off);
chat_event_post(inst, CHAT_EVT_BGP_INFO, (const uint8_t*)tbuf, off);
}
} else if (conn && !conn->links_up && node_id != inst->node_id) {
struct ll_queue* gl = inst->topo_groups ? inst->topo_groups->group_list : NULL;
@@ -484,13 +496,13 @@ static void collect_member_detail(uint64_t node_id) {
int off = snprintf(tbuf, sizeof(tbuf), "rtt:%ums hop:%u via bgp:", cr / 10, hc);
for (uint8_t hi = 0; hi < hc && off < (int)sizeof(tbuf) - 24; hi++) {
char hn[32] = {0};
get_node_name(hops[hi], hn, sizeof(hn));
get_node_name(inst, hops[hi], hn, sizeof(hn));
uint16_t hid = (uint16_t)(hops[hi] & 0xFFFF);
off += snprintf(tbuf + off, sizeof(tbuf) - (size_t)off,
" [%04X]%s", hid, hn[0] ? " " : "");
if (hn[0]) off += snprintf(tbuf + off, sizeof(tbuf) - (size_t)off, "%s", hn);
}
chat_event_post(CHAT_EVT_BGP_INFO, (const uint8_t*)tbuf, off);
chat_event_post(inst, CHAT_EVT_BGP_INFO, (const uint8_t*)tbuf, off);
}
break;
}
@@ -498,8 +510,9 @@ static void collect_member_detail(uint64_t node_id) {
}
void chat_core_collect_member_detail_trampoline(void* arg) {
uint64_t node_id;
memcpy(&node_id, arg, sizeof(node_id));
struct chat_node_arg* a = (struct chat_node_arg*)arg;
if (!a) return;
uint64_t node_id = a->node_id;
collect_member_detail(a->inst, node_id);
u_free(arg);
collect_member_detail(node_id);
}
+605 -638
View File
File diff suppressed because it is too large. Load diff
+15 -11
View File
@@ -6,12 +6,12 @@
* - запускает member_sync (синхронизацию участников каналов) через merkle_sync
* - управляет кешем каналов (периодический refresh из БД, 30s)
*
* Обслуживает протокол входа в канал между joiner и inviter:
* CHANNEL_INFO_REQ → CHANNEL_INFO_RESP → CHANNEL_JOIN → WELCOME
* Все сообщения с Ed25519-верификацией.
* Обслуживает протокол входа в канал (join) между джойнером, connection-узлом и инвайтером:
* JOIN_INFO_REQ → JOIN_INFO_RESP → JOIN_REQUEST → (форвард инвайтеру) → JOIN_READY.
* Полное описание flow и дерева приглашений — в chat_join.h.
*
* Обрабатывает invite-ссылки (chat_sync_connect_from_invite):
* - сохраняет pubkey и адреса invite-узла в БД (nodes, node_addresses)
* - сохраняет pubkey, join_key и адреса invite-узла в БД (nodes, node_addresses)
* - создаёт TOPO_GROUP, запускает conn_mgr для подключения
* - после поднятия ETCP-соединения запускает join-протокол
*
@@ -47,12 +47,11 @@ struct UASYNC;
#define CS_MSG_ACK_PUSH 0x05
#define CS_MSG_SYNC_DONE 0x06
#define CS_MSG_ERROR 0x07
#define CS_MSG_CHANNEL_INFO_REQ 0x09
#define CS_MSG_CHANNEL_INFO_RESP 0x0A
#define CS_MSG_CHANNEL_JOIN 0x0B
#define CS_MSG_WELCOME 0x0C
#define CS_MSG_PEER_UPSERT 0x0D
#define CS_MSG_CHANNEL_INVITE 0x0F /* push-invite: inviter → joiner */
#define CS_MSG_JOIN_INFO_REQ 0x10 /* joiner → connection: {join_key:8} */
#define CS_MSG_JOIN_INFO_RESP 0x11 /* connection → joiner: channel info + connection's member record */
#define CS_MSG_JOIN_REQUEST 0x12 /* joiner → connection: {join_key:8, member...} */
#define CS_MSG_JOIN_READY 0x13 /* connection → joiner: {group_id:8} */
/* Protocol constants */
#define CS_SEND_DATA_MAX 32
@@ -73,12 +72,13 @@ void chat_sync_destroy(struct UTUN_INSTANCE* inst);
void chat_sync_connect_node(struct UTUN_INSTANCE* inst, uint64_t node_id);
/* Подключиться к пиру по данным invite-ссылки (вызывается из uasync-потока).
join_key — 64-битный ключ входа из ссылки (регистрируется инвайтером на connection-узле).
password может быть NULL — тогда отправляется без пароля (совместимость с v1) */
void chat_sync_connect_from_invite(uint64_t channel_id, uint64_t node_id,
void chat_sync_connect_from_invite(struct UTUN_INSTANCE* inst, uint64_t channel_id, uint64_t node_id,
const uint8_t* pubkey_bin,
const uint8_t* addrs_data, int addr_count,
int addrs_data_len,
const char* password);
const char* password, uint64_t join_key);
/* Пригласить target_node_id в канал ch_id с адресами из invite-ссылки (inviter → joiner).
Принимает pubkey и адреса целевого узла. Если соединения ещё нет — строит TOPO_NODE
@@ -111,6 +111,10 @@ void chat_sync_join_channel(struct UTUN_INSTANCE* inst, const char* ch_id, uint6
Вызывается из auto_socket при появлении/изменении сетевой связности. */
void chat_sync_retry_channels_on_socket_change(struct UTUN_INSTANCE* inst);
/* Немедленно убрать канал из кеша g_cs->channels (локальное удаление канала),
* чтобы cs_on_conn_status не запускал member_sync для удалённого ns до следующего refresh. */
void chat_sync_on_channel_deleted(struct UTUN_INSTANCE* inst, const char* ch_id);
#ifdef __cplusplus
}
#endif
+17 -11
View File
@@ -38,6 +38,7 @@ static int g_initialized = 0;
/* очередь транскрипций: whisper не thread-safe, обрабатываем по одной */
struct wh_job {
struct UTUN_INSTANCE* inst;
char audio_path[1024];
char channel_id[64];
uint64_t reply_to_ts;
@@ -231,12 +232,14 @@ static float* resample_16k(const float* src, int n_src, int src_rate, int* n_dst
struct wh_work_ctx;
static void chat_whisper_transcribe_async_impl(
struct UTUN_INSTANCE* inst,
struct media_async* ma, struct UASYNC* ua,
const char* audio_path, const char* channel_id,
uint64_t reply_to_ts, uint64_t reply_to_node,
chat_whisper_done_fn done_cb, void* done_arg);
static void chat_whisper_transcribe_async(
struct UTUN_INSTANCE* inst,
struct media_async* ma, struct UASYNC* ua,
const char* audio_path, const char* channel_id,
uint64_t reply_to_ts, uint64_t reply_to_node,
@@ -360,7 +363,7 @@ static void wh_done_fn(void* raw, int err) {
if (g_pending_job) {
struct wh_job* nj = g_pending_job;
g_pending_job = NULL;
chat_whisper_transcribe_async_impl(nj->ma, nj->ua, nj->audio_path, nj->channel_id,
chat_whisper_transcribe_async_impl(nj->inst, nj->ma, nj->ua, nj->audio_path, nj->channel_id,
nj->reply_to_ts, nj->reply_to_node,
nj->done_cb, nj->done_arg);
u_free(nj);
@@ -369,8 +372,8 @@ static void wh_done_fn(void* raw, int err) {
/* ─── авто-поиск модели ─── */
static int find_model_path(char* out, size_t out_sz) {
const char* setting = chat_setting_get("whisper_model_path");
static int find_model_path(struct UTUN_INSTANCE* inst, char* out, size_t out_sz) {
const char* setting = chat_setting_get(inst, "whisper_model_path");
if (setting && setting[0] != '\0') {
/* если путь относительный, не проверяем доступ — вернём как есть */
if (access(setting, R_OK) == 0) { snprintf(out, out_sz, "%s", setting); return 0; }
@@ -397,7 +400,7 @@ static int find_model_path(char* out, size_t out_sz) {
/* ─── публичный API ─── */
int chat_whisper_init(void) {
int chat_whisper_init(struct UTUN_INSTANCE* inst) {
if (g_initialized) return 0;
g_initialized = 1;
@@ -405,13 +408,13 @@ int chat_whisper_init(void) {
DEBUG_WARN(DEBUG_CATEGORY_CHAT, "%s: compiled without whisper support", CW_ID);
return -1;
#else
if (!chat_setting_get_int("whisper_enabled", 0)) {
if (!chat_setting_get_int(inst, "whisper_enabled", 0)) {
DEBUG_INFO(DEBUG_CATEGORY_CHAT, "%s: whisper disabled in settings", CW_ID);
return -1;
}
char model_path[512];
if (find_model_path(model_path, sizeof(model_path)) != 0) {
if (find_model_path(inst, model_path, sizeof(model_path)) != 0) {
DEBUG_ERROR(DEBUG_CATEGORY_CHAT, "%s: model not found (set whisper_model_path)", CW_ID);
return -1;
}
@@ -432,9 +435,9 @@ int chat_whisper_init(void) {
#endif /* HAVE_WHISPER */
}
int chat_whisper_available(void) {
int chat_whisper_available(struct UTUN_INSTANCE* inst) {
#ifdef HAVE_WHISPER
return g_whisper_ctx != NULL && chat_setting_get_int("whisper_enabled", 0);
return g_whisper_ctx != NULL && chat_setting_get_int(inst, "whisper_enabled", 0);
#else
return 0;
#endif
@@ -455,17 +458,19 @@ void chat_whisper_destroy(void) {
}
void chat_whisper_transcribe_async(
struct UTUN_INSTANCE* inst,
struct media_async* ma, struct UASYNC* ua,
const char* audio_path, const char* channel_id,
uint64_t reply_to_ts, uint64_t reply_to_node,
chat_whisper_done_fn done_cb, void* done_arg)
{
chat_whisper_transcribe_async_impl(ma, ua, audio_path, channel_id,
chat_whisper_transcribe_async_impl(inst, ma, ua, audio_path, channel_id,
reply_to_ts, reply_to_node,
done_cb, done_arg);
}
static void chat_whisper_transcribe_async_impl(
struct UTUN_INSTANCE* inst,
struct media_async* ma, struct UASYNC* ua,
const char* audio_path, const char* channel_id,
uint64_t reply_to_ts, uint64_t reply_to_node,
@@ -477,8 +482,8 @@ static void chat_whisper_transcribe_async_impl(
return;
}
if (!chat_whisper_available()) {
int rc = chat_whisper_init();
if (!chat_whisper_available(inst)) {
int rc = chat_whisper_init(inst);
if (rc != 0) {
DEBUG_WARN(DEBUG_CATEGORY_CHAT, "%s: whisper not available, skip transcription", CW_ID);
done_cb(done_arg, channel_id, NULL, -1, reply_to_ts, reply_to_node);
@@ -491,6 +496,7 @@ static void chat_whisper_transcribe_async_impl(
if (g_pending_job) { u_free(g_pending_job); } /* заменяем */
g_pending_job = u_malloc(sizeof(*g_pending_job));
if (!g_pending_job) { done_cb(done_arg, channel_id, NULL, -1, reply_to_ts, reply_to_node); return; }
g_pending_job->inst = inst;
snprintf(g_pending_job->audio_path, sizeof(g_pending_job->audio_path), "%s", audio_path);
snprintf(g_pending_job->channel_id, sizeof(g_pending_job->channel_id), "%s", channel_id);
g_pending_job->reply_to_ts = reply_to_ts;
+4 -2
View File
@@ -24,11 +24,12 @@ extern "C" {
struct media_async;
struct UASYNC;
struct UTUN_INSTANCE;
/* ── жизненный цикл ── */
int chat_whisper_init(void); /* ленивая загрузка модели */
int chat_whisper_available(void); /* модель загружена и готова */
int chat_whisper_init(struct UTUN_INSTANCE* inst); /* ленивая загрузка модели */
int chat_whisper_available(struct UTUN_INSTANCE* inst); /* модель загружена и готова */
void chat_whisper_destroy(void); /* выгрузка модели */
/* ── асинхронная транскрипция ── */
@@ -38,6 +39,7 @@ typedef void (*chat_whisper_done_fn)(void* arg, const char* channel_id,
uint64_t reply_ts, uint64_t reply_node);
typedef void (*chat_whisper_trigger_fn)(
struct UTUN_INSTANCE* inst,
struct media_async* ma, struct UASYNC* ua,
const char* audio_path, const char* channel_id,
uint64_t reply_ts, uint64_t reply_node,
+115 -58
View File
@@ -6,6 +6,7 @@
#include "../utun_instance.h"
#include "../routing_layer/topo_group.h"
#include "../routing_layer/topo_node_sqlite.h"
#include "member_sync.h"
#include "../transport_layer/etcp_dump.h"
#include "../transport_layer/secure_channel.h"
#include "../../lib/debug_config.h"
@@ -33,7 +34,7 @@ static void db_sync_peer_check_cb(void* arg);
static void db_sync_resume_peer_check(struct DB_SYNC* db);
static void db_sync_instance_ttl_cb(void* arg);
static int db_sync_send(struct DB_SYNC_INSTANCE* si, uint64_t dst_node_id, const uint8_t* payload, size_t len);
static int db_sync_send_hash(struct DB_SYNC* db, uint64_t node_id, uint64_t hash, const uint8_t* payload, size_t len);
static int db_sync_send_gid(struct DB_SYNC* db, uint64_t node_id, uint64_t group_id, const uint8_t* payload, size_t len);
static void db_sync_initiate_sync(struct DB_SYNC_INSTANCE* si, uint64_t peer_node_id);
static int si_send_data_batch(struct DB_SYNC_INSTANCE* si, uint64_t dst, uint32_t from, uint32_t count, uint32_t vp, uint32_t want_from);
static void si_delivery_update(struct DB_SYNC_INSTANCE* si, uint64_t ts, uint64_t author, uint64_t peer_id);
@@ -54,7 +55,7 @@ struct SI_PEER {
struct DB_SYNC_INSTANCE {
struct DB_SYNC* db_sync;
uint64_t hash;
uint64_t group_id;
char table_name[64];
uint64_t next_id;
uint64_t last_timestamp_ms;
@@ -68,6 +69,7 @@ struct DB_SYNC_INSTANCE {
int peer_count, peer_capacity;
db_sync_insert_cb on_insert;
void* on_insert_arg;
uint8_t sync_gated; /* 1 = не запускать авто-синхронизацию (ждём member_sync/pubkeys) */
};
struct db_sync_done_cbk_entry {
@@ -81,7 +83,7 @@ struct DB_SYNC {
sqlite3* db;
uint8_t shared_db; /* db == inst->topo_sqlite_db, do not close */
uint64_t last_connected_tb;
struct DB_SYNC_INSTANCE* instances;
struct DB_SYNC_INSTANCE** instances;
int instance_count, instance_capacity;
void* peer_check_timer;
void* peer_check_wait; /* standby_wait handle (Android) */
@@ -128,7 +130,7 @@ void db_sync_remove_done_cbk(struct UTUN_INSTANCE* inst, db_sync_done_fn fn, voi
#define SI_TBL(si) ((si)->table_name)
#define SI_SHRT(si) ({ \
static char _tbuf[28]; \
snprintf(_tbuf, sizeof(_tbuf), "%s.%04llX", SI_TBL(si), (unsigned long long)((si)->hash & 0xFFFF)); \
snprintf(_tbuf, sizeof(_tbuf), "%s.%04llX", SI_TBL(si), (unsigned long long)((si)->group_id & 0xFFFF)); \
_tbuf; \
})
@@ -220,30 +222,37 @@ static void db_chain_hash_compute(const uint8_t prev_chain_hash[32],
// Instance management
// ============================================================
// Ищет активный инстанс синхронизации по 64-битному хешу
static struct DB_SYNC_INSTANCE* db_instance_find(struct DB_SYNC* db, uint64_t hash)
// Ищет активный инстанс синхронизации по group_id канала
static struct DB_SYNC_INSTANCE* db_instance_find(struct DB_SYNC* db, uint64_t group_id)
{
for (int i = 0; i < db->instance_count; i++) {
if (db->instances[i].hash == hash && db->instances[i].enabled) return &db->instances[i];
if (db->instances[i]->group_id == group_id && db->instances[i]->enabled) return db->instances[i];
}
return NULL;
}
// Выделяет новый слот инстанса в массиве, расширяя его при необходимости
struct DB_SYNC_INSTANCE* db_sync_instance_find(struct UTUN_INSTANCE* inst, uint64_t group_id)
{
if (!inst || !inst->db_sync) return NULL;
return db_instance_find(inst->db_sync, group_id);
}
// Выделяет новый инстанс в куче (стабильный указатель) и регистрирует его в массиве
static struct DB_SYNC_INSTANCE* db_instance_alloc(struct DB_SYNC* db)
{
DEBUG_TRACE(DEBUG_CATEGORY_CHAT_SYNC, "count=%d cap=%d", db->instance_count, db->instance_capacity);
if (db->instance_count >= db->instance_capacity) {
int nc = db->instance_capacity ? db->instance_capacity * 2 : 4;
struct DB_SYNC_INSTANCE* np = u_realloc(db->instances, nc * sizeof(*db->instances));
struct DB_SYNC_INSTANCE** np = u_realloc(db->instances, nc * sizeof(*db->instances));
if (!np) return NULL;
db->instances = np;
db->instance_capacity = nc;
}
struct DB_SYNC_INSTANCE* si = &db->instances[db->instance_count++];
memset(si, 0, sizeof(*si));
struct DB_SYNC_INSTANCE* si = u_calloc(1, sizeof(*si));
if (!si) return NULL;
si->db_sync = db;
si->enabled = 1;
db->instances[db->instance_count++] = si;
return si;
}
@@ -672,24 +681,24 @@ static int db_record_insert_cascade(struct DB_SYNC_INSTANCE* si,
// Send
// ============================================================
// Отправляет сообщение узлу через ETCP с заголовком: [ETCP_RT_ID_DB_SYNC:1][hash_be:8][payload]. Возвращает код etcp_send
static int db_sync_send_hash(struct DB_SYNC* db, uint64_t node_id, uint64_t hash, const uint8_t* payload, size_t plen)
// Отправляет сообщение узлу через ETCP с заголовком: [ETCP_RT_ID_DB_SYNC:1][group_id_be:8][payload]. Возвращает код etcp_send
static int db_sync_send_gid(struct DB_SYNC* db, uint64_t node_id, uint64_t group_id, const uint8_t* payload, size_t plen)
{
struct ll_entry* entry = queue_entry_new(0);
if (!entry) { DEBUG_ERROR(DEBUG_CATEGORY_CHAT_SYNC, "queue_entry_new"); return -1; }
uint8_t* buf = u_malloc(plen + 9);
if (!buf) { DEBUG_ERROR(DEBUG_CATEGORY_CHAT_SYNC, "db_sync_send_hash: u_malloc(%zu) failed for dst=%016llx", plen + 9, (unsigned long long)node_id); queue_entry_free(entry); return -1; }
if (!buf) { DEBUG_ERROR(DEBUG_CATEGORY_CHAT_SYNC, "db_sync_send_gid: u_malloc(%zu) failed for dst=%016llx", plen + 9, (unsigned long long)node_id); queue_entry_free(entry); return -1; }
buf[0] = ETCP_RT_ID_DB_SYNC;
uint64_t hb = htobe64(hash);
memcpy(buf + 1, &hb, 8);
uint64_t gb = htobe64(group_id);
memcpy(buf + 1, &gb, 8);
memcpy(buf + 9, payload, plen);
entry->dgram = buf;
entry->len = plen + 9;
struct ETCP_CONN* conn = instance_find_conn(db->inst, node_id);
if (!conn || !conn->links_up) {
DEBUG_INFO(DEBUG_CATEGORY_CHAT_SYNC, "sync: send DROP — no conn to %04llX (hash=%016llx, type=%02x)",
(unsigned long long)(node_id >> 16), hash, plen > 0 ? payload[0] : 0,
DEBUG_INFO(DEBUG_CATEGORY_CHAT_SYNC, "sync: send DROP — no conn to %04llX (group=%016llx, type=%02x)",
(unsigned long long)(node_id >> 16), group_id, plen > 0 ? payload[0] : 0,
(void*)conn, conn ? conn->links_up : -1);
queue_entry_free(entry);
return -1;
@@ -700,10 +709,10 @@ static int db_sync_send_hash(struct DB_SYNC* db, uint64_t node_id, uint64_t hash
return ret;
}
// Отправляет сообщение через ETCP от имени конкретного инстанса (подставляет хеш инстанса)
// Отправляет сообщение через ETCP от имени конкретного инстанса (подставляет group_id инстанса)
static int db_sync_send(struct DB_SYNC_INSTANCE* si, uint64_t dst_node_id, const uint8_t* payload, size_t len)
{
return db_sync_send_hash(si->db_sync, dst_node_id, si->hash, payload, len);
return db_sync_send_gid(si->db_sync, dst_node_id, si->group_id, payload, len);
}
// ============================================================
@@ -855,8 +864,10 @@ static void db_handle_init_resp(struct DB_SYNC_INSTANCE* si, uint64_t src, const
struct SI_PEER* sp = si_peer_find(si, src);
// Peer empty → send ALL our data, then SYNC_DONE will be sent from DATA handler when slave responds
if (peer_ch8 == 0 && sc == 0) {
// Peer empty → send ALL our data, then SYNC_DONE will be sent from DATA handler when slave responds.
// Если мы тоже пусты (mc==0) — обе стороны пусты: не отправляем ничего, а завершаем sync через
// SYNC_DONE в ветке "exact match" ниже (иначе обе стороны вечно ждут ответа → TIMEOUT).
if (peer_ch8 == 0 && sc == 0 && mc > 0) {
uint32_t vp = (uint32_t)-1; if (sp) sp->verified_pos = vp;
uint32_t sent = 0;
while (sent < mc) {
@@ -1202,15 +1213,15 @@ static void db_sync_recv_cb(struct ETCP_CONN* conn, struct ll_entry* entry)
if (!db || !db->enabled) { queue_dgram_free(entry); queue_entry_free(entry); return; }
uint64_t src = conn ? conn->peer_node_id : 0;
uint64_t hash = be64toh(*(uint64_t*)(entry->dgram + 1));
uint64_t group_id = be64toh(*(uint64_t*)(entry->dgram + 1));
uint8_t type = entry->dgram[9];
const uint8_t* payload = entry->dgram + 10;
size_t plen = entry->len - 10;
DEBUG_TRACE(DEBUG_CATEGORY_CHAT_SYNC, "recv type=%02x from=%016llx hash=%016llx len=%zu", type, (unsigned long long)src, (unsigned long long)hash, plen);
DEBUG_TRACE(DEBUG_CATEGORY_CHAT_SYNC, "recv type=%02x from=%016llx group=%016llx len=%zu", type, (unsigned long long)src, (unsigned long long)group_id, plen);
struct DB_SYNC_INSTANCE* si = db_instance_find(db, hash);
struct DB_SYNC_INSTANCE* si = db_instance_find(db, group_id);
if (!si) {
/* lazy-register: find table msg_<ch_id> with matching SHA256 */
/* lazy-register: найти таблицу msg_<ch_id>, у которой strtoull(ch_id) == group_id */
sqlite3_stmt* st = NULL;
if (sqlite3_prepare_v2(db->db,
"SELECT name, substr(name,5) FROM sqlite_master WHERE type='table' AND name LIKE 'msg_%'",
@@ -1219,11 +1230,11 @@ static void db_sync_recv_cb(struct ETCP_CONN* conn, struct ll_entry* entry)
const char* tbl = (const char*)sqlite3_column_text(st, 0);
const char* ch_id = (const char*)sqlite3_column_text(st, 1);
if (!ch_id || !ch_id[0]) continue;
uint64_t h; { uint8_t sh[32]; SHA256((const uint8_t*)ch_id, strlen(ch_id), sh); memcpy(&h, sh, 8); }
if (h == hash) {
si = db_sync_instance_add(db->inst, tbl, hash, 1);
uint64_t gid = strtoull(ch_id, NULL, 10);
if (gid == group_id) {
si = db_sync_instance_add(db->inst, tbl, group_id, 1);
if (si) { si_peer_add(si, src); }
DEBUG_INFO(DEBUG_CATEGORY_CHAT_SYNC, "lazy-register: tbl=%s ch=%s hash=%016llx", tbl, ch_id, (unsigned long long)hash);
DEBUG_INFO(DEBUG_CATEGORY_CHAT_SYNC, "lazy-register: tbl=%s ch=%s group=%016llx", tbl, ch_id, (unsigned long long)group_id);
break;
}
}
@@ -1231,20 +1242,30 @@ static void db_sync_recv_cb(struct ETCP_CONN* conn, struct ll_entry* entry)
}
if (!si) {
DEBUG_TRACE(DEBUG_CATEGORY_CHAT_SYNC,
"recv msg type=0x%02x from %016llx hash=%016llx — instance NOT FOUND; sending DB_ERR_NOT_FOUND",
type, (unsigned long long)src, (unsigned long long)hash);
DEBUG_TRACE(DEBUG_CATEGORY_CHAT_SYNC, "→ instance not found for hash=%016llx", (unsigned long long)hash);
"recv msg type=0x%02x from %016llx group=%016llx — instance NOT FOUND; sending DB_ERR_NOT_FOUND",
type, (unsigned long long)src, (unsigned long long)group_id);
DEBUG_TRACE(DEBUG_CATEGORY_CHAT_SYNC, "→ instance not found for group=%016llx", (unsigned long long)group_id);
uint8_t err[2]; err[0] = DB_MSG_ERROR; err[1] = DB_ERR_NOT_FOUND;
db_sync_send_hash(db, src, hash, err, 2);
db_sync_send_gid(db, src, group_id, err, 2);
queue_dgram_free(entry); queue_entry_free(entry); return;
}
}
if (!si->enabled) {
uint8_t err[2]; err[0] = DB_MSG_ERROR; err[1] = DB_ERR_DISABLED;
db_sync_send_hash(db, src, hash, err, 2);
db_sync_send_gid(db, src, group_id, err, 2);
queue_dgram_free(entry); queue_entry_free(entry); return;
}
/* валидация клиента: pubkey из ETCP-соединения должен быть в таблице мемберов канала */
{
char ch_id[32]; snprintf(ch_id, sizeof(ch_id), "%llu", (unsigned long long)group_id);
if (conn && !member_sync_validate_pubkey(conn->instance, ch_id, conn->crypto_ctx.peer_public_key)) {
DEBUG_WARN(DEBUG_CATEGORY_CHAT_SYNC, "sync: recv DROP — peer=0x%016llx not member of ch=%s (type=%02x)",
(unsigned long long)src, ch_id, type);
queue_dgram_free(entry); queue_entry_free(entry); return;
}
}
switch (type) {
case DB_MSG_INIT_SYNC: DEBUG_TRACE(DEBUG_CATEGORY_CHAT_SYNC, "→ handle INIT_SYNC"); db_handle_init_sync(si, src, payload, plen); break;
case DB_MSG_INIT_RESP: DEBUG_TRACE(DEBUG_CATEGORY_CHAT_SYNC, "→ handle INIT_RESP"); db_handle_init_resp(si, src, payload, plen); break;
@@ -1255,8 +1276,8 @@ static void db_sync_recv_cb(struct ETCP_CONN* conn, struct ll_entry* entry)
case DB_MSG_SYNC_DONE: DEBUG_TRACE(DEBUG_CATEGORY_CHAT_SYNC, "→ handle SYNC_DONE"); db_handle_sync_done(si, src, payload, plen); break;
case DB_MSG_ERROR: DEBUG_TRACE(DEBUG_CATEGORY_CHAT_SYNC, "→ handle ERROR"); db_handle_error(si, src, payload, plen); break;
default:
DEBUG_WARN(DEBUG_CATEGORY_CHAT_SYNC, "sync: unknown msg type 0x%02x from %04llX (hash=%016llx) — dropped",
type, (unsigned long long)(src >> 16), hash);
DEBUG_WARN(DEBUG_CATEGORY_CHAT_SYNC, "sync: unknown msg type 0x%02x from %04llX (group=%016llx) — dropped",
type, (unsigned long long)(src >> 16), group_id);
break;
}
queue_dgram_free(entry);
@@ -1292,11 +1313,12 @@ static void db_sync_on_conn_up(struct ETCP_CONN* conn, void* arg)
int synced = 0;
char tbl_list[256] = "";
for (int i = 0; i < db->instance_count; i++) {
struct DB_SYNC_INSTANCE* si = &db->instances[i];
struct DB_SYNC_INSTANCE* si = db->instances[i];
if (!si->enabled) continue;
struct SI_PEER* p = si_peer_add(si, pid);
if (!p) continue;
if (p->sync_state != 0) continue;
if (si->sync_gated) continue;
if (!conn->initialized || !conn->links_up) continue;
{ uint8_t ek[32]; if (db_get_ed25519_pubkey(db, pid, ek) != 0) continue; }
p->sync_state = 1;
@@ -1321,14 +1343,14 @@ static void db_sync_on_conn_down(struct ETCP_CONN* conn, void* arg)
DEBUG_TRACE(DEBUG_CATEGORY_CHAT_SYNC, "peer=%016llx", (unsigned long long)pid);
for (int i = 0; i < db->instance_count; i++) {
struct SI_PEER* p = si_peer_find(&db->instances[i], pid);
struct SI_PEER* p = si_peer_find(db->instances[i], pid);
if (p) p->sync_state = 0;
}
int any = 0;
for (int i = 0; i < db->instance_count; i++) {
for (int j = 0; j < db->instances[i].peer_count; j++) {
if (db->instances[i].peers[j].sync_state >= 1) { any = 1; goto cd_done; }
for (int j = 0; j < db->instances[i]->peer_count; j++) {
if (db->instances[i]->peers[j].sync_state >= 1) { any = 1; goto cd_done; }
}
}
cd_done:
@@ -1447,6 +1469,35 @@ void db_sync_peer_set_state(struct DB_SYNC_INSTANCE* si, uint64_t node_id, uint8
if (p) p->sync_state = state;
}
// Гейт авто-синхронизации: пока gated — инстанс не запускает sync автоматически
// (ждёт member_sync/pubkeys). При снятии гейта запускает sync к пирам в sync_state==0.
void db_sync_instance_set_gated(struct DB_SYNC_INSTANCE* si, int gated)
{
if (!si || !si->enabled) return;
gated = gated ? 1 : 0;
if (si->sync_gated == (uint8_t)gated) return;
si->sync_gated = (uint8_t)gated;
if (!gated) {
struct DB_SYNC* db = si->db_sync;
int launched = 0;
for (int i = 0; i < si->peer_count; i++) {
struct SI_PEER* p = &si->peers[i];
if (p->sync_state != 0) continue;
uint8_t ek[32];
if (db_get_ed25519_pubkey(db, p->node_id, ek) != 0) continue;
p->sync_state = 1; p->sync_start_tb = get_time_tb();
db_sync_initiate_sync(si, p->node_id);
launched++;
}
if (launched == 0) db_sync_resume_peer_check(db);
DEBUG_INFO(DEBUG_CATEGORY_CHAT_SYNC, "sync [%s] → un-gated: launched sync for %d peers (mc=%u)",
SI_SHRT(si), launched, db_count(si));
} else {
DEBUG_INFO(DEBUG_CATEGORY_CHAT_SYNC, "sync [%s] → gated: auto-sync suspended (mc=%u)",
SI_SHRT(si), db_count(si));
}
}
// Принудительно перезапускает синхронизацию с указанным пиром (для тестов)
void db_sync_reinitiate(struct DB_SYNC_INSTANCE* si, uint64_t node_id)
{
@@ -1476,7 +1527,7 @@ static int db_sync_has_pending_work(struct DB_SYNC* db)
{
if (!db || !db->enabled) return 0;
for (int i = 0; i < db->instance_count; i++) {
struct DB_SYNC_INSTANCE* si = &db->instances[i];
struct DB_SYNC_INSTANCE* si = db->instances[i];
if (!si->enabled) continue;
for (int j = 0; j < si->peer_count; j++)
if (si->peers[j].sync_state != 2) return 1;
@@ -1515,8 +1566,9 @@ static void db_sync_peer_check_cb(void* arg)
char launched_list[256] = "";
if (g) {
for (int i = 0; i < db->instance_count; i++) {
struct DB_SYNC_INSTANCE* si = &db->instances[i];
struct DB_SYNC_INSTANCE* si = db->instances[i];
if (!si->enabled) continue;
if (si->sync_gated) continue;
int peers_found = 0;
struct ll_entry* e = g->senders_list->head;
@@ -1559,7 +1611,7 @@ static void db_sync_peer_check_cb(void* arg)
uint64_t now = get_time_tb();
uint64_t to_tb = DB_SYNC_SYNC_TIMEOUT * 10000u;
for (int i = 0; i < db->instance_count; i++) {
struct DB_SYNC_INSTANCE* si = &db->instances[i];
struct DB_SYNC_INSTANCE* si = db->instances[i];
if (!si->enabled) continue;
for (int j = 0; j < si->peer_count; j++) {
struct SI_PEER* p = &si->peers[j];
@@ -1690,10 +1742,11 @@ void db_sync_destroy(struct UTUN_INSTANCE* inst)
#endif
for (int i = 0; i < db->instance_count; i++) {
struct DB_SYNC_INSTANCE* si = &db->instances[i];
struct DB_SYNC_INSTANCE* si = db->instances[i];
if (si->ttl_timer) { uasync_cancel_timeout(inst->ua, si->ttl_timer); si->ttl_timer = NULL; }
if (si->recalc.timer) { uasync_cancel_timeout(inst->ua, si->recalc.timer); si->recalc.timer = NULL; }
if (si->peers) u_free(si->peers);
u_free(si);
}
etcp_remove_conn_status_cbk(inst, db_sync_on_conn_status, NULL);
@@ -1707,19 +1760,19 @@ void db_sync_destroy(struct UTUN_INSTANCE* inst)
}
// Создаёт SQLite-таблицу для синхронизации, регистрирует пиров, запускает первичную синхронизацию и TTL-таймер
struct DB_SYNC_INSTANCE* db_sync_instance_add(struct UTUN_INSTANCE* inst, const char* table_name, uint64_t hash, int auto_sync)
struct DB_SYNC_INSTANCE* db_sync_instance_add(struct UTUN_INSTANCE* inst, const char* table_name, uint64_t group_id, int auto_sync)
{
if (!inst || !inst->db_sync || !table_name || !table_name[0]) { DEBUG_ERROR(DEBUG_CATEGORY_CHAT_SYNC, "instance_add invalid args"); return NULL; }
DEBUG_TRACE(DEBUG_CATEGORY_CHAT_SYNC, "table=%s hash=%016llx", table_name, (unsigned long long)hash);
DEBUG_TRACE(DEBUG_CATEGORY_CHAT_SYNC, "table=%s group=%016llx", table_name, (unsigned long long)group_id);
struct DB_SYNC* db = inst->db_sync;
if (!db->enabled || !db->db) return NULL;
struct DB_SYNC_INSTANCE* si = db_instance_find(db, hash);
if (si) { DEBUG_INFO(DEBUG_CATEGORY_CHAT_SYNC, "instance already exists hash=%016llx", (unsigned long long)hash); return si; }
struct DB_SYNC_INSTANCE* si = db_instance_find(db, group_id);
if (si) { DEBUG_INFO(DEBUG_CATEGORY_CHAT_SYNC, "instance already exists group=%016llx", (unsigned long long)group_id); return si; }
si = db_instance_alloc(db);
if (!si) return NULL;
si->hash = hash;
si->group_id = group_id;
snprintf(si->table_name, sizeof(si->table_name), "%s", table_name);
// Create table — PK is (timestamp, node_id), author_signature NOT NULL
@@ -1762,7 +1815,7 @@ struct DB_SYNC_INSTANCE* db_sync_instance_add(struct UTUN_INSTANCE* inst, const
db_verify_chain(si);
DEBUG_DEBUG(DEBUG_CATEGORY_CHAT_SYNC, "instance_add: tbl=%s mc=%u next_id=%llu hash=%016llx", SI_TBL(si), db_count(si), (unsigned long long)si->next_id, (unsigned long long)si->hash);
DEBUG_DEBUG(DEBUG_CATEGORY_CHAT_SYNC, "instance_add: tbl=%s mc=%u next_id=%llu group=%016llx", SI_TBL(si), db_count(si), (unsigned long long)si->next_id, (unsigned long long)si->group_id);
si->ttl_timer = uasync_set_timeout(inst->ua, DB_SYNC_TTL_INTERVAL * 10000u, si, db_sync_instance_ttl_cb, "db_sync_ttl");
@@ -1776,15 +1829,15 @@ struct DB_SYNC_INSTANCE* db_sync_instance_add(struct UTUN_INSTANCE* inst, const
peers_found++;
struct SI_PEER* p = si_peer_add(si, pid);
if (!p) { DEBUG_ERROR(DEBUG_CATEGORY_CHAT_SYNC, "si_peer_add failed for %N in %s", (unsigned long long)pid, SI_TBL(si)); continue; }
if (p->sync_state == 0 && auto_sync) { p->sync_state = 1; db_sync_initiate_sync(si, pid); peers_synced++; }
if (p->sync_state == 0 && auto_sync && !si->sync_gated) { p->sync_state = 1; db_sync_initiate_sync(si, pid); peers_synced++; }
}
entry = entry->next;
}
}
DEBUG_INFO(DEBUG_CATEGORY_CHAT_SYNC,
"instance added table=%s tbl=%s hash=%016llx next_id=%llu peers_found=%d peers_synced=%d mc=%u",
table_name, si->table_name, (unsigned long long)hash,
"instance added table=%s tbl=%s group=%016llx next_id=%llu peers_found=%d peers_synced=%d mc=%u",
table_name, si->table_name, (unsigned long long)group_id,
(unsigned long long)si->next_id, peers_found, peers_synced, db_count(si));
return si;
}
@@ -1793,7 +1846,7 @@ struct DB_SYNC_INSTANCE* db_sync_instance_add(struct UTUN_INSTANCE* inst, const
void db_sync_instance_remove(struct DB_SYNC_INSTANCE* si)
{
if (!si) return;
DEBUG_TRACE(DEBUG_CATEGORY_CHAT_SYNC, "tbl=%s hash=%016llx", si->table_name, (unsigned long long)si->hash);
DEBUG_TRACE(DEBUG_CATEGORY_CHAT_SYNC, "tbl=%s group=%016llx", si->table_name, (unsigned long long)si->group_id);
struct DB_SYNC* db = si->db_sync;
struct UTUN_INSTANCE* inst = db->inst;
@@ -1802,13 +1855,17 @@ void db_sync_instance_remove(struct DB_SYNC_INSTANCE* si)
if (si->recalc.timer) { uasync_cancel_timeout(inst->ua, si->recalc.timer); si->recalc.timer = NULL; }
if (si->peers) { u_free(si->peers); si->peers = NULL; si->peer_count = si->peer_capacity = 0; }
int idx = (int)(si - db->instances);
if (idx >= 0 && idx < db->instance_count) {
int idx = -1;
for (int i = 0; i < db->instance_count; i++) {
if (db->instances[i] == si) { idx = i; break; }
}
if (idx >= 0) {
memmove(&db->instances[idx], &db->instances[idx + 1],
(db->instance_count - idx - 1) * sizeof(*db->instances));
db->instance_count--;
}
DEBUG_INFO(DEBUG_CATEGORY_CHAT_SYNC, "instance removed tbl=%s hash=%016llx", si->table_name, (unsigned long long)si->hash);
DEBUG_INFO(DEBUG_CATEGORY_CHAT_SYNC, "instance removed tbl=%s group=%016llx", si->table_name, (unsigned long long)si->group_id);
u_free(si);
}
// Вставляет подписанную запись: проверяет подпись, вставляет в БД, рассылает PUSH всем синхронизированным пирам
+8 -3
View File
@@ -24,8 +24,7 @@
//
// Синхронизация:
// - При поднятии ETCP-соединения с пиром для каждого инстанса запускается полная синхронизация
// - Каждое сообщение содержит instance_hash (первые 64 бита SHA256(name||id_be)),
// что позволяет маршрутизировать сообщения к нужному инстансу на приёмной стороне
// - Каждое сообщение содержит group_id (uint64, binary), что позволяет маршрутизировать сообщения к нужному инстансу на приёмной стороне
// - Если пир не имеет инстанса с таким хешем — возвращает DB_MSG_ERROR(0x01)
// - Если пир добавляет инстанс позже — сам инициирует sync в нашу сторону
// - Протокол: сравнение chain_hash для поиска расхождений (7 типов сообщений)
@@ -97,8 +96,10 @@ int db_sync_init(struct UTUN_INSTANCE* inst);
void db_sync_destroy(struct UTUN_INSTANCE* inst);
// Instance management
struct DB_SYNC_INSTANCE* db_sync_instance_add(struct UTUN_INSTANCE* inst, const char* table_name, uint64_t hash, int auto_sync);
struct DB_SYNC_INSTANCE* db_sync_instance_add(struct UTUN_INSTANCE* inst, const char* table_name, uint64_t group_id, int auto_sync);
void db_sync_instance_remove(struct DB_SYNC_INSTANCE* si);
// Найти живой инстанс по group_id (поиск в текущем массиве db->instances — указатель всегда актуален).
struct DB_SYNC_INSTANCE* db_sync_instance_find(struct UTUN_INSTANCE* inst, uint64_t group_id);
// Data operations (per-instance)
// db_sync_insert_signed: sig MUST be non-NULL, 64 bytes.
@@ -134,6 +135,10 @@ int db_sync_chain_verify(struct DB_SYNC_INSTANCE* si);
// Peer state control (for testing — disable PUSH to isolate sync protocol)
void db_sync_peer_set_state(struct DB_SYNC_INSTANCE* si, uint64_t node_id, uint8_t state);
// Gate auto-sync: while gated the instance does not auto-initiate sync (waits for
// member_sync/pubkeys). Un-gating launches sync to peers in sync_state==0.
void db_sync_instance_set_gated(struct DB_SYNC_INSTANCE* si, int gated);
// Force re-initiate sync to a specific peer (for testing)
void db_sync_reinitiate(struct DB_SYNC_INSTANCE* si, uint64_t node_id);
+466
View File
@@ -0,0 +1,466 @@
/*
* invite_build.c — сборка invite-ссылок utun:// (выбор узла + адреса)
*
* Общий модуль для всех GUI (desktop chatgui, Android, headless CLI).
* Логика выбора «лучшего узла» перенесена из tools/chatgui-android/jni_bridge.
* Работает в uasync-потоке, использует per-instance контекст.
*/
#include "invite_build.h"
#include "invite_link.h"
#include "chat_core_priv.h"
#include "chat_event.h"
#include "chat_join.h"
#include "../routing_layer/topo_group.h"
#include "../routing_layer/topo_node.h"
#include "../utun_instance.h"
#include "../transport_layer/etcp_connections.h"
#include "../../lib/mem.h"
#include "../../lib/platform_compat.h"
#include <string.h>
#include <stdio.h>
#include <stdlib.h>
#include <openssl/rand.h>
/* ── живость узла в канале (BGP/прямое/через посредника) ── */
static struct TOPO_GROUP* resolve_chat_group(struct UTUN_INSTANCE* inst, uint64_t channel_id) {
if (!inst || !inst->topo_groups) return NULL;
struct TOPO_GROUP* grp = topo_groups_find(inst->topo_groups, channel_id);
if (!grp || grp->group_type != TOPO_GROUP_TYPE_CHAT) return NULL;
return grp;
}
static int node_is_alive(struct UTUN_INSTANCE* inst, struct TOPO_GROUP* grp, uint64_t node_id) {
struct chat_core_ctx* cc = CC(inst);
if (cc && node_id == cc->my_node_id) return 1; /* self всегда доступен */
if (!grp) return 0;
struct TOPO_GROUP_NODE* nq = topo_node_find_by_id(grp, node_id);
if (!nq) return 0;
return (nq->conn_presence & (NCONN_DIRECT | NCONN_INDIRECT | NCONN_BGP)) != 0;
}
/* есть ли у узла публичные адреса v4 и v6 (addr_type=DIRECT) */
static int node_has_both_public(struct UTUN_INSTANCE* inst, uint64_t node_id) {
struct chat_core_ctx* cc = CC(inst);
int has_v4 = 0, has_v6 = 0;
sqlite3_stmt* st = NULL;
char sql[256];
snprintf(sql, sizeof(sql),
"SELECT family FROM node_addresses WHERE node_id=? AND addr_type=%d", ADDR_TYPE_DIRECT);
if (sqlite3_prepare_v2(cc->db, sql, -1, &st, NULL) != SQLITE_OK) return 0;
sqlite3_bind_int64(st, 1, (sqlite3_int64)node_id);
while (sqlite3_step(st) == SQLITE_ROW) {
int fam = sqlite3_column_int(st, 0);
if (fam == 4) has_v4 = 1;
else if (fam == 6) has_v6 = 1;
}
sqlite3_finalize(st);
return (has_v4 && has_v6) ? 1 : 0;
}
/* ── Фаза 1-2: лучший достижимый узел в канале ── */
static uint64_t pick_best_node(struct UTUN_INSTANCE* inst, uint64_t channel_id, int* out_has_both_public, int* out_priority) {
struct chat_core_ctx* cc = CC(inst);
uint64_t best_nid = 0;
int best_priority = 4; /* 1=regular, 2=self, 3=super, 4=invalid */
int best_has_both = 0;
struct TOPO_GROUP* grp = resolve_chat_group(inst, channel_id);
if (!grp) {
*out_has_both_public = 0;
*out_priority = best_priority;
return 0;
}
char ch_str[32];
snprintf(ch_str, sizeof(ch_str), "%llu", (unsigned long long)channel_id);
char peers_tbl[80];
peers_table_name(ch_str, peers_tbl, sizeof(peers_tbl));
char sql[512];
snprintf(sql, sizeof(sql), "SELECT node_id, node_type FROM \"%s\" ORDER BY node_id", peers_tbl);
sqlite3_stmt* pst = NULL;
if (sqlite3_prepare_v2(cc->db, sql, -1, &pst, NULL) != SQLITE_OK) {
DEBUG_ERROR(DEBUG_CATEGORY_CHAT_SYNC, "invite_build: pick_best — query failed ch=%llu err=%s",
(unsigned long long)channel_id, sqlite3_errmsg(cc->db));
*out_has_both_public = 0;
*out_priority = best_priority;
return 0;
}
while (sqlite3_step(pst) == SQLITE_ROW) {
uint64_t nid = (uint64_t)sqlite3_column_int64(pst, 0);
int ntype = sqlite3_column_int(pst, 1);
if (!node_is_alive(inst, grp, nid)) continue;
int has_both = node_has_both_public(inst, nid);
int priority;
if (nid == cc->my_node_id) priority = 2; /* self */
else if (ntype == 4) priority = 3; /* supernode */
else priority = 1; /* regular */
if (priority <= best_priority) {
best_nid = nid;
best_priority = priority;
best_has_both = has_both;
}
}
sqlite3_finalize(pst);
*out_has_both_public = best_has_both;
*out_priority = best_priority;
return best_nid;
}
/* авто-узел: pick_best_node + фолбэк на self */
static uint64_t resolve_auto_node(struct UTUN_INSTANCE* inst, uint64_t channel_id, int* out_has_both, int* out_priority) {
struct chat_core_ctx* cc = CC(inst);
int has_both = 0, prio = 4;
uint64_t best = pick_best_node(inst, channel_id, &has_both, &prio);
if (best == 0) best = cc->my_node_id;
if (out_has_both) *out_has_both = has_both;
if (out_priority) *out_priority = prio;
return best;
}
/* ── Фаза 3: pubkey узла ── */
static int get_node_pubkey(struct UTUN_INSTANCE* inst, uint64_t node_id, int is_self, uint8_t pubkey_out[INVITE_PUBKEY_SIZE]) {
struct chat_core_ctx* cc = CC(inst);
if (is_self) {
if (!inst) return -1;
memcpy(pubkey_out, inst->my_keys.public_key, INVITE_PUBKEY_SIZE);
return 0;
}
sqlite3_stmt* st = NULL;
if (sqlite3_prepare_v2(cc->db, "SELECT x25519_pubkey FROM nodes WHERE node_id=?",
-1, &st, NULL) != SQLITE_OK) return -1;
sqlite3_bind_int64(st, 1, (sqlite3_int64)node_id);
int ok = 0;
if (sqlite3_step(st) == SQLITE_ROW) {
const void* pk = sqlite3_column_blob(st, 0);
int pklen = sqlite3_column_bytes(st, 0);
if (pk && pklen >= INVITE_PUBKEY_SIZE) { memcpy(pubkey_out, pk, INVITE_PUBKEY_SIZE); ok = 1; }
}
sqlite3_finalize(st);
return ok ? 0 : -1;
}
/* ── Фаза 4a: свои адреса из сокетов ── */
static int collect_self_addrs(struct UTUN_INSTANCE* inst, struct InviteAddr* out, int max_cnt) {
int cnt = 0;
if (!inst) return 0;
struct ETCP_SOCKET* s;
/* UDP */
for (s = inst->etcp_sockets; s && cnt < max_cnt; s = s->next) {
if (s->is_tcp) continue;
struct sockaddr_storage* sa = s->interface_addr.ss_family ? &s->interface_addr : &s->local_addr;
if (sa->ss_family == AF_INET) {
struct sockaddr_in* sin = (struct sockaddr_in*)sa;
out[cnt].family = 4; out[cnt].socketId = s->sock_id; out[cnt].proto = INVITE_PROTO_UDP;
memcpy(out[cnt].address, &sin->sin_addr, 4);
out[cnt].port = ntohs(sin->sin_port); cnt++;
} else if (sa->ss_family == AF_INET6) {
struct sockaddr_in6* sin6 = (struct sockaddr_in6*)sa;
out[cnt].family = 6; out[cnt].socketId = s->sock_id; out[cnt].proto = INVITE_PROTO_UDP;
memcpy(out[cnt].address, &sin6->sin6_addr, 16);
out[cnt].port = ntohs(sin6->sin6_port); cnt++;
}
}
/* TCP */
for (s = inst->etcp_sockets; s && cnt < max_cnt; s = s->next) {
if (!s->is_tcp) continue;
struct sockaddr_storage* sa = s->interface_addr.ss_family ? &s->interface_addr : &s->local_addr;
uint8_t proto = INVITE_PROTO_TCP | (s->reality_enabled ? INVITE_PROTO_REALITY : 0);
if (sa->ss_family == AF_INET) {
struct sockaddr_in* sin = (struct sockaddr_in*)sa;
out[cnt].family = 4; out[cnt].socketId = s->sock_id; out[cnt].proto = proto;
memcpy(out[cnt].address, &sin->sin_addr, 4);
out[cnt].port = ntohs(sin->sin_port); cnt++;
} else if (sa->ss_family == AF_INET6) {
struct sockaddr_in6* sin6 = (struct sockaddr_in6*)sa;
out[cnt].family = 6; out[cnt].socketId = s->sock_id; out[cnt].proto = proto;
memcpy(out[cnt].address, &sin6->sin6_addr, 16);
out[cnt].port = ntohs(sin6->sin6_port); cnt++;
}
}
return cnt;
}
/* reality-параметры self-узла из конфига (node-global, как в topo_node_update_my_addresses). */
static void collect_self_reality(struct UTUN_INSTANCE* inst, struct InviteReality* out) {
memset(out, 0, sizeof(*out));
if (!inst || !inst->config) return;
const struct reality_config* rc = &inst->config->global.reality;
if (!rc->enabled || !rc->has_private_key) return;
int any = 0;
for (struct ETCP_SOCKET* s = inst->etcp_sockets; s; s = s->next)
if (s->is_tcp && s->reality_enabled) { any = 1; break; }
if (!any) return;
uint8_t pub[32];
if (reality_pubkey_from_priv(rc->private_key, pub) != REALITY_OK) return;
out->has = 1;
if (rc->short_id_count > 0) memcpy(out->short_id, rc->short_ids[0], 8);
else memcpy(out->short_id, rc->short_id, 8);
memcpy(out->server_pubkey, pub, 32);
memcpy(out->version, rc->version, 3);
strncpy(out->server_name, rc->server_name, INVITE_SNAME_MAX - 1);
out->server_name[INVITE_SNAME_MAX - 1] = '\0';
}
/* ── Фаза 4b: адреса узла из БД ── */
static int collect_node_addrs_from_db(struct UTUN_INSTANCE* inst, uint64_t node_id, struct InviteAddr* out, int max_cnt, int dir_only,
struct InviteReality* reality) {
struct chat_core_ctx* cc = CC(inst);
char sql[256];
if (dir_only)
snprintf(sql, sizeof(sql),
"SELECT family,socket_id,protocol,address,port,options FROM node_addresses"
" WHERE node_id=? AND addr_type=%d", ADDR_TYPE_DIRECT);
else
snprintf(sql, sizeof(sql),
"SELECT family,socket_id,protocol,address,port,options FROM node_addresses WHERE node_id=?");
sqlite3_stmt* st = NULL;
if (sqlite3_prepare_v2(cc->db, sql, -1, &st, NULL) != SQLITE_OK) return 0;
sqlite3_bind_int64(st, 1, (sqlite3_int64)node_id);
int cnt = 0;
while (sqlite3_step(st) == SQLITE_ROW && cnt < max_cnt) {
int family = sqlite3_column_int(st, 0);
out[cnt].family = family;
out[cnt].socketId = (uint8_t)sqlite3_column_int(st, 1);
out[cnt].proto = (uint8_t)sqlite3_column_int(st, 2);
const uint8_t* addr = sqlite3_column_blob(st, 3);
int addr_len = sqlite3_column_bytes(st, 3);
int ip_len = (family == 6) ? 16 : 4;
memset(out[cnt].address, 0, sizeof(out[cnt].address));
if (addr && addr_len >= ip_len) memcpy(out[cnt].address, addr, (size_t)ip_len);
out[cnt].port = (uint16_t)sqlite3_column_int(st, 4);
if (reality && !reality->has && (out[cnt].proto & INVITE_PROTO_REALITY)
&& sqlite3_column_type(st, 5) == SQLITE_BLOB
&& sqlite3_column_bytes(st, 5) == TOPO_ADDR_REALITY_OPTS_SIZE) {
const struct TOPO_ADDR_REALITY_OPTS* o = (const struct TOPO_ADDR_REALITY_OPTS*)sqlite3_column_blob(st, 5);
reality->has = 1;
memcpy(reality->short_id, o->short_id, 8);
memcpy(reality->server_pubkey, o->server_pubkey, 32);
memcpy(reality->version, o->version, 3);
memcpy(reality->server_name, o->server_name, INVITE_SNAME_MAX);
reality->server_name[INVITE_SNAME_MAX - 1] = '\0';
}
cnt++;
}
sqlite3_finalize(st);
return cnt;
}
/* ── публичный API ── */
int chat_invite_build_link(struct UTUN_INSTANCE* inst, uint64_t channel_id, uint64_t target_node_id,
const char* password, char* out, size_t out_size) {
struct chat_core_ctx* cc = CC(inst);
if (!out || out_size == 0) return -1;
out[0] = '\0';
if (!cc || !cc->initialized || !inst || !cc->db) {
DEBUG_ERROR(DEBUG_CATEGORY_CHAT_SYNC, "invite_build: not initialized (inst=%p db=%p)",
(void*)inst, (void*)(cc ? cc->db : NULL));
return -1;
}
if (channel_id == 0) {
DEBUG_ERROR(DEBUG_CATEGORY_CHAT_SYNC, "invite_build: invalid channel_id");
return -1;
}
/* ── выбор узла ── */
uint64_t target_nid = target_node_id;
int use_self = 0, has_both_public = 0;
if (target_nid == 0) {
int prio = 4;
target_nid = resolve_auto_node(inst, channel_id, &has_both_public, &prio);
use_self = (target_nid == cc->my_node_id) ? 1 : 0;
DEBUG_INFO(DEBUG_CATEGORY_CHAT_SYNC, "invite_build: auto ch=%llu best=0x%016llx prio=%d self=%d both=%d",
(unsigned long long)channel_id, (unsigned long long)target_nid,
prio, use_self, has_both_public);
} else {
if (target_nid == cc->my_node_id) {
use_self = 1;
} else {
struct TOPO_GROUP* grp = resolve_chat_group(inst, channel_id);
if (!node_is_alive(inst, grp, target_nid)) {
DEBUG_ERROR(DEBUG_CATEGORY_CHAT_SYNC,
"invite_build: node 0x%016llx not reachable in ch=%llu",
(unsigned long long)target_nid, (unsigned long long)channel_id);
return -1;
}
has_both_public = node_has_both_public(inst, target_nid);
}
DEBUG_INFO(DEBUG_CATEGORY_CHAT_SYNC, "invite_build: manual ch=%llu target=0x%016llx self=%d both=%d",
(unsigned long long)channel_id, (unsigned long long)target_nid, use_self, has_both_public);
}
/* ── pubkey ── */
struct InviteData data;
memset(&data, 0, sizeof(data));
data.channelId = channel_id;
data.nodeId = target_nid;
/* join-ключ: генерируется инвайтером, кладётся в ссылку и регистрируется на connection-узле */
if (RAND_bytes((uint8_t*)&data.join_key, sizeof(data.join_key)) != 1 || data.join_key == 0) {
DEBUG_ERROR(DEBUG_CATEGORY_CHAT_SYNC, "invite_build: join_key generation failed");
return -1;
}
if (get_node_pubkey(inst, target_nid, use_self, data.pubkey) != 0) {
DEBUG_ERROR(DEBUG_CATEGORY_CHAT_SYNC, "invite_build: no pubkey for node=0x%016llx",
(unsigned long long)target_nid);
return -1;
}
/* ── адреса ── */
if (use_self) {
data.addrCount = (uint8_t)collect_self_addrs(inst, data.addrs, INVITE_ADDR_MAX);
collect_self_reality(inst, &data.reality);
} else {
int dir_only = has_both_public ? 1 : 0;
data.addrCount = (uint8_t)collect_node_addrs_from_db(inst, target_nid, data.addrs, INVITE_ADDR_MAX, dir_only, &data.reality);
}
if (data.addrCount == 0) {
DEBUG_ERROR(DEBUG_CATEGORY_CHAT_SYNC, "invite_build: no addrs for node=0x%016llx",
(unsigned long long)target_nid);
return -1;
}
DEBUG_INFO(DEBUG_CATEGORY_CHAT_SYNC, "invite_build: node=0x%016llx addrs=%d dir_only=%d",
(unsigned long long)target_nid, data.addrCount, (!use_self && has_both_public) ? 1 : 0);
/* ── encode ── */
if (invite_link_encode(&data, password, out, out_size) < 0) {
DEBUG_ERROR(DEBUG_CATEGORY_CHAT_SYNC, "invite_build: encode failed ch=%llu",
(unsigned long long)channel_id);
return -1;
}
/* ── регистрация join-ключа на connection-узле (или локально, если target==self) ── */
chat_join_register_key(inst, channel_id, target_nid, data.join_key);
return 0;
}
int chat_invite_best_node(struct UTUN_INSTANCE* inst, uint64_t channel_id, uint64_t* out_node_id) {
struct chat_core_ctx* cc = CC(inst);
if (!out_node_id) return -1;
if (!cc || !cc->initialized || !inst || !cc->db) {
DEBUG_ERROR(DEBUG_CATEGORY_CHAT_SYNC, "invite_build: best_node — not initialized");
return -1;
}
*out_node_id = resolve_auto_node(inst, channel_id, NULL, NULL);
return 0;
}
int chat_invite_candidate_nodes(struct UTUN_INSTANCE* inst, uint64_t channel_id, uint64_t* out_ids, int max_ids) {
struct chat_core_ctx* cc = CC(inst);
if (!out_ids || max_ids <= 0) return -1;
if (!cc || !cc->initialized || !inst || !cc->db) {
DEBUG_ERROR(DEBUG_CATEGORY_CHAT_SYNC, "invite_build: candidates — not initialized");
return -1;
}
struct TOPO_GROUP* grp = resolve_chat_group(inst, channel_id);
if (!grp) {
DEBUG_ERROR(DEBUG_CATEGORY_CHAT_SYNC, "invite_build: candidates — group not found ch=%llu",
(unsigned long long)channel_id);
return -1;
}
char ch_str[32];
snprintf(ch_str, sizeof(ch_str), "%llu", (unsigned long long)channel_id);
char peers_tbl[80];
peers_table_name(ch_str, peers_tbl, sizeof(peers_tbl));
char sql[512];
snprintf(sql, sizeof(sql), "SELECT node_id FROM \"%s\" ORDER BY node_id", peers_tbl);
sqlite3_stmt* st = NULL;
if (sqlite3_prepare_v2(cc->db, sql, -1, &st, NULL) != SQLITE_OK) {
DEBUG_ERROR(DEBUG_CATEGORY_CHAT_SYNC, "invite_build: candidates — query failed ch=%llu err=%s",
(unsigned long long)channel_id, sqlite3_errmsg(cc->db));
return -1;
}
int cnt = 0;
while (sqlite3_step(st) == SQLITE_ROW && cnt < max_ids) {
uint64_t nid = (uint64_t)sqlite3_column_int64(st, 0);
if (node_is_alive(inst, grp, nid)) out_ids[cnt++] = nid;
}
sqlite3_finalize(st);
return cnt;
}
/* ── трамплины для gui_bridge ── */
void chat_invite_candidates_trampoline(void* arg) {
struct chat_invite_cand_req* req = (struct chat_invite_cand_req*)arg;
if (!req) return;
char ch_id[64];
strncpy(ch_id, req->ch_id, sizeof(ch_id) - 1);
ch_id[sizeof(ch_id) - 1] = '\0';
u_free(arg);
if (!CC(req->inst) || !CC(req->inst)->initialized) { DEBUG_ERROR(DEBUG_CATEGORY_CHAT_SYNC, "invite_build: candidates — not initialized"); return; }
uint64_t channel_id = strtoull(ch_id, NULL, 10);
uint64_t auto_id = 0;
chat_invite_best_node(req->inst, channel_id, &auto_id);
uint64_t ids[INVITE_ADDR_MAX];
int cnt = chat_invite_candidate_nodes(req->inst, channel_id, ids, INVITE_ADDR_MAX);
if (cnt < 0) cnt = 0;
size_t ch_len = strlen(ch_id);
size_t evt_sz = 1 + ch_len + 8 + 2 + (size_t)cnt * 8;
uint8_t* evt = u_malloc(evt_sz);
if (!evt) return;
evt[0] = (uint8_t)ch_len;
memcpy(evt + 1, ch_id, ch_len);
memcpy(evt + 1 + ch_len, &auto_id, 8);
uint16_t u16 = (uint16_t)cnt;
memcpy(evt + 1 + ch_len + 8, &u16, 2);
for (int i = 0; i < cnt; i++) memcpy(evt + 1 + ch_len + 8 + 2 + i * 8, &ids[i], 8);
chat_event_post(req->inst, CHAT_EVT_INVITE_CANDIDATES, evt, (int)evt_sz);
u_free(evt);
}
void chat_invite_build_link_trampoline(void* arg) {
struct chat_invite_build_req* req = (struct chat_invite_build_req*)arg;
if (!req) return;
uint64_t channel_id = strtoull(req->ch_id, NULL, 10);
uint64_t target = req->target_node_id;
char password[INVITE_PASS_MAX];
password[0] = '\0';
const char* pass = NULL;
if (req->password[0]) {
strncpy(password, req->password, sizeof(password) - 1);
password[sizeof(password) - 1] = '\0';
pass = password;
}
char link[1024];
int ok = chat_invite_build_link(req->inst, channel_id, target, pass, link, sizeof(link));
size_t ch_len = strlen(req->ch_id);
size_t link_len = (ok == 0) ? strlen(link) : 0;
size_t evt_sz = 1 + ch_len + 2 + link_len;
uint8_t* evt = u_malloc(evt_sz);
if (!evt) { u_free(req); return; }
evt[0] = (uint8_t)ch_len;
memcpy(evt + 1, req->ch_id, ch_len);
uint16_t u16 = (uint16_t)link_len;
memcpy(evt + 1 + ch_len, &u16, 2);
if (link_len) memcpy(evt + 1 + ch_len + 2, link, link_len);
chat_event_post(req->inst, CHAT_EVT_INVITE_LINK_READY, evt, (int)evt_sz);
u_free(evt);
u_free(req);
}
+53
View File
@@ -0,0 +1,53 @@
/*
* invite_build.h — сборка invite-ссылок utun:// (выбор узла + адреса)
*
* Общий модуль для всех GUI (desktop chatgui, Android, headless CLI).
* Логика выбора «лучшего узла» перенесена из tools/chatgui-android/jni_bridge.
* Работает в uasync-потоке, использует общий контекст g_cc.
*/
#ifndef INVITE_BUILD_H
#define INVITE_BUILD_H
#include <stdint.h>
#include <stddef.h>
#ifdef __cplusplus
extern "C" {
#endif
struct UTUN_INSTANCE;
/* Собрать invite-ссылку для канала.
* target_node_id = 0 → auto (лучший достижимый мембер, иначе self).
* target_node_id != 0 → конкретный узел (должен быть достижимым мембером канала).
* password = NULL → без пароля.
* Возвращает 0 и заполняет out, либо -1 (детали в логе). */
int chat_invite_build_link(struct UTUN_INSTANCE* inst, uint64_t channel_id, uint64_t target_node_id,
const char* password, char* out, size_t out_size);
/* Список достижимых узлов-кандидатов канала (включая self) — тот же фильтр, что в auto.
* Возвращает количество (0..max_ids), либо -1. */
int chat_invite_candidate_nodes(struct UTUN_INSTANCE* inst, uint64_t channel_id, uint64_t* out_ids, int max_ids);
/* Авто-узел: лучший достижимый мембер, иначе self. Возвращает 0 и заполняет out_node_id, либо -1. */
int chat_invite_best_node(struct UTUN_INSTANCE* inst, uint64_t channel_id, uint64_t* out_node_id);
/* Трамплин для gui_bridge (arg = u_malloc(struct chat_invite_build_req)).
* Результат: CHAT_EVT_INVITE_LINK_READY. */
struct chat_invite_build_req {
struct UTUN_INSTANCE* inst;
char ch_id[64];
uint64_t target_node_id; /* 0 = auto */
char password[128];
};
void chat_invite_build_link_trampoline(void* arg);
/* Трамплин (arg = u_malloc(struct chat_invite_cand_req)). Результат: CHAT_EVT_INVITE_CANDIDATES
* (формат: [ch_id_len:1][ch_id][auto_node_id:8][count:2][node_id:8]*). */
struct chat_invite_cand_req { struct UTUN_INSTANCE* inst; char ch_id[64]; };
void chat_invite_candidates_trampoline(void* arg);
#ifdef __cplusplus
}
#endif
#endif /* INVITE_BUILD_H */
+59 -8
View File
@@ -63,12 +63,13 @@ int invite_link_decode(const char* link, size_t link_len, struct InviteData* out
int off = 0;
uint8_t ver = raw[off++];
if (ver != INVITE_LINK_VERSION && ver != 0x02) {
if (ver != INVITE_LINK_VERSION && ver != 0x02 && ver != 0x01) {
if (error_buf && error_buf_size) snprintf(error_buf, error_buf_size, "unsupported version 0x%02x", ver); return -1;
}
out->password[0] = '\0'; out->password_len = 0;
out->reality.has = 0;
if (ver == 0x02) {
if (ver == 0x02 || ver == INVITE_LINK_VERSION) {
if (off + 1 > raw_len) { if (error_buf && error_buf_size) snprintf(error_buf, error_buf_size, "truncated at pass_len"); return -1; }
uint8_t plen = raw[off++];
if (plen > 0) {
@@ -83,6 +84,28 @@ int invite_link_decode(const char* link, size_t link_len, struct InviteData* out
uint64_t chId = 0;
for (int i = 0; i < 8; i++) chId = (chId << 8) | raw[off++];
out->channelId = chId;
if (off + 8 > raw_len) { if (error_buf && error_buf_size) snprintf(error_buf, error_buf_size, "truncated at join_key"); return -1; }
uint64_t jk = 0;
for (int i = 0; i < 8; i++) jk = (jk << 8) | raw[off++];
out->join_key = jk;
if (ver == INVITE_LINK_VERSION) {
if (off + 1 > raw_len) { if (error_buf && error_buf_size) snprintf(error_buf, error_buf_size, "truncated at reality"); return -1; }
if (raw[off++]) {
if (off + 8 + 32 + 3 + 1 > raw_len) { if (error_buf && error_buf_size) snprintf(error_buf, error_buf_size, "truncated at reality params"); return -1; }
memcpy(out->reality.short_id, raw + off, 8); off += 8;
memcpy(out->reality.server_pubkey, raw + off, 32); off += 32;
memcpy(out->reality.version, raw + off, 3); off += 3;
uint8_t snlen = raw[off++];
if (snlen > INVITE_SNAME_MAX - 1) snlen = INVITE_SNAME_MAX - 1;
if (off + snlen > raw_len) { if (error_buf && error_buf_size) snprintf(error_buf, error_buf_size, "truncated at reality server_name"); return -1; }
memcpy(out->reality.server_name, raw + off, snlen); out->reality.server_name[snlen] = '\0';
off += snlen;
out->reality.has = 1;
}
}
memset(out->pubkey, 0, INVITE_PUBKEY_SIZE);
out->addrCount = 0;
@@ -111,10 +134,25 @@ int invite_link_decode(const char* link, size_t link_len, struct InviteData* out
int invite_serialize_addrs(const struct InviteData* data, uint8_t* buf, size_t buf_size) {
if (!data || !buf) return -1;
size_t needed = 0;
size_t needed = 1;
if (data->reality.has) {
size_t snlen = strlen(data->reality.server_name);
if (snlen > INVITE_SNAME_MAX - 1) snlen = INVITE_SNAME_MAX - 1;
needed += 8 + 32 + 3 + 1 + snlen;
}
for (int i = 0; i < (int)data->addrCount; i++) needed += (data->addrs[i].family == 6) ? 21U : 9U;
if (needed > buf_size) return -1;
size_t pos = 0;
buf[pos++] = data->reality.has ? 1 : 0;
if (data->reality.has) {
memcpy(buf + pos, data->reality.short_id, 8); pos += 8;
memcpy(buf + pos, data->reality.server_pubkey, 32); pos += 32;
memcpy(buf + pos, data->reality.version, 3); pos += 3;
size_t snlen = strlen(data->reality.server_name);
if (snlen > INVITE_SNAME_MAX - 1) snlen = INVITE_SNAME_MAX - 1;
buf[pos++] = (uint8_t)snlen;
memcpy(buf + pos, data->reality.server_name, snlen); pos += snlen;
}
for (int i = 0; i < (int)data->addrCount; i++) {
const struct InviteAddr* a = &data->addrs[i];
int ip_len = (a->family == 6) ? 16 : 4;
@@ -128,15 +166,28 @@ int invite_serialize_addrs(const struct InviteData* data, uint8_t* buf, size_t b
int invite_link_encode(const struct InviteData* data, const char* password, char* out, size_t out_size) {
if (!data || !out || data->pubkey[0] == 0) return -1;
if (data->addrCount == 0) return -1;
int has_pass = (password && password[0]) ? 1 : 0;
uint8_t raw[4096]; size_t pos = 0;
raw[pos++] = has_pass ? 0x02 : INVITE_LINK_VERSION;
if (has_pass) {
size_t plen = strlen(password);
raw[pos++] = INVITE_LINK_VERSION;
{
size_t plen = (password && password[0]) ? strlen(password) : 0;
if (plen > INVITE_PASS_MAX - 1) return -1;
raw[pos++] = (uint8_t)plen; memcpy(raw + pos, password, plen); pos += plen;
raw[pos++] = (uint8_t)plen;
if (plen) { memcpy(raw + pos, password, plen); pos += plen; }
}
for (int i = 7; i >= 0; i--) raw[pos++] = (uint8_t)((data->channelId >> (i * 8)) & 0xFF);
for (int i = 7; i >= 0; i--) raw[pos++] = (uint8_t)((data->join_key >> (i * 8)) & 0xFF);
raw[pos++] = data->reality.has ? 1 : 0;
if (data->reality.has) {
memcpy(raw + pos, data->reality.short_id, 8); pos += 8;
memcpy(raw + pos, data->reality.server_pubkey, 32); pos += 32;
memcpy(raw + pos, data->reality.version, 3); pos += 3;
size_t snlen = strlen(data->reality.server_name);
if (snlen > INVITE_SNAME_MAX - 1) snlen = INVITE_SNAME_MAX - 1;
raw[pos++] = (uint8_t)snlen;
memcpy(raw + pos, data->reality.server_name, snlen); pos += snlen;
}
uint8_t pk[32]; memcpy(pk, data->pubkey, 32);
int ia = 0;
+14 -1
View File
@@ -10,13 +10,15 @@
#include <stdint.h>
#include <stddef.h>
#define INVITE_LINK_VERSION 0x01
#define INVITE_LINK_VERSION 0x03
#define INVITE_PUBKEY_SIZE 32
#define INVITE_ADDR_MAX 32
#define INVITE_PASS_MAX 128
#define INVITE_SNAME_MAX 64
#define INVITE_PROTO_UDP 0x01
#define INVITE_PROTO_TCP 0x02
#define INVITE_PROTO_REALITY 0x04 /* флаг reality-камуфляжа поверх TCP (reality-TCP = 0x06) */
#ifdef __cplusplus
extern "C" {
@@ -30,14 +32,25 @@ struct InviteAddr {
uint16_t port;
};
/* Reality-параметры узла (node-global: одинаковы для всех reality-сокетов). */
struct InviteReality {
uint8_t has; /* 1 = reality присутствует в ссылке */
uint8_t short_id[8];
uint8_t server_pubkey[32];
uint8_t version[3];
char server_name[INVITE_SNAME_MAX];
};
struct InviteData {
uint64_t channelId;
uint64_t join_key; /* 64-битный ключ входа (генерируется инвайтером, хранится на connection-узле 10 мин) */
uint8_t pubkey[INVITE_PUBKEY_SIZE];
uint64_t nodeId;
uint8_t addrCount;
struct InviteAddr addrs[INVITE_ADDR_MAX];
char password[INVITE_PASS_MAX];
uint8_t password_len;
struct InviteReality reality;
};
int invite_link_decode(const char* link, size_t link_len, struct InviteData* out,
+202 -60
View File
@@ -41,6 +41,7 @@ static void _compute_member_hash(uint64_t node_id, const uint8_t* x25519,
const uint8_t* join_sig, uint64_t join_ts,
const uint8_t* update_sig, uint64_t update_ts,
const char* adm_tags, const uint8_t* adm_tags_sig,
uint64_t signed_by, const uint8_t* signature,
uint8_t hash_out[MT_HASH_SIZE]) {
EVP_MD_CTX* ctx = EVP_MD_CTX_new();
EVP_DigestInit_ex(ctx, EVP_sha256(), NULL);
@@ -57,6 +58,9 @@ static void _compute_member_hash(uint64_t node_id, const uint8_t* x25519,
if (atl) EVP_DigestUpdate(ctx, adm_tags, atl);
if (adm_tags_sig) EVP_DigestUpdate(ctx, adm_tags_sig, 64); else { static const uint8_t z64[64]; EVP_DigestUpdate(ctx, z64, 64); }
EVP_DigestUpdate(ctx, &signed_by, 8);
if (signature) EVP_DigestUpdate(ctx, signature, 64); else { static const uint8_t zsig[64]; EVP_DigestUpdate(ctx, zsig, 64); }
EVP_DigestFinal_ex(ctx, hash_out, NULL);
EVP_MD_CTX_free(ctx);
}
@@ -119,7 +123,7 @@ static int _member_update_bucket_hash(void* ctx, const char* ns, uint8_t level,
char sql[512]; snprintf(sql, sizeof(sql),
"SELECT node_id, x25519_pubkey, ed25519_pubkey,"
" join_sig, join_ts, update_sig, update_ts, adm_tags, adm_tags_sig, source"
" join_sig, join_ts, update_sig, update_ts, adm_tags, adm_tags_sig, signed_by, signature, source"
" FROM \"%s\""
" WHERE (node_id & %lld) == %lld ORDER BY node_id ASC",
peers_tbl, (long long)mask, (long long)prefix64);
@@ -138,11 +142,13 @@ static int _member_update_bucket_hash(void* ctx, const char* ns, uint8_t level,
uint64_t uts = (uint64_t)sqlite3_column_int64(stmt, 6);
const char* atags = (const char*)sqlite3_column_text(stmt, 7);
const uint8_t* atsig = (const uint8_t*)sqlite3_column_blob(stmt, 8);
int src = sqlite3_column_int(stmt, 9);
uint64_t sb = (uint64_t)sqlite3_column_int64(stmt, 9);
const uint8_t* sgn = (const uint8_t*)sqlite3_column_blob(stmt, 10);
int src = sqlite3_column_int(stmt, 11);
if (!x25 || !ed || src != 0) continue;
uint8_t mh[MT_HASH_SIZE];
_compute_member_hash(nid, x25, ed, sig, jts, usig, uts, atags, atsig, mh);
_compute_member_hash(nid, x25, ed, sig, jts, usig, uts, atags, atsig, sb, sgn, mh);
EVP_DigestUpdate(sha_ctx, mh, MT_HASH_SIZE);
count++;
}
@@ -164,7 +170,8 @@ static int _member_get_items(void* ctx, const char* ns, uint8_t level,
char sql[512]; snprintf(sql, sizeof(sql),
"SELECT node_id, x25519_pubkey, ed25519_pubkey,"
" join_sig, join_ts, update_sig, update_ts, userinfo, adm_tags, adm_tags_sig, source"
" join_sig, join_ts, update_sig, update_ts, userinfo, adm_tags, adm_tags_sig,"
" signed_by, signature, source"
" FROM \"%s\""
" WHERE (node_id & %lld) == %lld ORDER BY node_id ASC",
peers_tbl, (long long)mask, (long long)prefix);
@@ -187,13 +194,15 @@ static int _member_get_items(void* ctx, const char* ns, uint8_t level,
const char* nm = (const char*)sqlite3_column_text(stmt, 7);
const char* atags = (const char*)sqlite3_column_text(stmt, 8);
const uint8_t* atsig = (const uint8_t*)sqlite3_column_blob(stmt, 9);
int src = sqlite3_column_int(stmt, 10);
uint64_t sb = (uint64_t)sqlite3_column_int64(stmt, 10);
const uint8_t* sgn = (const uint8_t*)sqlite3_column_blob(stmt, 11);
int src = sqlite3_column_int(stmt, 12);
if (!x25 || !ed || src != 0) continue;
uint8_t nl = nm ? (uint8_t)strnlen(nm, 255) : 0;
uint8_t atl = atags ? (uint8_t)strnlen(atags, 255) : 0;
uint8_t flags = (sig && jts) ? PEERS_FLAG_HAS_JOIN : 0;
size_t need = 8 + 32 + 32 + 1 + (flags ? 72ULL : 0ULL) + 64 + 8 + 1 + (size_t)nl + 1 + (size_t)atl + 64;
size_t need = 8 + 32 + 32 + 1 + (flags ? 72ULL : 0ULL) + 64 + 8 + 1 + (size_t)nl + 1 + (size_t)atl + 64 + 8 + 64;
if (off + need > *len) { sqlite3_finalize(stmt); return -2; }
memcpy(buf + off, &nid, 8); off += 8;
memcpy(buf + off, x25, 32); off += 32;
@@ -216,6 +225,9 @@ static int _member_get_items(void* ctx, const char* ns, uint8_t level,
if (atl) { memcpy(buf + off, atags, atl); off += atl; }
if (atsig && sqlite3_column_bytes(stmt, 9) >= 64) { memcpy(buf + off, atsig, 64); off += 64; }
else { memset(buf + off, 0, 64); off += 64; }
memcpy(buf + off, &sb, 8); off += 8;
if (sgn && sqlite3_column_bytes(stmt, 11) >= 64) { memcpy(buf + off, sgn, 64); off += 64; }
else { memset(buf + off, 0, 64); off += 64; }
(*cnt)++;
}
sqlite3_finalize(stmt);
@@ -228,25 +240,25 @@ static int _member_get_items(void* ctx, const char* ns, uint8_t level,
/* ── node_props_changed (adm_tags) callbacks ── */
struct ms_props_cbk { node_props_changed_fn fn; void* arg; struct ms_props_cbk* next; };
static struct ms_props_cbk* g_props_cbks = NULL;
static void _fire_props_changed(uint64_t node_id, const char* adm_tags) {
struct ms_props_cbk* pc = g_props_cbks;
while (pc) { pc->fn(node_id, adm_tags, pc->arg); pc = pc->next; }
static void _fire_props_changed(struct UTUN_INSTANCE* inst, uint64_t node_id, const char* adm_tags, const char* channel_id) {
if (!inst || !CC(inst)) return;
struct ms_props_cbk* pc = CC(inst)->props_cbks;
while (pc) { pc->fn(node_id, adm_tags, channel_id, pc->arg); pc = pc->next; }
}
void member_sync_add_props_cbk(node_props_changed_fn fn, void* arg) {
if (!fn) return;
void member_sync_add_props_cbk(struct UTUN_INSTANCE* inst, node_props_changed_fn fn, void* arg) {
if (!inst || !fn || !CC(inst)) return;
struct ms_props_cbk* e = u_malloc(sizeof(*e));
if (!e) return;
e->fn = fn; e->arg = arg;
e->next = g_props_cbks;
g_props_cbks = e;
e->next = CC(inst)->props_cbks;
CC(inst)->props_cbks = e;
}
void member_sync_remove_props_cbk(node_props_changed_fn fn, void* arg) {
if (!fn) return;
struct ms_props_cbk** p = &g_props_cbks;
void member_sync_remove_props_cbk(struct UTUN_INSTANCE* inst, node_props_changed_fn fn, void* arg) {
if (!inst || !fn || !CC(inst)) return;
struct ms_props_cbk** p = &CC(inst)->props_cbks;
while (*p) {
if ((*p)->fn == fn && (*p)->arg == arg) {
struct ms_props_cbk* r = *p;
@@ -257,6 +269,40 @@ void member_sync_remove_props_cbk(node_props_changed_fn fn, void* arg) {
}
}
/* ── Коллбэк применения рекорда мембера (ready-хендлинг join на connection-узле) ── */
struct ms_apply_cbk { member_apply_cbk_fn fn; void* arg; struct ms_apply_cbk* next; };
static void _fire_apply_cbk(struct UTUN_INSTANCE* inst, const char* ch_id, uint64_t node_id,
const uint8_t* x25519, uint64_t signed_by,
const uint8_t* signature) {
if (!inst || !CC(inst)) return;
struct ms_apply_cbk* ac = CC(inst)->apply_cbks;
while (ac) { ac->fn(ch_id, node_id, x25519, signed_by, signature, ac->arg); ac = ac->next; }
}
void member_sync_add_apply_cbk(struct UTUN_INSTANCE* inst, member_apply_cbk_fn fn, void* arg) {
if (!inst || !fn || !CC(inst)) return;
struct ms_apply_cbk* e = u_malloc(sizeof(*e));
if (!e) return;
e->fn = fn; e->arg = arg;
e->next = CC(inst)->apply_cbks;
CC(inst)->apply_cbks = e;
}
void member_sync_remove_apply_cbk(struct UTUN_INSTANCE* inst, member_apply_cbk_fn fn, void* arg) {
if (!inst || !fn || !CC(inst)) return;
struct ms_apply_cbk** p = &CC(inst)->apply_cbks;
while (*p) {
if ((*p)->fn == fn && (*p)->arg == arg) {
struct ms_apply_cbk* r = *p;
*p = r->next; u_free(r);
return;
}
p = &(*p)->next;
}
}
/* ── BGP node event callback: BGP сообщает member_sync об обновлении адресов/классификации.
Классификацию node_type/storage пересчитывает сам BGP (nodeinfo_updated);
member_sync только рассылает node_props_changed потребителям. ── */
@@ -283,7 +329,7 @@ static void _ms_on_bgp_node(struct TOPO_GROUP* group, uint64_t node_id, int even
DEBUG_DEBUG(DEBUG_CATEGORY_MEMBER_SYNC, "%s: bgp_node ev=%d nid=0x%016llx tags=%s",
MS_ID, event, (unsigned long long)node_id, tags);
_fire_props_changed(node_id, tags[0] ? tags : NULL);
_fire_props_changed(inst, node_id, tags[0] ? tags : NULL, group->channel_id);
}
static int _sig_is_zero64(const uint8_t* sig) {
@@ -316,6 +362,60 @@ static int _verify_join_sig(sqlite3* db, const char* ch_id, const struct ms_memb
return ok ? 1 : 0;
}
/* ── Дерево приглашений: подпись pubkey узла ── */
int member_sync_build_sign_msg(const uint8_t* x25519, uint8_t* out, int out_sz) {
if (!x25519 || !out || out_sz < MEMBER_SIGN_MSG_SIZE) return -1;
memcpy(out, x25519, SC_PUBKEY_SIZE);
memcpy(out + SC_PUBKEY_SIZE, MEMBER_SIGN_SUFFIX, MEMBER_SIGN_SUFFIX_LEN);
return MEMBER_SIGN_MSG_SIZE;
}
int member_sync_sign_pubkey(const uint8_t* signer_ed_priv, const uint8_t* x25519, uint8_t sig[64]) {
if (!signer_ed_priv || !x25519 || !sig) return -1;
uint8_t msg[MEMBER_SIGN_MSG_SIZE];
if (member_sync_build_sign_msg(x25519, msg, (int)sizeof(msg)) < 0) return -1;
if (sc_ed25519_sign(signer_ed_priv, msg, sizeof(msg), sig) != SC_OK) {
DEBUG_ERROR(DEBUG_CATEGORY_MEMBER_SYNC, "%s: sign_pubkey — sc_ed25519_sign FAILED", MS_ID);
return -1;
}
return 0;
}
int member_sync_verify_signature(sqlite3* db, const char* ch_id, const struct ms_member_rec* m) {
if (!db || !ch_id || !m || !m->x25519 || !m->signature || _sig_is_zero64(m->signature)) return 0;
uint8_t msg[MEMBER_SIGN_MSG_SIZE];
if (member_sync_build_sign_msg(m->x25519, msg, (int)sizeof(msg)) < 0) return -1;
uint8_t signer_pub[32];
if (m->signed_by == 0) {
if (topo_node_sqlite_channel_get(db, ch_id, NULL, 0, NULL, NULL, signer_pub, NULL) != 0) {
DEBUG_DEBUG(DEBUG_CATEGORY_MEMBER_SYNC, "%s: verify_signature — no channel key ns=%s (defer)", MS_ID, ch_id);
return -1;
}
} else {
if (topo_node_sqlite_get_ed25519_pubkey(db, m->signed_by, signer_pub) != 0) {
DEBUG_DEBUG(DEBUG_CATEGORY_MEMBER_SYNC, "%s: verify_signature — no ed25519 pubkey for signed_by=0x%016llx ns=%s (defer)", MS_ID, (unsigned long long)m->signed_by, ch_id);
return -1;
}
}
int ok = sc_ed25519_verify(signer_pub, msg, sizeof(msg), m->signature) == SC_OK;
if (!ok)
DEBUG_ERROR(DEBUG_CATEGORY_MEMBER_SYNC, "%s: verify_signature FAIL nid=0x%016llx signed_by=0x%016llx ns=%s",
MS_ID, (unsigned long long)m->node_id, (unsigned long long)m->signed_by, ch_id);
return ok ? 1 : 0;
}
int member_sync_validate_pubkey(struct UTUN_INSTANCE* inst, const char* ch_id,
const uint8_t* x25519_pubkey) {
if (!inst || !ch_id || !x25519_pubkey) return 0;
sqlite3* db = _db(inst); if (!db) return 0;
uint64_t node_id = sc_derive_node_id_from_pubkey(x25519_pubkey);
if (node_id == 0) return 0;
return topo_node_sqlite_member_in_channel(db, ch_id, node_id);
}
int member_sync_apply_record(struct UTUN_INSTANCE* inst, const char* ch_id,
uint64_t from_peer, const struct ms_member_rec* m) {
if (!inst || !ch_id || !m || !m->x25519 || !m->ed25519) {
@@ -336,10 +436,12 @@ int member_sync_apply_record(struct UTUN_INSTANCE* inst, const char* ch_id,
uint8_t local_join_sig[64] = {0}; uint64_t local_join_ts = 0;
uint64_t local_update_ts = 0;
char local_tags[256] = "";
uint64_t local_signed_by = 0;
uint8_t local_signature[64] = {0};
{
sqlite3_stmt* st = NULL;
char sql[512]; snprintf(sql, sizeof(sql),
"SELECT join_sig, join_ts, update_ts, adm_tags FROM \"%s\" WHERE node_id=?", peers_tbl);
"SELECT join_sig, join_ts, update_ts, adm_tags, signed_by, signature FROM \"%s\" WHERE node_id=?", peers_tbl);
if (sqlite3_prepare_v2(db, sql, -1, &st, NULL) == SQLITE_OK) {
sqlite3_bind_int64(st, 1, (sqlite3_int64)m->node_id);
if (sqlite3_step(st) == SQLITE_ROW) {
@@ -350,6 +452,9 @@ int member_sync_apply_record(struct UTUN_INSTANCE* inst, const char* ch_id,
local_update_ts = (uint64_t)sqlite3_column_int64(st, 2);
const char* tags = (const char*)sqlite3_column_text(st, 3);
if (tags) snprintf(local_tags, sizeof(local_tags), "%s", tags);
local_signed_by = (uint64_t)sqlite3_column_int64(st, 4);
const uint8_t* ls = (const uint8_t*)sqlite3_column_blob(st, 5);
if (ls && sqlite3_column_bytes(st, 5) >= 64) memcpy(local_signature, ls, 64);
}
sqlite3_finalize(st);
}
@@ -371,6 +476,15 @@ int member_sync_apply_record(struct UTUN_INSTANCE* inst, const char* ch_id,
identity_ok = 0;
}
}
/* подпись дерева приглашений: -1 = родитель ещё не пришёл (defer до verify_local), 0 = явная подделка */
if (identity_ok && m->signature && !_sig_is_zero64(m->signature)) {
int sv = member_sync_verify_signature(db, ch_id, m);
if (sv == 0) {
DEBUG_ERROR(DEBUG_CATEGORY_MEMBER_SYNC, "%s: apply_record invalid tree-signature (forgery) nid=0x%016llx signed_by=0x%016llx ns=%s",
MS_ID, (unsigned long long)m->node_id, (unsigned long long)m->signed_by, ch_id);
identity_ok = 0;
}
}
}
int block_a_ok = 0;
@@ -462,9 +576,12 @@ int member_sync_apply_record(struct UTUN_INSTANCE* inst, const char* ch_id,
/* ── запись изменённых блоков ── */
if (block_a_changed) {
const uint8_t* jsig = m->join_sig ? m->join_sig : (has_local ? local_join_sig : NULL);
/* дерево приглашений: если в обновлении подписи нет — сохраняем локальную (set один раз при join/create) */
uint64_t w_signed_by = m->signature ? m->signed_by : local_signed_by;
const uint8_t* w_signature = m->signature ? m->signature : (has_local ? local_signature : NULL);
if (topo_node_sqlite_member_block_put(db, ch_id, m->node_id, jsig, m->join_ts,
m->update_sig, m->update_ts, m->x25519, m->ed25519,
m->userinfo ? m->userinfo : "", NULL) != 0) {
m->userinfo ? m->userinfo : "", NULL, w_signed_by, w_signature) != 0) {
DEBUG_ERROR(DEBUG_CATEGORY_MEMBER_SYNC, "%s: apply_record block_put FAILED ch=%s nid=0x%016llx", MS_ID, ch_id, (unsigned long long)m->node_id);
return -1;
}
@@ -497,6 +614,16 @@ int member_sync_apply_record(struct UTUN_INSTANCE* inst, const char* ch_id,
if (rc == 0) changed = 0; /* запись была no-op (данные идентичны) */
}
/* ── пассивное добавление: если узел уже подключён напрямую — добавить в CHAT-группу ── */
if (changed && inst->topo_groups) {
uint64_t gid = strtoull(ch_id, NULL, 10);
struct TOPO_GROUP* g = topo_groups_find(inst->topo_groups, gid);
if (g && g->group_type == TOPO_GROUP_TYPE_CHAT) {
struct ETCP_CONN* c = instance_find_conn(inst, m->node_id);
if (c) topo_group_new_conn(g, c);
}
}
DEBUG_DEBUG(DEBUG_CATEGORY_MEMBER_SYNC, "%s: apply_record ch=%s nid=0x%016llx → changed=%d stale=%d",
MS_ID, ch_id, (unsigned long long)m->node_id, changed, stale);
return (changed ? MS_APPLY_CHANGED : 0) | (stale ? MS_APPLY_STALE : 0);
@@ -562,6 +689,17 @@ int member_sync_verify_local_record(struct sqlite3* db, const char* ch_id, const
return 0;
}
}
/* дерево приглашений: подпись есть и невалидна (подписант резолвится) — битая запись.
-1 = родитель ещё не пришёл — не трогаем (не битая). */
if (m->signature && !_sig_is_zero64(m->signature)) {
int sv = member_sync_verify_signature(db, ch_id, m);
if (sv == 0) {
DEBUG_ERROR(DEBUG_CATEGORY_MEMBER_SYNC, "%s: verify_local — invalid tree-signature nid=0x%016llx signed_by=0x%016llx ns=%s",
MS_ID, (unsigned long long)m->node_id, (unsigned long long)m->signed_by, ch_id);
return 0;
}
}
return 1;
}
@@ -574,7 +712,8 @@ int member_sync_verify_and_purge(struct UTUN_INSTANCE* inst, const char* ch_id)
sqlite3_stmt* st = NULL;
char sql[512]; snprintf(sql, sizeof(sql),
"SELECT node_id, x25519_pubkey, ed25519_pubkey,"
" join_sig, join_ts, update_sig, update_ts, userinfo, adm_tags, adm_tags_sig, source"
" join_sig, join_ts, update_sig, update_ts, userinfo, adm_tags, adm_tags_sig,"
" signed_by, signature, source"
" FROM \"%s\" ORDER BY node_id ASC", peers_tbl);
if (sqlite3_prepare_v2(db, sql, -1, &st, NULL) != SQLITE_OK) return 0;
@@ -590,7 +729,9 @@ int member_sync_verify_and_purge(struct UTUN_INSTANCE* inst, const char* ch_id)
const char* nm = (const char*)sqlite3_column_text(st, 7);
const char* atags = (const char*)sqlite3_column_text(st, 8);
const uint8_t* atsig = (const uint8_t*)sqlite3_column_blob(st, 9);
int src = sqlite3_column_int(st, 10);
uint64_t sb = (uint64_t)sqlite3_column_int64(st, 10);
const uint8_t* sgn = (const uint8_t*)sqlite3_column_blob(st, 11);
int src = sqlite3_column_int(st, 12);
if (!x25 || !ed) continue;
if (src != 0) continue; /* плейсхолдер topo_group — не верифицируем */
@@ -602,12 +743,22 @@ int member_sync_verify_and_purge(struct UTUN_INSTANCE* inst, const char* ch_id)
m.update_sig = usig; m.update_ts = uts;
m.userinfo = nm;
m.adm_tags = atags; m.adm_tags_sig = atsig;
m.signed_by = sb; m.signature = sgn;
if (!member_sync_verify_local_record(db, ch_id, &m)) {
DEBUG_ERROR(DEBUG_CATEGORY_MEMBER_SYNC, "%s: verify_and_purge — PURGE broken record nid=0x%016llx ns=%s",
MS_ID, (unsigned long long)nid, ch_id);
topo_node_sqlite_member_del(db, ch_id, nid);
merkle_sync_recompute_path(inst, ch_id, nid);
/* пассивное удаление: если узел подключён напрямую — убрать из CHAT-группы */
if (inst->topo_groups) {
uint64_t gid = strtoull(ch_id, NULL, 10);
struct TOPO_GROUP* g = topo_groups_find(inst->topo_groups, gid);
if (g && g->group_type == TOPO_GROUP_TYPE_CHAT) {
struct ETCP_CONN* c = instance_find_conn(inst, nid);
if (c) topo_group_remove_conn(g, c);
}
}
purged++;
}
}
@@ -669,6 +820,8 @@ static int _member_apply_items(void* ctx, const char* ns, uint64_t from_peer,
const uint8_t* atsig = NULL;
if (atl && mrem >= atl) { memcpy(atags, mp, atl); atags[atl] = '\0'; mp += atl; mrem -= atl; }
if (mrem >= 64) { atsig = mp; mp += 64; mrem -= 64; }
uint64_t sb = 0; const uint8_t* sgn = NULL;
if (mrem >= 72) { memcpy(&sb, mp, 8); mp += 8; mrem -= 8; sgn = mp; mp += 64; mrem -= 64; }
struct ms_member_rec m;
memset(&m, 0, sizeof(m));
@@ -679,6 +832,7 @@ static int _member_apply_items(void* ctx, const char* ns, uint64_t from_peer,
m.userinfo = nm;
m.adm_tags = atags[0] ? atags : NULL;
m.adm_tags_sig = atsig;
m.signed_by = sb; m.signature = sgn;
int r = member_sync_apply_record(inst, ns, from_peer, &m);
DEBUG_DEBUG(DEBUG_CATEGORY_MEMBER_SYNC, "%s: member_sync recv ns=%s nid=0x%016llx r=%d", MS_ID, ns, (unsigned long long)nid, r);
@@ -686,11 +840,9 @@ static int _member_apply_items(void* ctx, const char* ns, uint64_t from_peer,
if (r & MS_APPLY_CHANGED) {
changed_count++;
/* fire node_props_changed callbacks */
struct ms_props_cbk* pc = g_props_cbks;
if (pc) {
const char* tags_final = atags[0] ? atags : NULL;
while (pc) { pc->fn(nid, tags_final, pc->arg); pc = pc->next; }
}
_fire_props_changed(inst, nid, atags[0] ? atags : NULL, ns);
/* fire apply callbacks (ready-хендлинг join на connection-узле) */
_fire_apply_cbk(inst, ns, nid, x25, sb, sgn);
}
if (r & MS_APPLY_STALE) {
/* у нас версия новее — отправить наш полный рекорд автору */
@@ -702,25 +854,19 @@ static int _member_apply_items(void* ctx, const char* ns, uint64_t from_peer,
return 0;
}
static member_sync_node_updated_fn g_node_updated_cb = NULL;
static int _ms_apply_update(void* ctx, const char* ns, uint64_t key,
uint8_t type, const uint8_t* data, size_t len) {
(void)ns;
static int _member_validate_peer(void* ctx, const char* ns, uint64_t peer) {
struct UTUN_INSTANCE* inst = (struct UTUN_INSTANCE*)ctx;
if (type == 0x01 && len >= 1) {
int online = data[0];
member_sync_set_online(inst, key, online);
if (g_node_updated_cb) g_node_updated_cb(key, online);
}
return 0;
sqlite3* db = _db(inst);
if (!db) return 0;
return topo_node_sqlite_member_in_channel(db, ns, peer);
}
static const struct merkle_sync_data_ops g_member_ops = {
.update_bucket_hash = _member_update_bucket_hash,
.get_items = _member_get_items,
.apply_items = _member_apply_items,
.apply_update = _ms_apply_update,
.apply_update = NULL, /* online-статус больше не рассылается через merkle */
.validate_peer = _member_validate_peer,
};
/* ── Public API ── */
@@ -808,6 +954,10 @@ void member_sync_cancel(struct UTUN_INSTANCE* inst, uint64_t peer, const char* c
merkle_sync_cancel(inst, peer, ch_id);
}
void member_sync_cancel_channel(struct UTUN_INSTANCE* inst, const char* ch_id) {
merkle_sync_cancel_ns(inst, ch_id);
}
/* Сериализует одного мембера в wire-формат [count:2][member...].
Возвращает 0 и *out_len, или -1 если мембер не найден/нет БД. */
static int _serialize_member(struct UTUN_INSTANCE* inst, const char* ch_id, uint64_t member_id,
@@ -819,7 +969,8 @@ static int _serialize_member(struct UTUN_INSTANCE* inst, const char* ch_id, uint
sqlite3_stmt* stmt = NULL;
char sql[512]; snprintf(sql, sizeof(sql),
"SELECT node_id, x25519_pubkey, ed25519_pubkey,"
" join_sig, join_ts, update_sig, update_ts, userinfo, adm_tags, adm_tags_sig"
" join_sig, join_ts, update_sig, update_ts, userinfo, adm_tags, adm_tags_sig,"
" signed_by, signature"
" FROM \"%s\" WHERE node_id=?", peers_tbl);
if (sqlite3_prepare_v2(db, sql, -1, &stmt, NULL) != SQLITE_OK) {
DEBUG_ERROR(DEBUG_CATEGORY_MEMBER_SYNC, "%s: serialize_member — query failed ch=%s nid=0x%016llx", MS_ID, ch_id, (unsigned long long)member_id);
@@ -838,13 +989,15 @@ static int _serialize_member(struct UTUN_INSTANCE* inst, const char* ch_id, uint
const char* nm = (const char*)sqlite3_column_text(stmt, 7);
const char* atags = (const char*)sqlite3_column_text(stmt, 8);
const uint8_t* atsig = (const uint8_t*)sqlite3_column_blob(stmt, 9);
uint64_t sb = (uint64_t)sqlite3_column_int64(stmt, 10);
const uint8_t* sgn = (const uint8_t*)sqlite3_column_blob(stmt, 11);
if (!x25 || !ed) { sqlite3_finalize(stmt); return -1; }
uint8_t nl = nm ? (uint8_t)strnlen(nm, 255) : 0;
uint8_t atl = atags ? (uint8_t)strnlen(atags, 255) : 0;
uint8_t flags = (sig && jts) ? PEERS_FLAG_HAS_JOIN : 0;
size_t need = 2 + 8 + 32 + 32 + 1 + (flags & PEERS_FLAG_HAS_JOIN ? 72 : 0)
+ 72 + 1 + (size_t)nl + 1 + (size_t)atl + 64;
+ 72 + 1 + (size_t)nl + 1 + (size_t)atl + 64 + 8 + 64;
if (need > buf_sz) { sqlite3_finalize(stmt); return -1; }
size_t off = 0;
uint16_t wcnt = 1; memcpy(buf, &wcnt, 2); off += 2;
@@ -860,6 +1013,10 @@ static int _serialize_member(struct UTUN_INSTANCE* inst, const char* ch_id, uint
if (atsig && sqlite3_column_bytes(stmt, 9) >= 64) { memcpy(buf + off, atsig, 64); }
else { memset(buf + off, 0, 64); }
off += 64;
memcpy(buf + off, &sb, 8); off += 8;
if (sgn && sqlite3_column_bytes(stmt, 11) >= 64) { memcpy(buf + off, sgn, 64); }
else { memset(buf + off, 0, 64); }
off += 64;
sqlite3_finalize(stmt);
*out_len = off;
@@ -887,9 +1044,10 @@ int member_sync_put(struct UTUN_INSTANCE* inst, const char* ch_id,
const uint8_t* join_sig, uint64_t join_ts,
const uint8_t* update_sig, uint64_t update_ts,
const char* userinfo,
const char* adm_tags, const uint8_t* adm_tags_sig, int storage) {
const char* adm_tags, const uint8_t* adm_tags_sig, int storage,
uint64_t signed_by, const uint8_t* signature) {
if (!inst || !ch_id) { DEBUG_ERROR(DEBUG_CATEGORY_MEMBER_SYNC, "%s: put — inst=%p ch_id=%s", MS_ID, (void*)inst, ch_id ? ch_id : "(null)"); return -1; }
DEBUG_TRACE(DEBUG_CATEGORY_MEMBER_SYNC, "%s: put ch=%s nid=%016llx userinfo=%s adm_tags=%s storage=%d", MS_ID, ch_id, (unsigned long long)member_id, userinfo ? userinfo : "", adm_tags ? adm_tags : "", storage);
DEBUG_TRACE(DEBUG_CATEGORY_MEMBER_SYNC, "%s: put ch=%s nid=%016llx userinfo=%s adm_tags=%s storage=%d signed_by=%016llx", MS_ID, ch_id, (unsigned long long)member_id, userinfo ? userinfo : "", adm_tags ? adm_tags : "", storage, (unsigned long long)signed_by);
struct ms_member_rec m;
memset(&m, 0, sizeof(m));
@@ -899,6 +1057,7 @@ int member_sync_put(struct UTUN_INSTANCE* inst, const char* ch_id,
m.update_sig = update_sig; m.update_ts = update_ts;
m.userinfo = userinfo;
m.adm_tags = adm_tags; m.adm_tags_sig = adm_tags_sig; m.storage = storage;
m.signed_by = signed_by; m.signature = signature;
return member_sync_apply_record(inst, ch_id, inst->node_id, &m);
}
@@ -916,23 +1075,6 @@ int member_sync_count(struct UTUN_INSTANCE* inst, const char* ch_id) {
return c;
}
void member_sync_set_node_updated_cb(member_sync_node_updated_fn cb) {
g_node_updated_cb = cb;
}
int member_sync_set_online(struct UTUN_INSTANCE* inst, uint64_t node_id, int online) {
/* ВАЖНО: онлайн-статус НЕ участвует в merkle sync — не хешируется и не
* версионируется. Только локальная запись в nodes.online. Рассылка online
* идёт отдельным лёгким push_update (MSG_ITEM_UPDATE), вне дерева. */
if (!inst) return 0;
DEBUG_TRACE(DEBUG_CATEGORY_MEMBER_SYNC, "%s: set_online nid=%016llx online=%d", MS_ID, (unsigned long long)node_id, online);
sqlite3* db = _db(inst); if (!db) return 0;
int val = online ? 1 : 0;
if (topo_node_sqlite_node_get_online(db, node_id) == val) return 0;
topo_node_sqlite_node_set_online(db, node_id, val);
return 1;
}
const uint8_t* member_sync_get_hash(struct UTUN_INSTANCE* inst, const char* ch_id,
uint8_t level, uint64_t prefix64) {
return merkle_sync_get_hash(inst, ch_id, level, prefix64);
+45 -29
View File
@@ -25,11 +25,6 @@
* adm_tags, adm_tags_sig, storage);
* // дерево автоматически пересчитано
*
* // онлайн-статус (из cs_on_conn_up / cs_on_conn_down)
* // только локальная запись; возвращает 1 если изменилось — тогда рассылка:
* if (member_sync_set_online(inst, node_id, 1))
* merkle_sync_push_update(inst, ch_id, node_id, 0x01, (uint8_t[]){1}, 1);
*
* // отмена синхронизации (коллбэк НЕ вызывается)
* member_sync_cancel(inst, peer, ch_id);
*
@@ -73,8 +68,41 @@ struct ms_member_rec {
const char* adm_tags; /* JSON владельца с "ver", может быть NULL */
const uint8_t* adm_tags_sig; /* 64 байта, может быть NULL */
int storage; /* 0/1, производная от adm_tags.storage */
uint64_t signed_by; /* дерево приглашений: 0 = корень (подписан ключом группы), иначе node_id инвайтера */
const uint8_t* signature; /* 64 байта, Ed25519 подпись x25519||"group_join" ключом родителя (или канальным ed25519) */
};
/* ── Дерево приглашений (подпись pubkey узла) ── */
#define MEMBER_SIGN_SUFFIX "group_join"
#define MEMBER_SIGN_SUFFIX_LEN 10
#define MEMBER_SIGN_MSG_SIZE (SC_PUBKEY_SIZE + MEMBER_SIGN_SUFFIX_LEN) /* 42 */
/* Собрать сообщение подписи: x25519(32) || "group_join". Возвращает 42 или -1. */
int member_sync_build_sign_msg(const uint8_t* x25519, uint8_t* out, int out_sz);
/* Подписать pubkey узла: sign(signer_ed_priv, x25519||"group_join"). Возвращает 0/-1. */
int member_sync_sign_pubkey(const uint8_t* signer_ed_priv, const uint8_t* x25519, uint8_t sig[64]);
/* Верифицировать подпись дерева мембера:
* signed_by==0 → ключ канала (ch_ed25519), иначе ed25519 pubkey узла signed_by.
* Возвращает 1 если подпись валидна, 0 если невалидна, -1 если подписанта нет в БД
* (родитель ещё не пришёл по merkle — верификацию откладываем). */
int member_sync_verify_signature(struct sqlite3* db, const char* ch_id, const struct ms_member_rec* m);
/* Проверить, что x25519 pubkey (из ETCP-соединения) есть в таблице мемберов канала.
* node_id = derive(x25519) → SELECT 1 FROM peers_<ch_id>. Возвращает 1/0. */
int member_sync_validate_pubkey(struct UTUN_INSTANCE* inst, const char* ch_id,
const uint8_t* x25519_pubkey);
/* ── Коллбэк приёма/применения рекорда мембера (ready-хендлинг join) ── */
typedef void (*member_apply_cbk_fn)(const char* ch_id, uint64_t node_id,
const uint8_t* x25519, uint64_t signed_by,
const uint8_t* signature, void* arg);
void member_sync_add_apply_cbk(struct UTUN_INSTANCE* inst, member_apply_cbk_fn fn, void* arg);
void member_sync_remove_apply_cbk(struct UTUN_INSTANCE* inst, member_apply_cbk_fn fn, void* arg);
/*
* Инициализировать модуль: создаёт merkle_sync с коллбэками для мемберов
* (ETCP сервис 0x31), регистрирует _on_node_updated для пересчёта дерева
@@ -101,6 +129,12 @@ int member_sync_start(struct UTUN_INSTANCE* inst, uint64_t peer,
*/
void member_sync_cancel(struct UTUN_INSTANCE* inst, uint64_t peer, const char* ch_id);
/*
* Отменить все merkle-сессии канала (локальное удаление канала).
* Делегирует в merkle_sync_cancel_ns().
*/
void member_sync_cancel_channel(struct UTUN_INSTANCE* inst, const char* ch_id);
/*
* Добавить/обновить мембера (локальная запись, доверенная) и пересчитать дерево.
* Делегирует в member_sync_apply_record() с from_peer = inst->node_id.
@@ -128,7 +162,8 @@ int member_sync_put(struct UTUN_INSTANCE* inst, const char* ch_id,
const uint8_t* join_sig, uint64_t join_ts,
const uint8_t* update_sig, uint64_t update_ts,
const char* userinfo,
const char* adm_tags, const uint8_t* adm_tags_sig, int storage);
const char* adm_tags, const uint8_t* adm_tags_sig, int storage,
uint64_t signed_by, const uint8_t* signature);
/*
* Строит сообщение для update_sig блока A (мембер):
@@ -191,38 +226,19 @@ int member_sync_count(struct UTUN_INSTANCE* inst, const char* ch_id);
Вызывается после локального изменения данных (adm_tags, имя, адреса). */
void member_sync_broadcast_one(struct UTUN_INSTANCE* inst, const char* ch_id, uint64_t member_id);
/*
* Установить онлайн-статус узла (nodes.online = 0/1).
*
* ВАЖНО: онлайн-статус НЕ участвует в merkle sync — он не хешируется и не
* версионируется (в _compute_member_hash поля online нет). Это только локальная
* запись в БД. Рассылка online идёт отдельным лёгким push_update (MSG_ITEM_UPDATE),
* вне дерева, и только если значение реально изменилось (возвращает 1).
*
* Возвращает 1 если значение изменилось, 0 если совпало.
*/
int member_sync_set_online(struct UTUN_INSTANCE* inst, uint64_t node_id, int online);
/* Получить хеш бакета из merkle_tree_hash (для тестов). */
const uint8_t* member_sync_get_hash(struct UTUN_INSTANCE* inst,
const char* ch_id, uint8_t level, uint64_t prefix64);
/*
* Коллбэк: узел изменился по инициативе удалённого пира (через MSG_ITEM_UPDATE).
* Вызывается из uasync-потока. Потребитель (chat_sync) может из него
* постить GUI-события.
*/
typedef void (*member_sync_node_updated_fn)(uint64_t node_id, int online);
void member_sync_set_node_updated_cb(member_sync_node_updated_fn cb);
/*
* Коллбэк: изменились adm_tags любого узла (включая себя).
* channel_id — канал (namespace), в котором произошло изменение.
* Вызывается при успешной обработке MSG_ITEM_UPDATE с adm_tags.
* Многоподписочный — можно добавить несколько подписчиков.
*/
typedef void (*node_props_changed_fn)(uint64_t node_id, const char* adm_tags, void* arg);
void member_sync_add_props_cbk(node_props_changed_fn fn, void* arg);
void member_sync_remove_props_cbk(node_props_changed_fn fn, void* arg);
typedef void (*node_props_changed_fn)(uint64_t node_id, const char* adm_tags, const char* channel_id, void* arg);
void member_sync_add_props_cbk(struct UTUN_INSTANCE* inst, node_props_changed_fn fn, void* arg);
void member_sync_remove_props_cbk(struct UTUN_INSTANCE* inst, node_props_changed_fn fn, void* arg);
#ifdef __cplusplus
}
Loaded 100 of 341 files, more files were not shown because too many files have changed in this diff. Show more