Files
+1 98652c6476 Include sub projects (#10201)
* Add bump-changes

* Add utility tests

* Add utility tests

* Bump to new version of esbuild and typescript

* v0.7.3

* use platform rig 0.7.10

* upgrade: memory engine optimized; change name  to  (was recommended by Copilot and Onnikov, TODO: CHANGE CLIENT TOO!!!)

Signed-off-by: Leonid Kaganov <lleo@lleo.me>

* Fix rate limits bug

* Bump versions

* Fix lock file

* Fix bug in queue cleanup

* Add more tests for queue

* Add api-test tests

* Initial commit

* Improve hierarchy + tests

Add tests for hierarchy and few performance/memory  optimizations.

* Add more hierarchy tests

* Move from Huly platform repository

* Add docker tests setup

* Fix test to be executed only once

* Add connection tests

* Fix package include source files

* More tests

* Create README.md

* Fix pnpm lock

* Fix packages publish

* Remove broken tests

* feat: adjust hulylake client for storage adapter

Signed-off-by: Alexander Onnikov <Alexander.Onnikov@xored.com>

* Fix export

* Fix publish

* Fix message update (#114)

Signed-off-by: Kristina Fefelova <kristin.fefelova@gmail.com>

* Bump version

Signed-off-by: Kristina Fefelova <kristin.fefelova@gmail.com>

* Bump versions

Signed-off-by: Alexander Onnikov <Alexander.Onnikov@xored.com>

* Fix lang store (#115)

Signed-off-by: Kristina Fefelova <kristin.fefelova@gmail.com>

* update hulylake client

Signed-off-by: Alexander Onnikov <Alexander.Onnikov@xored.com>

* Bump version

Signed-off-by: Alexander Onnikov <Alexander.Onnikov@xored.com>

* Add hulylake storage adapter

Signed-off-by: Alexander Onnikov <Alexander.Onnikov@xored.com>

* Bump version

Signed-off-by: Alexander Onnikov <Alexander.Onnikov@xored.com>

* fix validation issues

Signed-off-by: Alexander Onnikov <Alexander.Onnikov@xored.com>

* fix: do not fail on deseralization error and add logs

Signed-off-by: Alexander Onnikov <Alexander.Onnikov@xored.com>

* bump version -> 0.1.14

Signed-off-by: Alexander Onnikov <Alexander.Onnikov@xored.com>

* fix collaboration test

Signed-off-by: Alexander Onnikov <Alexander.Onnikov@xored.com>

* Update prettier and new update-deps script

Prettier + svelte support

* fix unstable ydoc tests

Signed-off-by: Alexander Onnikov <Alexander.Onnikov@xored.com>

* Add tx ordering middleware

* Fix ordering tests

* Fix Kafka close of admin

* Add tests for measurement and understand overhead

* Fix not updated lock file

* Fix update-deps

* Fix update-deps

* Use latest platform-rig

* Fix deps

* Add rush check to CI

* Use latest versions

* Bump versions

* Fix lock file

* validate json patch

Signed-off-by: Alexander Onnikov <Alexander.Onnikov@xored.com>

* bump version -> 0.1.15

Signed-off-by: Alexander Onnikov <Alexander.Onnikov@xored.com>

* fix merge unit tests

Signed-off-by: Alexander Onnikov <Alexander.Onnikov@xored.com>

* Script to sync eslint deps

* Fix deps

* Fix tests

* Fix platform-rig detection

* Update to latest platform-rig

* Update to latest platform rig and core

* Bump typescript

* Bump typescript

* Rollback eslint plugins

* Fix lock file

* Bump platform-rig

* Update to latest platform-rig

* update to latest platform-rig

* Allow to compile svelte files

* Add ui-test component for checking compile

* Fix log levels rename compile ui -> compile ui-esbuild

* Fix build

* Bump esbuild svelte version

* Chore: use fixed versions in update-deps

Signed-off-by: Alexey Zinoviev <alexey.zinoviev@xored.com>

* Chore: commit changes

Signed-off-by: Alexey Zinoviev <alexey.zinoviev@xored.com>

* Update deps

* Add tests for session manager

* Fix txOrdering implementation

* Bump ordering

* Prevent metrics zero values in measure

+ Fix format svelte files

* Revert update-deps script logic

* v0.7.19

* update to latest platform-rig

* Update deps

* Fix pnpm

* Session counters

* Fix pnpm lock

* Add storage client

Signed-off-by: Alexander Onnikov <Alexander.Onnikov@xored.com>

* Bump versions

Signed-off-by: Alexander Onnikov <Alexander.Onnikov@xored.com>

* fix versions

Signed-off-by: Alexander Onnikov <Alexander.Onnikov@xored.com>

* Bump core

* Fix pnpm

* Get rid of communication dependency

* Add copilot memory file

* Use proper name for instructions file

* Fix instructions

* Use domain instead of test name in gauges

* Update instructions file

* fix front service upload

Signed-off-by: Alexander Onnikov <Alexander.Onnikov@xored.com>

* remove incorrect test

Signed-off-by: Alexander Onnikov <Alexander.Onnikov@xored.com>

* Move packages to huly.core

* Move packages to core, since they are not utils

* Add global user profile

Signed-off-by: Alexey Zinoviev <alexey.zinoviev@xored.com>

* Fix lock file

Signed-off-by: Alexey Zinoviev <alexey.zinoviev@xored.com>

* Add support for memory limit check

* Bump version

* Fix pnpm

* report more accurate upload progress

Signed-off-by: Alexander Onnikov <Alexander.Onnikov@xored.com>

* fic validation issues

Signed-off-by: Alexander Onnikov <Alexander.Onnikov@xored.com>

* Fix deps

Signed-off-by: Kristina Fefelova <kristin.fefelova@gmail.com>

* Move LowLevelStorage to server

* Fix linting

* Revert "Fix linting"

This reverts commit 54631d353e.

* Revert "Move LowLevelStorage to server"

This reverts commit aafb8f6f12.

* feature: add regorus engine with permit file

Signed-off-by: Leonid Kaganov <lleo@lleo.me>

* Fix one second counters for memory usage

* Fix kafka test

* use fresh core

* Version bump

* fix: key parameter added

Signed-off-by: Leonid Kaganov <lleo@lleo.me>

* Fix readme and few author mistakes

* Export domain schemas

* Bump version

* Tests (#117)

Signed-off-by: Kristina Fefelova <kristin.fefelova@gmail.com>

* Bump version

Signed-off-by: Kristina Fefelova <kristin.fefelova@gmail.com>

* feat: compact compact worker (#4)

Signed-off-by: Alexander Onnikov <Alexander.Onnikov@xored.com>

* bump version -> 0.1.16

Signed-off-by: Alexander Onnikov <Alexander.Onnikov@xored.com>

* Add TypeIdentifier

Signed-off-by: Denis Bykhov <bykhov.denis@gmail.com>

* Add change logs

Signed-off-by: Denis Bykhov <bykhov.denis@gmail.com>

* rename send -> try_send

Signed-off-by: Alexander Onnikov <Alexander.Onnikov@xored.com>

* Bump

Signed-off-by: Denis Bykhov <bykhov.denis@gmail.com>

* Fix pnpm lock

Signed-off-by: Denis Bykhov <bykhov.denis@gmail.com>

* Add identifier middleware, bump core

Signed-off-by: Denis Bykhov <bykhov.denis@gmail.com>

* Add subsciption methods to account client

Signed-off-by: Alexey Zinoviev <alexey.zinoviev@xored.com>

* Fix lock file

Signed-off-by: Alexey Zinoviev <alexey.zinoviev@xored.com>

* Fix reaction notification (#118)

Signed-off-by: Kristina Fefelova <kristin.fefelova@gmail.com>

* Bump version

Signed-off-by: Kristina Fefelova <kristin.fefelova@gmail.com>

* Improve find methods schemas to convert to valid types

Signed-off-by: Nikolay Marchuk <nikolay.marchuk@hardcoreeng.com>

* Add change description

Signed-off-by: Nikolay Marchuk <nikolay.marchuk@hardcoreeng.com>

* Do not transcode while recording

Signed-off-by: Alexander Onnikov <Alexander.Onnikov@xored.com>

* Open telemetry support

Signed-off-by: Alexander Onnikov <Alexander.Onnikov@xored.com>

* use proper content type in multipart upload

Signed-off-by: Alexander Onnikov <Alexander.Onnikov@xored.com>

* Bump versions

Signed-off-by: Alexander Onnikov <Alexander.Onnikov@xored.com>

* fix build (#26)

Signed-off-by: Alexander Onnikov <Alexander.Onnikov@xored.com>

* Fix peers (#120)

Signed-off-by: Kristina Fefelova <kristin.fefelova@gmail.com>

* Bump version

Signed-off-by: Kristina Fefelova <kristin.fefelova@gmail.com>

* Add ActivityCollaborativeChange

Signed-off-by: Denis Bykhov <bykhov.denis@gmail.com>

* Update pnpm

Signed-off-by: Denis Bykhov <bykhov.denis@gmail.com>

* Allow to suspend errors on with

* Fix pnpm cache

* update versions

* v0.7.17 for all

* v0.7.11

* v0.7.14

* remove arc from worker

Signed-off-by: Alexander Onnikov <Alexander.Onnikov@xored.com>

* bump version -> 0.1.17

Signed-off-by: Alexander Onnikov <Alexander.Onnikov@xored.com>

* Rank for attributes

Signed-off-by: Denis Bykhov <bykhov.denis@gmail.com>

* Update pnpm

Signed-off-by: Denis Bykhov <bykhov.denis@gmail.com>

* Fix one second counters

* Fix withContext and allow pass options

* Fix formatting

* Use updated deps

* Bump versions

* Update deps

* Update deps to platform.core

* add support for textColor mark

Signed-off-by: Alexander Onnikov <Alexander.Onnikov@xored.com>

* add support for textStyle mark

Signed-off-by: Alexander Onnikov <Alexander.Onnikov@xored.com>

* Bump versions

Signed-off-by: Alexander Onnikov <Alexander.Onnikov@xored.com>

* Bump versions again

Signed-off-by: Alexander Onnikov <Alexander.Onnikov@xored.com>

* Fix

Signed-off-by: Denis Bykhov <bykhov.denis@gmail.com>

* Rework on second timers

* fix merge of large blobs feched from s3

Signed-off-by: Alexander Onnikov <Alexander.Onnikov@xored.com>

* bump version -> 0.1.18

Signed-off-by: Alexander Onnikov <Alexander.Onnikov@xored.com>

* New subscription methods in account-client

* Update lock file

Signed-off-by: Alexey Zinoviev <alexey.zinoviev@xored.com>

* Send error on find for wrong domain

* Suspend connect custom errors events in traces

* Bump client

* Bump core

* update deps

* Fix lock file

* Sorting for TypeIdentifier

Signed-off-by: Denis Bykhov <bykhov.denis@gmail.com>

* Bump version

Signed-off-by: Denis Bykhov <bykhov.denis@gmail.com>

* add workspace usage info

Signed-off-by: Alexander Onnikov <Alexander.Onnikov@xored.com>

* Bump versions

Signed-off-by: Alexander Onnikov <Alexander.Onnikov@xored.com>

* Improve pg security perfomance

Signed-off-by: Denis Bykhov <bykhov.denis@gmail.com>

* Fix identifier middleware

Signed-off-by: Denis Bykhov <bykhov.denis@gmail.com>

* Update TxAccessLevel interface

Signed-off-by: Alexey Zinoviev <alexey.zinoviev@xored.com>

* Allow guest to update its identities

Signed-off-by: Alexey Zinoviev <alexey.zinoviev@xored.com>

* Add password login locked platform status

Signed-off-by: Alexey Zinoviev <alexey.zinoviev@xored.com>

* Fix Uptrace normalizeMarkdown errors

Signed-off-by: Artem Savchenko <armisav@gmail.com>

* Add change log

Signed-off-by: Artem Savchenko <armisav@gmail.com>

* Add txMatch to permission

Signed-off-by: Denis Bykhov <bykhov.denis@gmail.com>

* update pnpm lock

Signed-off-by: Denis Bykhov <bykhov.denis@gmail.com>

* Bump

Signed-off-by: Denis Bykhov <bykhov.denis@gmail.com>

* Fix permission middleware

Signed-off-by: Denis Bykhov <bykhov.denis@gmail.com>

* Fix enum sorting

Signed-off-by: Denis Bykhov <bykhov.denis@gmail.com>

* Enable formatting check

Signed-off-by: Andrey Sobolev <haiodo@gmail.com>

* Enable formatting check

* Add change

Signed-off-by: Andrey Sobolev <haiodo@gmail.com>

* Fix Uptrace NaN error

Signed-off-by: Artem Savchenko <armisav@gmail.com>

* feature: removed actors, improved performance

Signed-off-by: Leonid Kaganov <lleo@lleo.me>

* feature: ping from server to clients added

Signed-off-by: Leonid Kaganov <lleo@lleo.me>

* feature: ping from server to clients added

Signed-off-by: Leonid Kaganov <lleo@lleo.me>

* Compress kafka messages and fix exception in findAll

Signed-off-by: Artem Savchenko <armisav@gmail.com>

* Bump versions

Signed-off-by: Artem Savchenko <armisav@gmail.com>

* Bump versions

Signed-off-by: Artem Savchenko <armisav@gmail.com>

* Rush change

Signed-off-by: Artem Savchenko <armisav@gmail.com>

* Fix compression param

Signed-off-by: Artem Savchenko <armisav@gmail.com>

* Trigger change

Signed-off-by: Artem Savchenko <armisav@gmail.com>

* Clean up

Signed-off-by: Artem Savchenko <armisav@gmail.com>

* Trigger change

Signed-off-by: Artem Savchenko <armisav@gmail.com>

* Bump markdown version

Signed-off-by: Artem Savchenko <armisav@gmail.com>

* Enable sub projects

* Fix wrong double symbol scripts

* Include foundation packages

* Add support for custom exclude filters

Add support for custom exclude filters - by Andrey Sobolev - haiodo@gmail.com

Signed-off-by: Andrey Sobolev <haiodo@gmail.com>

* Bump

Signed-off-by: Denis Bykhov <bykhov.denis@gmail.com>

* Fix Uptrace filter is not a function error

Signed-off-by: Artem Savchenko <armisav@gmail.com>

* Sync versions

Signed-off-by: Andrey Sobolev <haiodo@gmail.com>

---------

Signed-off-by: Leonid Kaganov <lleo@lleo.me>
Signed-off-by: Alexander Onnikov <Alexander.Onnikov@xored.com>
Signed-off-by: Kristina Fefelova <kristin.fefelova@gmail.com>
Signed-off-by: Alexey Zinoviev <alexey.zinoviev@xored.com>
Signed-off-by: Denis Bykhov <bykhov.denis@gmail.com>
Signed-off-by: Nikolay Marchuk <nikolay.marchuk@hardcoreeng.com>
Signed-off-by: Artem Savchenko <armisav@gmail.com>
Signed-off-by: Andrey Sobolev <haiodo@gmail.com>
Co-authored-by: Leonid Kaganov <lleo@lleo.me>
Co-authored-by: Alexander Onnikov <Alexander.Onnikov@xored.com>
Co-authored-by: Alexander Onnikov <Alexander.Onnikov@gmail.com>
Co-authored-by: Kristina <kristin.fefelova@gmail.com>
Co-authored-by: Alexey Zinoviev <alexey.zinoviev@xored.com>
Co-authored-by: Denis Bykhov <bykhov.denis@gmail.com>
Co-authored-by: Nikolay Marchuk <nikolay.marchuk@hardcoreeng.com>
Co-authored-by: Alexander Onnikov <aonnikov@hardcoreeng.com>
Co-authored-by: Artem Savchenko <armisav@gmail.com>
2025-11-26 19:15:30 +05:00

9.6 KiB

Quick Start Guide

Get started with Huly Virtual Network in under 10 minutes!

⚠️ Important Limitations

Network Service Constraints:

The Network Server (central coordinator) must run as a single instance only:

  • Does NOT support high availability or clustering
  • Cannot run multiple network servers simultaneously
  • Agents and containers DO support HA through stateless registration

For production, use process monitoring (systemd, PM2, Kubernetes) to ensure quick restarts. Agents automatically reconnect when the network service restarts.

Prerequisites

  • Node.js: 22.0.0 or higher
  • PNPM: 10.15 or higher (automatically installed via Rush)
  • ZeroMQ: Native dependencies (automatically installed)
  • Operating System: Linux, macOS, or Windows

Installation

# Clone the repository
git clone https://github.com/hcengineering/huly.net.git
cd huly.net

# Install dependencies
node common/scripts/install-run-rush.js install

# Build all packages
node common/scripts/install-run-rush.js build

# Run tests to verify
node common/scripts/install-run-rush.js test
# Pull the network server image
docker pull hardcoreeng/network-pod:latest

# Run the network server
docker run -d \
  --name huly-network \
  -p 3737:3737 \
  hardcoreeng/network-pod:latest

Option 3: NPM Packages (Coming Soon)

npm install @hcengineering/network-core \
            @hcengineering/network-client \
            @hcengineering/network-server

Your First Network Application

Step 1: Create a Container

Create a file my-container.ts:

import type { Container, ContainerUuid, ClientUuid } from '@hcengineering/network-core'

export class HelloWorldContainer implements Container {
  constructor(readonly uuid: ContainerUuid) {
    console.log(`Container ${uuid} created`)
  }

  async request(operation: string, data?: any): Promise<any> {
    switch (operation) {
      case 'greet':
        return { message: `Hello, ${data?.name || 'World'}!` }
      case 'status':
        return { status: 'running', uuid: this.uuid }
      default:
        return { error: 'Unknown operation' }
    }
  }

  async ping(): Promise<void> {
    // Health check
  }

  async terminate(): Promise<void> {
    console.log(`Container ${this.uuid} terminated`)
  }

  connect(clientId: ClientUuid, broadcast: (data: any) => Promise<void>): void {}
  disconnect(clientId: ClientUuid): void {}
}

Step 2: Start the Network Server

Create a file server.ts:

import { NetworkImpl, TickManagerImpl } from '@hcengineering/network-core'
import { NetworkServer } from '@hcengineering/network-server'

const tickManager = new TickManagerImpl(1000)
tickManager.start()

const network = new NetworkImpl(tickManager)
const server = new NetworkServer(network, tickManager, '*', 3737)

console.log('🚀 Network server started on port 3737')

// Graceful shutdown
process.on('SIGINT', async () => {
  console.log('\nShutting down...')
  await server.close()
  tickManager.stop()
  process.exit(0)
})

Step 3: Create an Agent

Create a file agent.ts:

import { createNetworkClient } from '@hcengineering/network-client'
import { HelloWorldContainer } from './my-container'
import type { GetOptions, ContainerUuid } from '@hcengineering/network-core'

// Create client and serve agent with container factory
const client = createNetworkClient('localhost:3737')
await client.waitConnection(5000)

await client.serveAgent('localhost:3738', {
  'hello-world': async (options: GetOptions) => {
    const uuid = options.uuid ?? (`hello-${Date.now()}` as ContainerUuid)
    const container = new HelloWorldContainer(uuid)
    return {
      uuid,
      container,
      endpoint: `hello://localhost/${uuid}` as any
    }
  }
})

console.log('🤖 Agent server started on port 3738')

// Graceful shutdown
process.on('SIGINT', async () => {
  console.log('\nShutting down agent...')
  await client.close()
  process.exit(0)
})

Step 4: Create a Client

Create a file client.ts:

import { createNetworkClient } from '@hcengineering/network-client'

async function main() {
  // Connect to network
  const client = createNetworkClient('localhost:3737')
  await client.waitConnection(5000)
  console.log('✅ Connected to network')

  // Register the agent (if running separately, do this in agent.ts)
  // await client.register(agent)

  // Get a container
  const containerRef = await client.get('hello-world' as any, {})
  console.log(`📦 Got container: ${containerRef.uuid}`)

  // Send requests
  const greeting = await containerRef.request('greet', { name: 'Alice' })
  console.log('Response:', greeting)

  const status = await containerRef.request('status')
  console.log('Status:', status)

  // Cleanup
  await containerRef.close()
  await client.close()
  console.log('👋 Disconnected')
}

main().catch(console.error)

Step 5: Run Everything

Open three terminals:

Terminal 1: Start Network Server

npx ts-node server.ts

Terminal 2: Start Agent

npx ts-node agent.ts

Terminal 3: Run Client

npx ts-node client.ts

You should see:

✅ Connected to network
📦 Got container: hello-1234567890
Response: { message: 'Hello, Alice!' }
Status: { status: 'running', uuid: 'hello-1234567890' }
👋 Disconnected

All-in-One Example

For a single-file demo, create demo.ts:

import { NetworkImpl, TickManagerImpl } from '@hcengineering/network-core'
import { NetworkServer } from '@hcengineering/network-server'
import { createNetworkClient } from '@hcengineering/network-client'
import type { Container, ContainerUuid, ClientUuid, GetOptions } from '@hcengineering/network-core'

class DemoContainer implements Container {
  constructor(readonly uuid: ContainerUuid) {}

  async request(op: string, data?: any): Promise<any> {
    return { message: `Processed ${op}`, data }
  }

  async ping(): Promise<void> {}
  async terminate(): Promise<void> {}
  connect(clientId: ClientUuid, broadcast: (data: any) => Promise<void>): void {}
  disconnect(clientId: ClientUuid): void {}
}

async function demo() {
  // 1. Start network (NOTE: Only one instance allowed - no HA support)
  const tickManager = new TickManagerImpl(1)
  tickManager.start()
  const network = new NetworkImpl(tickManager)
  const server = new NetworkServer(network, tickManager, '*', 3737)
  console.log('✅ Network started')

  // 2. Connect client and serve agent
  const client = createNetworkClient('localhost:3737')
  await client.waitConnection(5000)

  await client.serveAgent('localhost:3738', {
    demo: async (options: GetOptions) => {
      const uuid = options.uuid ?? (`demo-${Date.now()}` as ContainerUuid)
      return {
        uuid,
        container: new DemoContainer(uuid),
        endpoint: `demo://localhost/${uuid}` as any
      }
    }
  })
  console.log('✅ Agent started')
  console.log('✅ Client connected')

  // 3. Use container
  const ref = await client.get('demo' as any, {})
  const result = await ref.request('test', { value: 42 })
  console.log('✅ Result:', result)

  // 5. Cleanup
  await ref.close()
  await client.close()
  await agentServer.close()
  await server.close()
  tickManager.stop()
  console.log('✅ Demo complete!')
}

demo().catch(console.error)

Run it:

npx ts-node demo.ts

Next Steps

Now that you have a working setup:

  1. Learn Core Concepts: Read Core Concepts to understand the architecture
  2. Build Containers: Explore Container Development Guide
  3. Try Examples: Run examples from the examples/ directory
  4. Add Features: Implement event broadcasting, multi-tenancy, or HA
  5. Deploy: Follow Production Deployment Guide

Quick References

Container Interface

interface Container {
  request(operation: string, data?: any, clientId?: ClientUuid): Promise<any>
  ping(): Promise<void>
  terminate(): Promise<void>
  connect(clientId: ClientUuid, broadcast: (data: any) => Promise<void>): void
  disconnect(clientId: ClientUuid): void
  onTerminated?(): void
}

Common Operations

Get a container:

const ref = await client.get('kind' as ContainerKind, {
  uuid: 'optional-uuid',
  labels: ['tag1', 'tag2']
})

Send a request:

const result = await ref.request('operation', { data: 'value' })

Connect for events:

const connection = await ref.connect()
connection.on = async (event) => {
  console.log('Event:', event)
}

Cleanup:

await connection.close()
await ref.close()
await client.close()

Troubleshooting

Can't connect to network:

  • Ensure the network server is running
  • Check the port (default: 3737)
  • Verify firewall settings

Container not found:

  • Register the agent before requesting containers
  • Check that the agent supports that container kind
  • Ensure the agent is still alive

Timeout errors:

  • Increase timeout: createNetworkClient('host:port', 3600)
  • Check network latency
  • Verify services are responding

For more help, see the Troubleshooting Guide.

Resources


Ready to build something amazing? Start with the Container Development Guide! 🚀