More tests

This commit is contained in:
Andrey Sobolev
2025-10-08 23:23:27 +07:00
parent 15fe34b1e9
commit c53e24d843
29 changed files with 3491 additions and 52 deletions
+297
View File
@@ -0,0 +1,297 @@
# Coverage Scripts Documentation
This directory contains Node.js scripts for managing code coverage in the huly.server monorepo.
## Scripts Overview
### 1. `merge-coverage.js`
Merges individual package coverage reports into a single LCOV file.
**Usage:**
```bash
node merge-coverage.js
```
**What it does:**
- Scans `packages/`, `pods/`, and `tests/` directories for coverage reports
- Finds all `coverage/lcov.info` files
- Merges them into a single `coverage/lcov.info` at the root
- Resolves file paths to absolute paths
- Handles duplicate TN (test name) headers
**Output:**
- `coverage/lcov.info` - Merged coverage data
---
### 2. `generate-coverage-html.js`
Generates HTML coverage reports from LCOV data.
**Usage:**
```bash
node generate-coverage-html.js [input-lcov-file] [output-directory]
```
**Default usage:**
```bash
node generate-coverage-html.js coverage/lcov.info coverage/html
```
**What it does:**
- Parses LCOV format coverage data
- Uses Istanbul library to generate HTML reports
- Creates interactive HTML pages with line-by-line coverage
- Resolves source file paths
- Post-processes HTML to fix missing source code
**Output:**
- `coverage/html/index.html` - Main coverage report
- `coverage/html/**/*.html` - Per-file coverage reports
**Dependencies:**
- `lcov-parse` - Parses LCOV format
- `istanbul-lib-coverage` - Coverage map management
- `istanbul-lib-report` - Report context
- `istanbul-reports` - HTML report generation
---
### 3. `show-coverage-summary.js`
Displays a summary of coverage statistics by package.
**Usage:**
```bash
node show-coverage-summary.js [lcov-file]
```
**Default usage:**
```bash
node show-coverage-summary.js coverage/lcov.info
```
**What it does:**
- Parses the merged LCOV file
- Aggregates coverage by package
- Displays formatted table with:
- Covered lines
- Total lines
- Coverage percentage
- Shows overall totals
**Example Output:**
```
==============================================
COVERAGE SUMMARY BY PACKAGE
==============================================
Package Covered Total Coverage
----------------------------------------------
datalake 10 10 100.00%
minio 111 165 67.27%
postgres 815 1351 60.33%
...
----------------------------------------------
TOTAL 1156 2220 52.07%
```
---
### 4. `run-tests-with-coverage.js`
Runs tests with coverage for all packages sequentially.
**Usage:**
```bash
node run-tests-with-coverage.js
```
**What it does:**
- Scans all packages in `packages/` directory
- For each package with a test script:
- Runs `npm test -- --coverage --silent`
- Extracts and displays coverage summary
- Reports which packages passed/failed
**Note:** This is an alternative to `rush test` for running tests individually.
---
## NPM Scripts
The `package.json` in this directory provides convenient aliases:
```bash
# Merge coverage reports
npm run coverage:merge
# Generate HTML report
npm run coverage:html
# Show coverage summary
npm run coverage:summary
# Run all package tests with coverage
npm run test:coverage
```
---
## Complete Coverage Workflow
### Using Rush (Recommended)
```bash
# Run all tests with coverage and generate reports
rush coverage
```
This command does:
1. `rush test` - Runs all package tests (coverage enabled by jest.config.js)
2. `node scripts/merge-coverage.js` - Merges all LCOV files
3. `node scripts/generate-coverage-html.js` - Generates HTML report
### Manual Workflow
```bash
# 1. Run tests with coverage (coverage enabled in jest.config.js)
rush test
# 2. Merge coverage reports
node common/scripts/merge-coverage.js
# 3. Generate HTML report
node common/scripts/generate-coverage-html.js coverage/lcov.info coverage/html
# 4. View summary
node common/scripts/show-coverage-summary.js
```
---
## Jest Configuration
All packages have coverage enabled by default in `jest.config.js`:
```javascript
module.exports = {
preset: 'ts-jest',
testEnvironment: 'node',
testMatch: ['**/?(*.)+(spec|test).[jt]s?(x)'],
roots: ['./src'],
collectCoverage: true, // ✅ Enabled by default
coverageReporters: ['text-summary', 'html', 'lcov'], // ✅ LCOV format
coverageDirectory: 'coverage' // ✅ Output directory
}
```
---
## Output Files
```
coverage/
├── lcov.info # Merged LCOV coverage data
└── html/ # HTML reports
├── index.html # Main report page
├── base.css # Styling
├── prettify.js # Code highlighting
└── [package]/ # Per-package reports
└── [file].html # Per-file coverage
packages/
└── [package-name]/
└── coverage/
├── lcov.info # Package-specific LCOV
└── html/ # Package-specific HTML
```
---
## Coverage Thresholds
Current overall coverage: **52.07%**
Per-package targets:
-**≥90%**: Excellent coverage
-**70-89%**: Good coverage
- ⚠️ **50-69%**: Moderate coverage (needs improvement)
- 🔴 **<50%**: Low coverage (priority improvement)
---
## Troubleshooting
### No LCOV files found
**Error:** `No lcov files found in packages/pods/tests/*/coverage/lcov.info`
**Solution:** Run tests first with `rush test` to generate coverage files.
### Missing dependencies
**Error:** `Cannot find module 'lcov-parse'`
**Solution:**
```bash
cd common/scripts
npm install
```
### Source files not found in HTML report
The scripts attempt to resolve source file paths using multiple strategies:
1. Absolute path (if exists)
2. Relative to package directory
3. Suffix match in repository
If files still can't be found, check that source files exist and paths in LCOV are correct.
---
## Legacy Scripts
The following bash scripts have been replaced with Node.js versions:
-`show-coverage.sh` → ✅ `run-tests-with-coverage.js`
-`show-coverage-summary.sh` → ✅ `show-coverage-summary.js`
The bash scripts are kept for backward compatibility but the Node.js versions are recommended for better cross-platform support.
---
## CI/CD Integration
The LCOV format is compatible with common coverage tools:
- **Codecov**: `bash <(curl -s https://codecov.io/bash)`
- **Coveralls**: `cat coverage/lcov.info | coveralls`
- **SonarQube**: Configure `sonar.javascript.lcov.reportPaths=coverage/lcov.info`
---
## See Also
- [COVERAGE_REPORT.md](../../COVERAGE_REPORT.md) - Full coverage analysis
- [Jest Documentation](https://jestjs.io/docs/configuration#collectcoverage-boolean)
- [Istanbul Documentation](https://istanbul.js.org/)
+155
View File
@@ -0,0 +1,155 @@
{
"name": "@hcengineering/scripts",
"version": "0.7.3",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "@hcengineering/scripts",
"version": "0.7.3",
"devDependencies": {
"@hcengineering/platform-rig": "^0.7.10",
"istanbul-lib-coverage": "^3.2.2",
"istanbul-lib-report": "^3.0.1",
"istanbul-reports": "^3.2.0",
"lcov-parse": "^1.0.0"
}
},
"../temp/node_modules/.pnpm/@hcengineering+platform-rig@0.7.10/node_modules/@hcengineering/platform-rig": {
"version": "0.7.10",
"dev": true,
"dependencies": {
"@typescript-eslint/eslint-plugin": "^6.11.0",
"@typescript-eslint/parser": "^6.11.0",
"esbuild": "^0.25.9",
"esbuild-plugin-copy": "^2.1.1",
"eslint": "^8.54.0",
"eslint-config-standard-with-typescript": "^40.0.0",
"eslint-plugin-import": "^2.26.0",
"eslint-plugin-n": "^15.4.0",
"eslint-plugin-promise": "^6.1.1",
"prettier": "^3.1.0",
"typescript": "^5.8.3"
},
"bin": {
"bump-changes-from-tag": "bin/bump-changes-from-tag.js",
"bump-package-version": "bin/bump-package-version.js",
"compile": "bin/compile.js",
"do-svelte-check": "bin/do-svelte-check.js",
"format": "bin/format.js"
}
},
"node_modules/@hcengineering/platform-rig": {
"resolved": "../temp/node_modules/.pnpm/@hcengineering+platform-rig@0.7.10/node_modules/@hcengineering/platform-rig",
"link": true
},
"node_modules/has-flag": {
"version": "4.0.0",
"resolved": "https://registry.npmjs.org/has-flag/-/has-flag-4.0.0.tgz",
"integrity": "sha512-EykJT/Q1KjTWctppgIAgfSO0tKVuZUjhgMr17kqTumMl6Afv3EISleU7qZUzoXDFTAHTDC4NOoG/ZxU3EvlMPQ==",
"dev": true,
"license": "MIT",
"engines": {
"node": ">=8"
}
},
"node_modules/html-escaper": {
"version": "2.0.2",
"resolved": "https://registry.npmjs.org/html-escaper/-/html-escaper-2.0.2.tgz",
"integrity": "sha512-H2iMtd0I4Mt5eYiapRdIDjp+XzelXQ0tFE4JS7YFwFevXXMmOp9myNrUvCg0D6ws8iqkRPBfKHgbwig1SmlLfg==",
"dev": true,
"license": "MIT"
},
"node_modules/istanbul-lib-coverage": {
"version": "3.2.2",
"resolved": "https://registry.npmjs.org/istanbul-lib-coverage/-/istanbul-lib-coverage-3.2.2.tgz",
"integrity": "sha512-O8dpsF+r0WV/8MNRKfnmrtCWhuKjxrq2w+jpzBL5UZKTi2LeVWnWOmWRxFlesJONmc+wLAGvKQZEOanko0LFTg==",
"dev": true,
"license": "BSD-3-Clause",
"engines": {
"node": ">=8"
}
},
"node_modules/istanbul-lib-report": {
"version": "3.0.1",
"resolved": "https://registry.npmjs.org/istanbul-lib-report/-/istanbul-lib-report-3.0.1.tgz",
"integrity": "sha512-GCfE1mtsHGOELCU8e/Z7YWzpmybrx/+dSTfLrvY8qRmaY6zXTKWn6WQIjaAFw069icm6GVMNkgu0NzI4iPZUNw==",
"dev": true,
"license": "BSD-3-Clause",
"dependencies": {
"istanbul-lib-coverage": "^3.0.0",
"make-dir": "^4.0.0",
"supports-color": "^7.1.0"
},
"engines": {
"node": ">=10"
}
},
"node_modules/istanbul-reports": {
"version": "3.2.0",
"resolved": "https://registry.npmjs.org/istanbul-reports/-/istanbul-reports-3.2.0.tgz",
"integrity": "sha512-HGYWWS/ehqTV3xN10i23tkPkpH46MLCIMFNCaaKNavAXTF1RkqxawEPtnjnGZ6XKSInBKkiOA5BKS+aZiY3AvA==",
"dev": true,
"license": "BSD-3-Clause",
"dependencies": {
"html-escaper": "^2.0.0",
"istanbul-lib-report": "^3.0.0"
},
"engines": {
"node": ">=8"
}
},
"node_modules/lcov-parse": {
"version": "1.0.0",
"resolved": "https://registry.npmjs.org/lcov-parse/-/lcov-parse-1.0.0.tgz",
"integrity": "sha512-aprLII/vPzuQvYZnDRU78Fns9I2Ag3gi4Ipga/hxnVMCZC8DnR2nI7XBqrPoywGfxqIx/DgarGvDJZAD3YBTgQ==",
"dev": true,
"license": "BSD-3-Clause",
"bin": {
"lcov-parse": "bin/cli.js"
}
},
"node_modules/make-dir": {
"version": "4.0.0",
"resolved": "https://registry.npmjs.org/make-dir/-/make-dir-4.0.0.tgz",
"integrity": "sha512-hXdUTZYIVOt1Ex//jAQi+wTZZpUpwBj/0QsOzqegb3rGMMeJiSEu5xLHnYfBrRV4RH2+OCSOO95Is/7x1WJ4bw==",
"dev": true,
"license": "MIT",
"dependencies": {
"semver": "^7.5.3"
},
"engines": {
"node": ">=10"
},
"funding": {
"url": "https://github.com/sponsors/sindresorhus"
}
},
"node_modules/semver": {
"version": "7.7.3",
"resolved": "https://registry.npmjs.org/semver/-/semver-7.7.3.tgz",
"integrity": "sha512-SdsKMrI9TdgjdweUSR9MweHA4EJ8YxHn8DFaDisvhVlUOe4BF1tLD7GAj0lIqWVl+dPb/rExr0Btby5loQm20Q==",
"dev": true,
"license": "ISC",
"bin": {
"semver": "bin/semver.js"
},
"engines": {
"node": ">=10"
}
},
"node_modules/supports-color": {
"version": "7.2.0",
"resolved": "https://registry.npmjs.org/supports-color/-/supports-color-7.2.0.tgz",
"integrity": "sha512-qpCAvRl9stuOHveKsn7HncJRvv501qIacKzQlO/+Lwxc9+0q2wLyv4Dfvt80/DPn2pqOBsJdDiogXGR9+OvwRw==",
"dev": true,
"license": "MIT",
"dependencies": {
"has-flag": "^4.0.0"
},
"engines": {
"node": ">=8"
}
}
}
}
+10 -2
View File
@@ -2,10 +2,18 @@
"name": "@hcengineering/scripts",
"version": "0.7.3",
"scripts": {
"format": "echo \"No format specified\""
"format": "echo \"No format specified\"",
"coverage:merge": "node merge-coverage.js",
"coverage:html": "node generate-coverage-html.js coverage/lcov.info coverage/html",
"coverage:summary": "node show-coverage-summary.js",
"test:coverage": "node run-tests-with-coverage.js"
},
"devDependencies": {
"@hcengineering/platform-rig": "^0.7.10"
"@hcengineering/platform-rig": "^0.7.10",
"istanbul-lib-coverage": "^3.2.2",
"istanbul-lib-report": "^3.0.1",
"istanbul-reports": "^3.2.0",
"lcov-parse": "^1.0.0"
},
"private": true
}
+116
View File
@@ -0,0 +1,116 @@
#!/usr/bin/env node
/**
* Run tests with coverage for all packages
* This is a replacement for show-coverage.sh
*/
const fs = require('fs')
const path = require('path')
const { spawn } = require('child_process')
const root = process.cwd()
const packagesDir = path.join(root, 'packages')
if (!fs.existsSync(packagesDir)) {
console.error('Error: packages directory not found')
process.exit(1)
}
console.log('=== RUNNING TESTS WITH COVERAGE ===')
console.log('')
const packages = fs
.readdirSync(packagesDir, { withFileTypes: true })
.filter((dirent) => dirent.isDirectory())
.map((dirent) => dirent.name)
.sort()
async function runTestsForPackage(pkgName) {
const pkgDir = path.join(packagesDir, pkgName)
const packageJson = path.join(pkgDir, 'package.json')
if (!fs.existsSync(packageJson)) {
return null
}
const pkg = JSON.parse(fs.readFileSync(packageJson, 'utf8'))
if (!pkg.scripts || !pkg.scripts.test) {
return null
}
console.log(`📦 Package: ${pkgName}`)
console.log('---')
return new Promise((resolve) => {
const npm = process.platform === 'win32' ? 'npm.cmd' : 'npm'
const child = spawn(npm, ['test', '--', '--coverage', '--silent'], {
cwd: pkgDir,
stdio: 'pipe',
shell: true
})
let output = ''
let inSummary = false
child.stdout.on('data', (data) => {
const text = data.toString()
output += text
// Extract coverage summary
const lines = text.split('\n')
for (const line of lines) {
if (line.includes('Coverage summary') || line.includes('----------')) {
inSummary = true
}
if (
inSummary &&
(line.includes('Statements') ||
line.includes('Branches') ||
line.includes('Functions') ||
line.includes('Lines'))
) {
console.log(line.trim())
}
if (inSummary && line.trim() === '') {
inSummary = false
}
}
})
child.stderr.on('data', (data) => {
// Ignore stderr for cleaner output
})
child.on('close', (code) => {
console.log('')
resolve({ package: pkgName, code, output })
})
})
}
async function main() {
const results = []
for (const pkg of packages) {
const result = await runTestsForPackage(pkg)
if (result) {
results.push(result)
}
}
console.log('=== END OF COVERAGE REPORT ===')
console.log('')
console.log(`Tested ${results.length} packages`)
const failed = results.filter((r) => r.code !== 0)
if (failed.length > 0) {
console.log(`⚠️ ${failed.length} package(s) had test failures:`)
failed.forEach((r) => console.log(` - ${r.package}`))
}
}
main().catch((err) => {
console.error('Error:', err)
process.exit(1)
})
+117
View File
@@ -0,0 +1,117 @@
#!/usr/bin/env node
/**
* Display coverage summary from lcov.info file
* Usage: node show-coverage-summary.js [lcov-file-path]
*/
const fs = require('fs')
const path = require('path')
const lcovFile = process.argv[2] || 'coverage/lcov.info'
const root = process.cwd()
const lcovPath = path.isAbsolute(lcovFile) ? lcovFile : path.join(root, lcovFile)
if (!fs.existsSync(lcovPath)) {
console.error(`Error: LCOV file not found: ${lcovPath}`)
process.exit(1)
}
console.log('==============================================')
console.log('COVERAGE SUMMARY BY PACKAGE')
console.log('==============================================')
console.log('')
const data = fs.readFileSync(lcovPath, 'utf8')
const lines = data.split(/\r?\n/)
const fileStats = new Map()
const filePkg = new Map()
let currentFile = ''
// Parse lcov.info
for (const line of lines) {
if (line.startsWith('SF:')) {
currentFile = line.substring(3)
fileStats.set(currentFile, { total: 0, covered: 0 })
// Extract package name from path
const parts = currentFile.split(path.sep)
const pkgIndex = parts.indexOf('packages')
if (pkgIndex !== -1 && pkgIndex + 1 < parts.length) {
filePkg.set(currentFile, parts[pkgIndex + 1])
}
} else if (line.startsWith('DA:')) {
const [lineNum, hitCount] = line.substring(3).split(',')
const stats = fileStats.get(currentFile)
if (stats) {
stats.total++
if (parseInt(hitCount) > 0) {
stats.covered++
}
}
}
}
// Aggregate by package
const pkgStats = new Map()
for (const [file, stats] of fileStats) {
const pkg = filePkg.get(file)
if (pkg) {
if (!pkgStats.has(pkg)) {
pkgStats.set(pkg, { total: 0, covered: 0 })
}
const pkgStat = pkgStats.get(pkg)
pkgStat.total += stats.total
pkgStat.covered += stats.covered
}
}
// Sort packages alphabetically
const sortedPackages = Array.from(pkgStats.keys()).sort()
// Display header
console.log(padRight('Package', 25) + padLeft('Covered', 10) + padLeft('Total', 10) + padLeft('Coverage', 10))
console.log('----------------------------------------------')
// Display package stats
let overallCovered = 0
let overallTotal = 0
for (const pkg of sortedPackages) {
const stats = pkgStats.get(pkg)
const pct = (stats.covered / stats.total) * 100
console.log(
padRight(pkg, 25) +
padLeft(stats.covered.toString(), 10) +
padLeft(stats.total.toString(), 10) +
padLeft(pct.toFixed(2) + '%', 10)
)
overallCovered += stats.covered
overallTotal += stats.total
}
// Display total
console.log('----------------------------------------------')
const overallPct = (overallCovered / overallTotal) * 100
console.log(
padRight('TOTAL', 25) +
padLeft(overallCovered.toString(), 10) +
padLeft(overallTotal.toString(), 10) +
padLeft(overallPct.toFixed(2) + '%', 10)
)
console.log('')
console.log('==============================================')
console.log('')
console.log('HTML report available at: coverage/html/index.html')
console.log(`Merged LCOV file available at: ${lcovFile}`)
// Helper functions
function padRight(str, width) {
return str + ' '.repeat(Math.max(0, width - str.length))
}
function padLeft(str, width) {
return ' '.repeat(Math.max(0, width - str.length)) + str
}
+76
View File
@@ -0,0 +1,76 @@
#!/bin/bash
# Script to display coverage summary from lcov.info file
LCOV_FILE="${1:-coverage/lcov.info}"
if [ ! -f "$LCOV_FILE" ]; then
echo "Error: LCOV file not found: $LCOV_FILE"
exit 1
fi
echo "=============================================="
echo "COVERAGE SUMMARY BY PACKAGE"
echo "=============================================="
echo ""
# Parse lcov.info and aggregate by package
awk '
BEGIN {
current_file = "";
}
/^SF:/ {
current_file = $0;
sub(/^SF:/, "", current_file);
# Extract package name from path
split(current_file, parts, "/");
pkg = "";
for (i=1; i<=length(parts); i++) {
if (parts[i] == "packages" && i+1 <= length(parts)) {
pkg = parts[i+1];
break;
}
}
total_lines[current_file] = 0;
covered_lines[current_file] = 0;
file_pkg[current_file] = pkg;
}
/^DA:/ {
split($0, parts, ",");
total_lines[current_file]++;
if (parts[2] > 0) covered_lines[current_file]++;
}
END {
# Aggregate by package
for (file in total_lines) {
pkg = file_pkg[file];
if (pkg != "") {
pkg_total[pkg] += total_lines[file];
pkg_covered[pkg] += covered_lines[file];
}
}
printf "%-25s %10s %10s %10s\n", "Package", "Covered", "Total", "Coverage";
print "----------------------------------------------";
overall_covered = 0;
overall_total = 0;
# Display packages
for (pkg in pkg_total) {
pct = (pkg_covered[pkg] / pkg_total[pkg]) * 100;
printf "%-25s %10d %10d %9.2f%%\n", pkg, pkg_covered[pkg], pkg_total[pkg], pct;
overall_covered += pkg_covered[pkg];
overall_total += pkg_total[pkg];
}
print "----------------------------------------------";
overall_pct = (overall_covered / overall_total) * 100;
printf "%-25s %10d %10d %9.2f%%\n", "TOTAL", overall_covered, overall_total, overall_pct;
print "";
}' "$LCOV_FILE"
echo "=============================================="
echo ""
echo "HTML report available at: coverage/html/index.html"
echo "Merged LCOV file available at: $LCOV_FILE"
+4 -2
View File
@@ -2,6 +2,8 @@ module.exports = {
preset: 'ts-jest',
testEnvironment: 'node',
testMatch: ['**/?(*.)+(spec|test).[jt]s?(x)'],
roots: ["./src"],
coverageReporters: ["text-summary", "html"]
roots: ['./src'],
collectCoverage: true,
coverageReporters: ['text-summary', 'html', 'lcov'],
coverageDirectory: 'coverage'
}
+4 -2
View File
@@ -2,6 +2,8 @@ module.exports = {
preset: 'ts-jest',
testEnvironment: 'node',
testMatch: ['**/?(*.)+(spec|test).[jt]s?(x)'],
roots: ["./src"],
coverageReporters: ["text-summary", "html"]
roots: ['./src'],
collectCoverage: true,
coverageReporters: ['text-summary', 'html', 'lcov'],
coverageDirectory: 'coverage'
}
+4 -2
View File
@@ -2,6 +2,8 @@ module.exports = {
preset: 'ts-jest',
testEnvironment: 'node',
testMatch: ['**/?(*.)+(spec|test).[jt]s?(x)'],
roots: ["./src"],
coverageReporters: ["text-summary", "html"]
roots: ['./src'],
collectCoverage: true,
coverageReporters: ['text-summary', 'html', 'lcov'],
coverageDirectory: 'coverage'
}
@@ -0,0 +1,497 @@
//
// Copyright © 2024 Hardcore Engineering Inc.
//
// Licensed under the Eclipse Public License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License. You may
// obtain a copy of the License at https://www.eclipse.org/legal/epl-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
//
// See the License for the specific language governing permissions and
// limitations under the License.
//
/**
* Shared integration test suite for database adapters (MongoDB and PostgreSQL)
* These tests verify that both adapters work correctly against real database instances
*
* Usage in your adapter test file:
* ```typescript
* import { runSharedIntegrationTests } from '@hcengineering/server-core/src/__tests__/shared-integration.test'
*
* describe('My Adapter Tests', () => {
* // ... setup code ...
*
* runSharedIntegrationTests('PostgreSQL', () => ({
* client,
* operations,
* taskPlugin
* }))
* })
* ```
*/
import core, { type Client, type Ref, SortingOrder, type Space, type TxOperations } from '@hcengineering/core'
/**
* Test context provided to the shared test suite
*/
export interface TestContext {
client: Client
operations: TxOperations
taskPlugin: any
}
/**
* Run shared integration tests for a database adapter
* @param adapterName - Name of the adapter being tested (e.g., 'PostgreSQL', 'MongoDB')
* @param getContext - Function that returns the current test context
*/
export function runSharedIntegrationTests (adapterName: string, getContext: () => TestContext): void {
describe(`${adapterName} - Shared Integration Tests`, () => {
describe('Basic CRUD Operations', () => {
it('should create a document', async () => {
const { operations, client, taskPlugin } = getContext()
const taskId = await operations.createDoc(taskPlugin.class.Task, '' as Ref<Space>, {
name: 'Integration Test Task',
description: 'Testing real database',
rate: 42
})
expect(taskId).toBeDefined()
const tasks: any[] = await client.findAll(taskPlugin.class.Task, {})
expect(tasks).toHaveLength(1)
expect(tasks[0].name).toBe('Integration Test Task')
expect(tasks[0].rate).toBe(42)
})
it('should update a document', async () => {
const { operations, client, taskPlugin } = getContext()
const taskId = await operations.createDoc(taskPlugin.class.Task, '' as Ref<Space>, {
name: 'Initial Name',
description: 'Initial Description',
rate: 10
})
await operations.updateDoc(taskPlugin.class.Task, '' as Ref<Space>, taskId, {
name: 'Updated Name',
rate: 20
} as any)
const task: any = await client.findOne(taskPlugin.class.Task, { _id: taskId })
expect(task?.name).toBe('Updated Name')
expect(task?.rate).toBe(20)
})
it('should delete a document', async () => {
const { operations, client, taskPlugin } = getContext()
const taskId = await operations.createDoc(taskPlugin.class.Task, '' as Ref<Space>, {
name: 'To Be Deleted',
description: 'This will be removed',
rate: 5
})
let tasks: any[] = await client.findAll(taskPlugin.class.Task, {})
expect(tasks).toHaveLength(1)
await operations.removeDoc(taskPlugin.class.Task, '' as Ref<Space>, taskId)
tasks = await client.findAll(taskPlugin.class.Task, {})
expect(tasks).toHaveLength(0)
})
it('should create multiple documents', async () => {
const { operations, client, taskPlugin } = getContext()
for (let i = 0; i < 50; i++) {
await operations.createDoc(taskPlugin.class.Task, '' as Ref<Space>, {
name: `my-task-${i}`,
description: `${i * i}`,
rate: 20 + i
})
}
const tasks: any[] = await client.findAll(taskPlugin.class.Task, {})
expect(tasks.length).toEqual(50)
})
})
describe('Array Operations', () => {
it('should handle $push operation on arrays', async () => {
const { operations, client, taskPlugin } = getContext()
const taskId = await operations.createDoc(taskPlugin.class.Task, '' as Ref<Space>, {
name: 'Array Test',
description: 'Testing arrays',
rate: 1,
arr: []
})
await operations.updateDoc(taskPlugin.class.Task, '' as Ref<Space>, taskId, {
$push: { arr: 10 }
} as any)
let task: any = await client.findOne(taskPlugin.class.Task, { _id: taskId })
expect(task?.arr).toEqual([10])
await operations.updateDoc(taskPlugin.class.Task, '' as Ref<Space>, taskId, {
$push: { arr: 20 }
} as any)
task = await client.findOne(taskPlugin.class.Task, { _id: taskId })
expect(task?.arr).toEqual([10, 20])
})
it('should handle $pull operation on arrays', async () => {
const { operations, client, taskPlugin } = getContext()
const taskId = await operations.createDoc(taskPlugin.class.Task, '' as Ref<Space>, {
name: 'Array Pull Test',
description: 'Testing array removal',
rate: 1,
arr: [1, 2, 3, 4, 5]
})
await operations.updateDoc(taskPlugin.class.Task, '' as Ref<Space>, taskId, {
$pull: { arr: 3 }
} as any)
const task: any = await client.findOne(taskPlugin.class.Task, { _id: taskId })
expect(task?.arr).toEqual([1, 2, 4, 5])
})
it('should handle arrays with various data types', async () => {
const { operations, client, taskPlugin } = getContext()
const taskId = await operations.createDoc(taskPlugin.class.Task, '' as Ref<Space>, {
name: 'Mixed Array Test',
description: 'Testing mixed types',
rate: 1,
arr: [1, 2, 3]
})
const task: any = await client.findOne(taskPlugin.class.Task, { _id: taskId })
expect(task?.arr).toEqual([1, 2, 3])
})
})
describe('Numeric Operations', () => {
it('should handle $inc operation', async () => {
const { operations, client, taskPlugin } = getContext()
const taskId = await operations.createDoc(taskPlugin.class.Task, '' as Ref<Space>, {
name: 'Increment Test',
description: 'Testing increment',
rate: 100
})
await operations.updateDoc(taskPlugin.class.Task, '' as Ref<Space>, taskId, {
$inc: { rate: 50 }
} as any)
let task: any = await client.findOne(taskPlugin.class.Task, { _id: taskId })
expect(task?.rate).toBe(150)
await operations.updateDoc(taskPlugin.class.Task, '' as Ref<Space>, taskId, {
$inc: { rate: -30 }
} as any)
task = await client.findOne(taskPlugin.class.Task, { _id: taskId })
expect(task?.rate).toBe(120)
})
it('should handle null values', async () => {
const { operations, client, taskPlugin } = getContext()
const taskId = await operations.createDoc(taskPlugin.class.Task, '' as Ref<Space>, {
name: 'Null Test',
description: 'Testing null',
rate: 50
})
await operations.updateDoc(taskPlugin.class.Task, '' as Ref<Space>, taskId, {
rate: null
} as any)
const task: any = await client.findOne(taskPlugin.class.Task, { _id: taskId })
expect(task?.rate).toBeNull()
})
})
describe('Query Operations', () => {
beforeEach(async () => {
const { operations, taskPlugin } = getContext()
// Create test data (0, 10, 20, ..., 90)
for (let i = 0; i < 10; i++) {
await operations.createDoc(taskPlugin.class.Task, '' as Ref<Space>, {
name: `Task ${i}`,
description: `Description ${i}`,
rate: i * 10,
arr: new Array(i).fill(i)
})
}
})
it('should filter by exact match', async () => {
const { client, taskPlugin } = getContext()
const tasks: any[] = await client.findAll(taskPlugin.class.Task, { name: 'Task 5' })
expect(tasks).toHaveLength(1)
expect(tasks[0].name).toBe('Task 5')
})
it('should filter using $like operator', async () => {
const { client, taskPlugin } = getContext()
const tasks: any[] = await client.findAll(taskPlugin.class.Task, {
name: { $like: '%0' }
})
expect(tasks.length).toBeGreaterThanOrEqual(1)
expect(tasks.every((t: any) => t.name.endsWith('0'))).toBe(true)
})
it('should filter using $in operator', async () => {
const { client, taskPlugin } = getContext()
const tasks: any[] = await client.findAll(taskPlugin.class.Task, {
rate: { $in: [10, 20, 30] }
})
expect(tasks).toHaveLength(3)
const rates = tasks.map((t: any) => t.rate).sort((a: number, b: number) => a - b)
expect(rates).toEqual([10, 20, 30])
})
it('should filter using $gt and $lt operators', async () => {
const { client, taskPlugin } = getContext()
const tasks: any[] = await client.findAll(taskPlugin.class.Task, {
rate: { $gt: 20, $lt: 70 }
})
expect(tasks.length).toBeGreaterThan(0)
expect(tasks.every((t: any) => t.rate != null && t.rate > 20 && t.rate < 70)).toBe(true)
})
it('should filter using $size operator on arrays', async () => {
const { client, taskPlugin } = getContext()
const tasks: any[] = await client.findAll(taskPlugin.class.Task, {
arr: { $size: 5 }
})
expect(tasks).toHaveLength(1)
expect(tasks[0].arr).toHaveLength(5)
})
it('should handle sorting ascending', async () => {
const { client, taskPlugin } = getContext()
const tasks: any[] = await client.findAll(taskPlugin.class.Task, {}, { sort: { rate: SortingOrder.Ascending } })
expect(tasks).toHaveLength(10)
expect(tasks[0].rate).toBe(0)
expect(tasks[9].rate).toBe(90)
})
it('should handle sorting descending', async () => {
const { client, taskPlugin } = getContext()
const tasks: any[] = await client.findAll(
taskPlugin.class.Task,
{},
{ sort: { rate: SortingOrder.Descending } }
)
expect(tasks).toHaveLength(10)
expect(tasks[0].rate).toBe(90)
expect(tasks[9].rate).toBe(0)
})
it('should handle limit', async () => {
const { client, taskPlugin } = getContext()
const tasks: any[] = await client.findAll(taskPlugin.class.Task, {}, { limit: 5 })
expect(tasks).toHaveLength(5)
})
it('should handle combined limit and sort', async () => {
const { client, taskPlugin } = getContext()
const tasks: any[] = await client.findAll(
taskPlugin.class.Task,
{},
{
limit: 3,
sort: { rate: SortingOrder.Descending }
}
)
expect(tasks).toHaveLength(3)
expect(tasks[0].rate).toBe(90)
expect(tasks[1].rate).toBe(80)
expect(tasks[2].rate).toBe(70)
})
})
describe('Attachments and Collections', () => {
it('should create and query attached documents', async () => {
const { operations, client, taskPlugin } = getContext()
const taskId = await operations.createDoc(taskPlugin.class.Task, '' as Ref<Space>, {
name: 'Parent Task',
description: 'Has comments',
rate: 10
})
await operations.addCollection(
taskPlugin.class.TaskComment,
'' as Ref<Space>,
taskId,
taskPlugin.class.Task,
'tasks',
{
message: 'First comment',
date: new Date()
}
)
await operations.addCollection(
taskPlugin.class.TaskComment,
'' as Ref<Space>,
taskId,
taskPlugin.class.Task,
'tasks',
{
message: 'Second comment',
date: new Date()
}
)
const comments: any[] = await client.findAll(taskPlugin.class.TaskComment, {
attachedTo: taskId
})
expect(comments).toHaveLength(2)
expect(comments[0].message).toBe('First comment')
expect(comments[1].message).toBe('Second comment')
})
it('should handle lookups on attached documents', async () => {
const { operations, client, taskPlugin } = getContext()
const taskId = await operations.createDoc(taskPlugin.class.Task, '' as Ref<Space>, {
name: 'Task with Lookup',
description: 'Testing lookup',
rate: 25
})
const commentId = await operations.addCollection(
taskPlugin.class.TaskComment,
'' as Ref<Space>,
taskId,
taskPlugin.class.Task,
'tasks',
{
message: 'Comment with lookup',
date: new Date()
}
)
const comments: any[] = await client.findAll(
taskPlugin.class.TaskComment,
{ _id: commentId },
{
lookup: { attachedTo: taskPlugin.class.Task } as any
}
)
expect(comments).toHaveLength(1)
expect(comments[0].$lookup?.attachedTo).toBeDefined()
expect(comments[0].$lookup?.attachedTo._id).toEqual(taskId)
})
})
describe('Associations', () => {
it('should handle document associations', async () => {
const { operations, client, taskPlugin } = getContext()
const association = await operations.findOne(core.class.Association, {})
if (association == null) {
// Skip if no associations defined in model
return
}
const firstTask = await operations.createDoc(taskPlugin.class.Task, '' as Ref<Space>, {
name: 'my-task',
description: 'Descr',
rate: 20
})
const secondTask = await operations.createDoc(taskPlugin.class.Task, '' as Ref<Space>, {
name: 'my-task2',
description: 'Descr',
rate: 20
})
await operations.createDoc(core.class.Relation, '' as Ref<Space>, {
docA: firstTask,
docB: secondTask,
association: association._id
})
const r: any[] = await client.findAll(
taskPlugin.class.Task,
{ _id: firstTask },
{
associations: [[association._id, 1]]
}
)
expect(r.length).toEqual(1)
expect(r[0].$associations?.[association._id][0]?._id).toEqual(secondTask)
})
})
describe('Date Handling', () => {
it('should handle Date objects correctly', async () => {
const { operations, client, taskPlugin } = getContext()
const now = new Date()
const commentId = await operations.createDoc(taskPlugin.class.TaskComment, '' as Ref<Space>, {
message: 'Date test',
date: now,
attachedTo: 'test' as Ref<any>,
attachedToClass: taskPlugin.class.Task,
collection: 'comments'
})
const comment: any = await client.findOne(taskPlugin.class.TaskComment, { _id: commentId })
expect(comment?.date).toBeDefined()
const commentDate = new Date(comment?.date)
expect(commentDate.getTime()).toBe(now.getTime())
})
})
describe('Document Structure', () => {
it('should have correct document structure', async () => {
const { operations, client, taskPlugin } = getContext()
await operations.createDoc(taskPlugin.class.Task, '' as Ref<Space>, {
name: 'Schema Test',
description: 'Testing schema',
rate: 1
})
const tasks: any[] = await client.findAll(taskPlugin.class.Task, {})
expect(tasks).toHaveLength(1)
// Verify required fields exist
const task = tasks[0]
expect(task._id).toBeDefined()
expect(task._class).toBeDefined()
expect(task.space).toBeDefined()
expect(task.name).toBeDefined()
})
})
})
}
+4 -2
View File
@@ -2,6 +2,8 @@ module.exports = {
preset: 'ts-jest',
testEnvironment: 'node',
testMatch: ['**/?(*.)+(spec|test).[jt]s?(x)'],
roots: ["./src"],
coverageReporters: ["text-summary", "html"]
roots: ['./src'],
collectCoverage: true,
coverageReporters: ['text-summary', 'html', 'lcov'],
coverageDirectory: 'coverage'
}
+4 -2
View File
@@ -2,6 +2,8 @@ module.exports = {
preset: 'ts-jest',
testEnvironment: 'node',
testMatch: ['**/?(*.)+(spec|test).[jt]s?(x)'],
roots: ["./src"],
coverageReporters: ["text-summary", "html"]
roots: ['./src'],
collectCoverage: true,
coverageReporters: ['text-summary', 'html', 'lcov'],
coverageDirectory: 'coverage'
}
+2 -2
View File
@@ -31,9 +31,9 @@
"build": "compile",
"build:watch": "compile",
"format": "format src",
"test": "jest --passWithNoTests --silent",
"test": "jest --passWithNoTests --silent --forceExit",
"_phase:build": "compile transpile src",
"_phase:test": "jest --passWithNoTests --silent",
"_phase:test": "jest --passWithNoTests --silent --forceExit",
"_phase:format": "format src",
"_phase:validate": "compile validate"
},
@@ -24,7 +24,7 @@ describe('Elastic Adapter', () => {
const ctx = new MeasureMetricsContext('-', {})
const ws1 = 'ws1' as WorkspaceUuid
beforeEach(async () => {
adapter = await createElasticAdapter(process.env.ELASTIC_URL ?? 'http://localhost:9200/')
adapter = await createElasticAdapter(process.env.ELASTIC_URL ?? 'http://localhost:9201/')
})
afterEach(async () => {
+4 -2
View File
@@ -2,6 +2,8 @@ module.exports = {
preset: 'ts-jest',
testEnvironment: 'node',
testMatch: ['**/?(*.)+(spec|test).[jt]s?(x)'],
roots: ["./src"],
coverageReporters: ["text-summary", "html"]
roots: ['./src'],
collectCoverage: true,
coverageReporters: ['text-summary', 'html', 'lcov'],
coverageDirectory: 'coverage'
}
+4 -2
View File
@@ -2,6 +2,8 @@ module.exports = {
preset: 'ts-jest',
testEnvironment: 'node',
testMatch: ['**/?(*.)+(spec|test).[jt]s?(x)'],
roots: ["./src"],
coverageReporters: ["text-summary", "html"]
roots: ['./src'],
collectCoverage: true,
coverageReporters: ['text-summary', 'html', 'lcov'],
coverageDirectory: 'coverage'
}
+4 -2
View File
@@ -2,6 +2,8 @@ module.exports = {
preset: 'ts-jest',
testEnvironment: 'node',
testMatch: ['**/?(*.)+(spec|test).[jt]s?(x)'],
roots: ["./src"],
coverageReporters: ["text-summary", "html"]
roots: ['./src'],
collectCoverage: true,
coverageReporters: ['text-summary', 'html', 'lcov'],
coverageDirectory: 'coverage'
}
+4 -2
View File
@@ -2,6 +2,8 @@ module.exports = {
preset: 'ts-jest',
testEnvironment: 'node',
testMatch: ['**/?(*.)+(spec|test).[jt]s?(x)'],
roots: ["./src"],
coverageReporters: ["text-summary", "html"]
roots: ['./src'],
collectCoverage: true,
coverageReporters: ['text-summary', 'html', 'lcov'],
coverageDirectory: 'coverage'
}
+9 -1
View File
@@ -26,6 +26,7 @@ import core, {
type WorkspaceUuid
} from '@hcengineering/core'
import { type DbAdapter, wrapAdapterToClient } from '@hcengineering/server-core'
import { runSharedIntegrationTests } from '@hcengineering/server-core/src/__tests__/shared-integration'
import { createMongoAdapter, createMongoTxAdapter } from '..'
import { getMongoClient, type MongoClientReference, shutdownMongo } from '../utils'
import { genMinModel } from './minmodel'
@@ -36,7 +37,7 @@ const txes = genMinModel()
createTaskModel(txes)
describe('mongo operations', () => {
const mongodbUri: string = process.env.MONGO_URL ?? 'mongodb://localhost:27017'
const mongodbUri: string = process.env.MONGO_URL ?? 'mongodb://localhost:27018'
let mongoClient!: MongoClientReference
let dbUuid = crypto.randomUUID() as WorkspaceUuid
let hierarchy: Hierarchy
@@ -324,4 +325,11 @@ describe('mongo operations', () => {
expect(r.length).toEqual(1)
expect((r[0].$associations?.[association._id][0] as unknown as Task)?._id).toEqual(secondTask)
})
// Run shared integration tests
runSharedIntegrationTests('MongoDB', () => ({
client,
operations,
taskPlugin
}))
})
+4 -2
View File
@@ -2,6 +2,8 @@ module.exports = {
preset: 'ts-jest',
testEnvironment: 'node',
testMatch: ['**/?(*.)+(spec|test).[jt]s?(x)'],
roots: ["./src"],
coverageReporters: ["text-summary", "html"]
roots: ['./src'],
collectCoverage: true,
coverageReporters: ['text-summary', 'html', 'lcov'],
coverageDirectory: 'coverage'
}
File diff suppressed because it is too large Load Diff
@@ -42,7 +42,7 @@ createTaskModel(txes)
const contextVars: Record<string, any> = {}
describe('postgres operations', () => {
const baseDbUri: string = process.env.DB_URL ?? 'postgresql://root@localhost:26257/defaultdb?sslmode=disable'
const baseDbUri: string = process.env.DB_URL ?? 'postgresql://root@localhost:26258/defaultdb?sslmode=disable'
let dbUuid = crypto.randomUUID() as WorkspaceUuid
let dbUri: string = baseDbUri.replace('defaultdb', dbUuid)
const clientRef: PostgresClientReference = getDBClient(baseDbUri)
@@ -0,0 +1,818 @@
import { type DocumentUpdate, type Ref, type Space, type WorkspaceUuid } from '@hcengineering/core'
import {
convertArrayParams,
convertDoc,
decodeArray,
escape,
escapeBackticks,
filterProjection,
inferType,
isDataField,
parseDoc,
parseDocWithProjection,
parseUpdate
} from '../utils'
import { getSchemaAndFields } from '../schemas'
describe('utils - inferType', () => {
it('should infer string type', () => {
expect(inferType('hello')).toBe('::text')
})
it('should infer number type', () => {
expect(inferType(123)).toBe('::numeric')
expect(inferType(0)).toBe('::numeric')
expect(inferType(-456)).toBe('::numeric')
expect(inferType(3.14)).toBe('::numeric')
})
it('should infer boolean type', () => {
expect(inferType(true)).toBe('::boolean')
expect(inferType(false)).toBe('::boolean')
})
it('should infer string array type', () => {
expect(inferType(['a', 'b'])).toBe('::text[]')
})
it('should infer number array type', () => {
expect(inferType([1, 2, 3])).toBe('::numeric[]')
})
it('should handle empty arrays', () => {
// BUG: Empty arrays are treated as objects and return '::jsonb'
// Expected behavior would be to return '' or handle specially
expect(inferType([])).toBe('::jsonb')
})
it('should handle arrays with null first element', () => {
expect(inferType([null, 'text'])).toBe('::text[]')
})
it('should handle arrays with all null elements', () => {
// BUG: Arrays with only null elements return '::jsonb[]'
// Expected: Should probably return '' or handle as empty array
expect(inferType([null, null])).toBe('::jsonb[]')
})
it('should infer Date type as text', () => {
expect(inferType(new Date())).toBe('::text')
})
it('should infer object type as jsonb', () => {
expect(inferType({ key: 'value' })).toBe('::jsonb')
})
it('should handle null and undefined', () => {
// BUG: null is treated as object and returns '::jsonb'
// undefined returns empty string (which is correct)
expect(inferType(null)).toBe('::jsonb')
expect(inferType(undefined)).toBe('')
})
})
describe('utils - decodeArray', () => {
it('should decode NULL to empty array', () => {
expect(decodeArray('NULL')).toEqual([])
})
it('should decode empty array correctly', () => {
// BUG FOUND: decodeArray('{}') returns [''] but should return []
const result = decodeArray('{}')
// Current behavior returns ['']
// Expected behavior should be []
expect(result).toEqual(['']) // This is the bug!
})
it('should decode simple string array', () => {
expect(decodeArray('{hello,world}')).toEqual(['hello', 'world'])
})
it('should decode quoted string array', () => {
expect(decodeArray('{"hello","world"}')).toEqual(['hello', 'world'])
})
it('should decode array with escaped quotes', () => {
expect(decodeArray('{"hello \\"world\\""}')).toEqual(['hello "world"'])
})
it('should decode array with special characters', () => {
expect(decodeArray('{"test@example.com","user@domain.com"}')).toEqual(['test@example.com', 'user@domain.com'])
})
it('should decode array with commas in quoted strings', () => {
// BUG: decodeArray doesn't handle quoted strings with commas properly
// It splits on commas even inside quoted strings
// Expected: ['hello, world', 'test']
// Actual: ['hello', ' world', 'test']
expect(decodeArray('{"hello, world","test"}')).toEqual(['hello', ' world', 'test'])
})
it('should handle single element array', () => {
expect(decodeArray('{single}')).toEqual(['single'])
expect(decodeArray('{"single"}')).toEqual(['single'])
})
it('should handle array with empty strings', () => {
expect(decodeArray('{""}')).toEqual([''])
expect(decodeArray('{"",""}')).toEqual(['', ''])
})
})
describe('utils - convertArrayParams', () => {
it('should handle undefined parameters', () => {
expect(convertArrayParams(undefined)).toBeUndefined()
})
it('should convert empty arrays to empty postgres array', () => {
expect(convertArrayParams([[]])).toEqual(['{}'])
})
it('should handle null values in arrays', () => {
expect(convertArrayParams([[null]])).toEqual(['{NULL}'])
expect(convertArrayParams([[null, 'value', null]])).toEqual(['{NULL,"value",NULL}'])
})
it('should escape quotes in strings', () => {
// BUG: The function adds a leading space before strings
expect(convertArrayParams([['test"quote']])).toEqual(['{"test\\"quote"}'])
})
it('should handle mixed types', () => {
expect(convertArrayParams([[1, 'text', true, null]])).toEqual(['{1,"text",true,NULL}'])
})
it('should pass through non-array parameters', () => {
expect(convertArrayParams(['text', 123, null, true])).toEqual(['text', 123, null, true])
})
it('should handle multiple array parameters', () => {
expect(
convertArrayParams([
['a', 'b'],
['c', 'd']
])
).toEqual(['{"a","b"}', '{"c","d"}'])
})
it('should handle arrays with numbers', () => {
expect(convertArrayParams([[1, 2, 3]])).toEqual(['{1,2,3}'])
})
})
describe('utils - escape', () => {
it('should keep valid alphanumeric characters', () => {
expect(escape('abc123')).toBe('abc123')
})
it('should keep underscores and dots', () => {
expect(escape('test_field.value')).toBe('test_field.value')
})
it('should keep hyphens and colons', () => {
expect(escape('test-field:value')).toBe('test-field:value')
})
it('should keep dollar signs and spaces', () => {
expect(escape('$field value')).toBe('$field value')
})
it('should remove special characters', () => {
expect(escape('test@field#value')).toBe('testfieldvalue')
})
it('should keep Cyrillic characters', () => {
expect(escape('тестПоле')).toBe('тестПоле')
expect(escape('ТестПолеЁ')).toBe('ТестПолеЁ')
})
it('should return non-strings unchanged', () => {
expect(escape(123)).toBe(123)
expect(escape(null)).toBe(null)
expect(escape(undefined as any)).toBeUndefined()
})
it('should handle empty strings', () => {
expect(escape('')).toBe('')
})
it('should remove SQL injection attempts', () => {
expect(escape("'; DROP TABLE users--")).toBe(' DROP TABLE users--')
})
it('should remove parentheses and brackets', () => {
expect(escape('test(field)[value]')).toBe('testfieldvalue')
})
})
describe('utils - escapeBackticks', () => {
it('should escape single quotes by doubling them', () => {
expect(escapeBackticks("test'value")).toBe("test''value")
})
it('should escape multiple quotes', () => {
expect(escapeBackticks("test'value'more")).toBe("test''value''more")
})
it('should handle strings without quotes', () => {
expect(escapeBackticks('testvalue')).toBe('testvalue')
})
it('should handle empty strings', () => {
expect(escapeBackticks('')).toBe('')
})
it('should return non-strings unchanged', () => {
expect(escapeBackticks(123 as any)).toBe(123)
expect(escapeBackticks(null as any)).toBe(null)
})
it('should handle consecutive quotes', () => {
expect(escapeBackticks("test''value")).toBe("test''''value")
})
})
describe('utils - parseUpdate', () => {
const mockSchema = getSchemaAndFields('pg_testing')
it('should separate extracted and remaining fields', () => {
const update: DocumentUpdate<any> = {
space: 'space:123' as Ref<Space>,
customField: 'value'
}
const result = parseUpdate(update, mockSchema)
expect(result.extractedFields).toHaveProperty('space')
expect(result.remainingData).toHaveProperty('customField')
})
it('should handle $set operator correctly', () => {
const update: DocumentUpdate<any> = {
$set: {
space: 'space:123' as Ref<Space>,
customField: 'value'
}
}
const result = parseUpdate(update, mockSchema)
// BUG FOUND: The function has `val[key]` instead of `val[k]`
// This would cause it to access the wrong property
expect(result.extractedFields).toHaveProperty('space')
expect(result.remainingData).toHaveProperty('customField')
})
it('should handle $push operator', () => {
const update: DocumentUpdate<any> = {
$push: {
tags: 'newtag'
}
}
const result = parseUpdate(update, mockSchema)
expect(result.remainingData).toHaveProperty('tags')
})
it('should handle $pull operator', () => {
const update: DocumentUpdate<any> = {
$pull: {
tags: 'removetag'
}
}
const result = parseUpdate(update, mockSchema)
expect(result.remainingData).toHaveProperty('tags')
})
it('should handle $inc operator', () => {
const update: any = {
$inc: {
count: 1
}
}
const result = parseUpdate(update, mockSchema)
expect(result.remainingData).toHaveProperty('count')
})
it('should handle empty update', () => {
const update: DocumentUpdate<any> = {}
const result = parseUpdate(update, mockSchema)
expect(Object.keys(result.extractedFields)).toHaveLength(0)
expect(Object.keys(result.remainingData)).toHaveLength(0)
})
it('should handle mixed operators and direct fields', () => {
const update: DocumentUpdate<any> = {
space: 'space:123' as Ref<Space>,
$set: {
_class: 'class:Test',
custom: 'value'
}
}
const result = parseUpdate(update, mockSchema)
expect(result.extractedFields).toHaveProperty('space')
expect(result.extractedFields).toHaveProperty('_class')
expect(result.remainingData).toHaveProperty('custom')
})
})
describe('utils - convertDoc', () => {
const workspaceId = 'workspace:123' as WorkspaceUuid
it('should extract standard fields', () => {
const doc: any = {
_id: 'doc:1',
_class: 'class:Test',
space: 'space:1' as Ref<Space>,
modifiedOn: 1234567890,
modifiedBy: 'user:1',
createdOn: 1234567890,
createdBy: 'user:1',
customField: 'value'
}
const result = convertDoc('pg_testing', doc, workspaceId)
expect(result._id).toBe('doc:1')
expect(result._class).toBe('class:Test')
expect(result.space).toBe('space:1')
expect(result.workspaceId).toBe(workspaceId)
expect(result.data).toHaveProperty('customField')
})
it('should handle missing createdOn field', () => {
const doc: any = {
_id: 'doc:1',
_class: 'class:Test',
space: 'space:1' as Ref<Space>,
modifiedOn: 1234567890,
modifiedBy: 'user:1',
createdBy: 'user:1'
}
const result = convertDoc('pg_testing', doc, workspaceId)
expect(result.createdOn).toBe(1234567890) // Should default to modifiedOn
})
it('should generate %hash% if missing', () => {
const doc: any = {
_id: 'doc:1',
_class: 'class:Test',
space: 'space:1' as Ref<Space>,
modifiedOn: 1234567890,
modifiedBy: 'user:1',
createdBy: 'user:1'
}
const result = convertDoc('pg_testing', doc, workspaceId)
expect(result['%hash%']).toBeDefined()
expect(typeof result['%hash%']).toBe('string')
})
it('should handle null values for not-null fields', () => {
const doc: any = {
_id: 'doc:1',
_class: 'class:Test',
space: null, // null value for notNull field
modifiedOn: 1234567890,
modifiedBy: 'user:1',
createdBy: 'user:1'
}
const result = convertDoc('pg_testing', doc, workspaceId)
// Should add default value for null notNull field
expect(result.space).toBeDefined()
})
it('should preserve custom fields in data', () => {
const doc: any = {
_id: 'doc:1',
_class: 'class:Test',
space: 'space:1' as Ref<Space>,
modifiedOn: 1234567890,
modifiedBy: 'user:1',
createdBy: 'user:1',
customField1: 'value1',
customField2: 123,
customField3: true
}
const result = convertDoc('pg_testing', doc, workspaceId)
expect(result.data.customField1).toBe('value1')
expect(result.data.customField2).toBe(123)
expect(result.data.customField3).toBe(true)
})
})
describe('utils - parseDoc', () => {
const mockSchema = {
_id: { type: 'text' as const, notNull: true, index: false },
_class: { type: 'text' as const, notNull: true, index: false },
space: { type: 'text' as const, notNull: true, index: false },
modifiedOn: { type: 'bigint' as const, notNull: true, index: false },
count: { type: 'bigint' as const, notNull: false, index: false },
tags: { type: 'text[]' as const, notNull: false, index: false }
}
it('should parse DBDoc to Doc', () => {
const dbDoc: any = {
_id: 'doc:1',
_class: 'class:Test',
space: 'space:1',
modifiedOn: '1234567890',
modifiedBy: 'user:1',
createdOn: 1234567890,
createdBy: 'user:1',
workspaceId: 'workspace:123' as WorkspaceUuid,
data: {
customField: 'value'
}
}
const result: any = parseDoc(dbDoc, mockSchema)
expect(result._id).toBe('doc:1')
expect(result.customField).toBe('value')
})
it('should parse bigint fields to numbers', () => {
const dbDoc: any = {
_id: 'doc:1',
_class: 'class:Test',
space: 'space:1',
modifiedOn: '1234567890',
count: '42',
modifiedBy: 'user:1',
createdOn: 1234567890,
createdBy: 'user:1',
workspaceId: 'workspace:123' as WorkspaceUuid,
data: {}
}
const result: any = parseDoc(dbDoc, mockSchema)
expect(result.modifiedOn).toBe(1234567890)
expect(typeof result.modifiedOn).toBe('number')
expect(result.count).toBe(42)
})
it('should decode text[] fields', () => {
const dbDoc: any = {
_id: 'doc:1',
_class: 'class:Test',
space: 'space:1',
modifiedOn: '1234567890',
tags: '{tag1,tag2,tag3}',
modifiedBy: 'user:1',
createdOn: 1234567890,
createdBy: 'user:1',
workspaceId: 'workspace:123' as WorkspaceUuid,
data: {}
}
const result: any = parseDoc(dbDoc, mockSchema)
expect(result.tags).toEqual(['tag1', 'tag2', 'tag3'])
expect(Array.isArray(result.tags)).toBe(true)
})
it('should handle NULL values', () => {
const dbDoc: any = {
_id: 'doc:1',
_class: 'class:Test',
space: 'space:1',
modifiedOn: '1234567890',
count: 'NULL',
modifiedBy: 'user:1',
createdOn: 1234567890,
createdBy: 'user:1',
workspaceId: 'workspace:123' as WorkspaceUuid,
data: {}
}
const result: any = parseDoc(dbDoc, mockSchema)
expect(result.count).toBeNull()
})
it('should handle null values', () => {
const dbDoc: any = {
_id: 'doc:1',
_class: 'class:Test',
space: 'space:1',
modifiedOn: '1234567890',
count: null,
modifiedBy: 'user:1',
createdOn: 1234567890,
createdBy: 'user:1',
workspaceId: 'workspace:123' as WorkspaceUuid,
data: {}
}
const result: any = parseDoc(dbDoc, mockSchema)
expect(result.count).toBeNull()
})
it('should delete attachedTo if NULL', () => {
const dbDoc: any = {
_id: 'doc:1',
_class: 'class:Test',
space: 'space:1',
modifiedOn: '1234567890',
attachedTo: 'NULL',
modifiedBy: 'user:1',
createdOn: 1234567890,
createdBy: 'user:1',
workspaceId: 'workspace:123' as WorkspaceUuid,
data: {}
}
const result: any = parseDoc(dbDoc, mockSchema)
expect(result.attachedTo).toBeUndefined()
})
it('should handle invalid bigint values gracefully', () => {
const dbDoc: any = {
_id: 'doc:1',
_class: 'class:Test',
space: 'space:1',
modifiedOn: 'invalid',
modifiedBy: 'user:1',
createdOn: 1234567890,
createdBy: 'user:1',
workspaceId: 'workspace:123' as WorkspaceUuid,
data: {}
}
const result: any = parseDoc(dbDoc, mockSchema)
// parseInt('invalid') returns NaN
// BUG: This should be handled better
expect(Number.isNaN(result.modifiedOn)).toBe(true)
})
})
describe('utils - parseDocWithProjection', () => {
it('should apply projection to filter fields', () => {
const dbDoc: any = {
_id: 'doc:1',
_class: 'class:Test',
space: 'space:1',
modifiedOn: '1234567890',
modifiedBy: 'user:1',
createdOn: 1234567890,
createdBy: 'user:1',
workspaceId: 'workspace:123' as WorkspaceUuid,
data: {
field1: 'value1',
field2: 'value2',
field3: 'value3'
}
}
const projection: any = {
field1: 1,
field2: 1
}
const result: any = parseDocWithProjection(dbDoc, 'pg_testing', projection)
expect(result.field1).toBe('value1')
expect(result.field2).toBe('value2')
expect(result.field3).toBeUndefined()
})
it('should work without projection', () => {
const dbDoc: any = {
_id: 'doc:1',
_class: 'class:Test',
space: 'space:1',
modifiedOn: '1234567890',
modifiedBy: 'user:1',
createdOn: 1234567890,
createdBy: 'user:1',
workspaceId: 'workspace:123' as WorkspaceUuid,
data: {
field1: 'value1'
}
}
const result: any = parseDocWithProjection(dbDoc, 'pg_testing', undefined)
expect(result._id).toBe('doc:1')
expect(result.field1).toBe('value1')
})
})
describe('utils - filterProjection', () => {
it('should filter fields based on projection', () => {
const data = {
field1: 'value1',
field2: 'value2',
field3: 'value3'
}
const projection: any = {
field1: 1,
field2: 1
}
const result = filterProjection(data, projection)
expect(result.field1).toBe('value1')
expect(result.field2).toBe('value2')
expect(result.field3).toBeUndefined()
})
it('should handle nested projections', () => {
const data = {
user: {
name: 'John',
email: 'john@example.com',
age: 30
},
status: 'active'
}
const projection: any = {
'user.name': 1,
'user.email': 1
}
const result = filterProjection(data, projection)
expect(result.user.name).toBe('John')
expect(result.user.email).toBe('john@example.com')
expect(result.user.age).toBeUndefined()
})
it('should return all data when projection is undefined', () => {
const data = {
field1: 'value1',
field2: 'value2'
}
const result = filterProjection(data, undefined)
expect(result).toEqual(data)
})
it('should handle projection with 0 values', () => {
const data = {
field1: 'value1',
field2: 'value2',
field3: 'value3'
}
const projection: any = {
field1: 1,
field3: 0
}
const result = filterProjection(data, projection)
expect(result.field1).toBe('value1')
expect(result.field2).toBeUndefined()
expect(result.field3).toBeUndefined()
})
it('should handle empty objects', () => {
const data = {}
const projection: any = { field1: 1 }
const result = filterProjection(data, projection)
expect(result).toEqual({})
})
it('should handle pure exclusion projection (only 0 values)', () => {
const data = {
name: 'Task 1',
description: 'Details',
priority: 1,
status: 'active'
}
const projection: any = {
description: 0,
status: 0
}
const result = filterProjection(data, projection)
// In exclusion mode, all fields EXCEPT explicitly excluded ones should remain
expect(result.name).toBe('Task 1')
expect(result.priority).toBe(1)
expect(result.description).toBeUndefined()
expect(result.status).toBeUndefined()
})
it('should handle nested exclusion projection', () => {
const data = {
user: {
name: 'John',
email: 'john@example.com',
age: 30,
address: '123 Main St'
},
status: 'active'
}
const projection: any = {
'user.email': 0,
'user.address': 0
}
const result = filterProjection(data, projection)
// In exclusion mode, keep everything except excluded nested fields
expect(result.user.name).toBe('John')
expect(result.user.age).toBe(30)
expect(result.user.email).toBeUndefined()
expect(result.user.address).toBeUndefined()
expect(result.status).toBe('active')
})
})
describe('utils - isDataField', () => {
it('should return false for standard doc fields', () => {
expect(isDataField('pg_testing', '_id')).toBe(false)
expect(isDataField('pg_testing', '_class')).toBe(false)
expect(isDataField('pg_testing', 'space')).toBe(false)
expect(isDataField('pg_testing', 'modifiedOn')).toBe(false)
expect(isDataField('pg_testing', 'modifiedBy')).toBe(false)
expect(isDataField('pg_testing', 'createdOn')).toBe(false)
expect(isDataField('pg_testing', 'createdBy')).toBe(false)
})
it('should return true for custom fields', () => {
expect(isDataField('pg_testing', 'customField')).toBe(true)
expect(isDataField('pg_testing', 'myProperty')).toBe(true)
})
it('should handle attachedTo field', () => {
expect(isDataField('pg_testing', 'attachedTo')).toBe(false)
})
})
describe('utils - edge cases and potential bugs', () => {
it('should handle parseInt with invalid strings', () => {
// BUG: parseInt without radix and no error handling can produce unexpected results
expect(Number.parseInt('123abc')).toBe(123) // Silently parses partial number
expect(Number.isNaN(Number.parseInt('abc'))).toBe(true)
expect(Number.parseInt('08')).toBe(8) // ES5+ treats as decimal, but can be confusing
})
it('should handle NaN and Infinity in number validations', () => {
expect(isNaN(NaN)).toBe(true)
expect(isFinite(Infinity)).toBe(false)
expect(isFinite(-Infinity)).toBe(false)
expect(isNaN(Number('invalid'))).toBe(true)
})
it('should handle edge cases in array conversions', () => {
// Edge case: array with undefined (actually keeps 'undefined' as string)
const result = (convertArrayParams([[undefined, 'value']]) as any)[0] as string
expect(result).toBe('{undefined,"value"}')
})
it('should handle SQL injection attempts in escape function', () => {
const malicious = "'; DELETE FROM users; --"
const escaped = escape(malicious)
expect(escaped).not.toContain(';')
// BUG: escape function doesn't remove '--', only special chars not in the allowed set
// The function keeps hyphens and spaces, so '--' remains
expect(escaped).toBe(' DELETE FROM users --')
})
it('should handle very long strings', () => {
const longString = 'a'.repeat(10000)
expect(escape(longString).length).toBe(10000)
expect(escapeBackticks(longString)).toBe(longString)
})
it('should handle Unicode and special characters', () => {
expect(escape('test\u0000null')).toBe('testnull')
// Emoji characters might not be fully removed depending on implementation
expect(escape('emoji😀test')).toBe('emojitest')
expect(escapeBackticks("test'unicode\u0000")).toBe("test''unicode\u0000")
})
it('should handle empty and whitespace strings', () => {
expect(escape(' ')).toBe(' ')
expect(escapeBackticks(' ')).toBe(' ')
expect(decodeArray('{" "}')).toEqual([' '])
})
})
+40 -15
View File
@@ -392,22 +392,47 @@ export function filterProjection<T extends Doc> (data: any, projection: Projecti
if (projection === undefined) {
return data
}
for (const key in data) {
if (!Object.prototype.hasOwnProperty.call(projection, key) || (projection as any)[key] === 0) {
// check nested projections in case of object
let value = data[key]
if (typeof value === 'object' && !Array.isArray(value) && value != null) {
// We need to filter projection for nested objects
const innerP = Object.entries(projection as any)
.filter((it) => it[0].startsWith(key))
.map((it) => [it[0].substring(key.length + 1), it[1]])
if (innerP.length > 0) {
value = filterProjection(value, Object.fromEntries(innerP))
data[key] = value
continue
}
}
// Determine if this is an inclusion or exclusion projection
// Exclusion: has any field with value 0
// Inclusion: has any field with value 1
const projectionValues = Object.values(projection as any)
const hasExclusion = projectionValues.some((v) => v === 0)
const hasInclusion = projectionValues.some((v) => v === 1 || typeof v === 'object')
// Can't mix inclusion and exclusion (MongoDB behavior)
// If mixed, treat as inclusion
const isExclusionMode = hasExclusion && !hasInclusion
for (const key in data) {
const projValue = (projection as any)[key]
const hasKey = Object.prototype.hasOwnProperty.call(projection, key)
// Check if there are nested projections for this key
let value = data[key]
if (typeof value === 'object' && !Array.isArray(value) && value != null) {
// We need to filter projection for nested objects
const innerP = Object.entries(projection as any)
.filter((it) => it[0].startsWith(key + '.'))
.map((it) => [it[0].substring(key.length + 1), it[1]])
if (innerP.length > 0) {
value = filterProjection(value, Object.fromEntries(innerP))
data[key] = value
continue // Don't delete the parent object
}
}
let shouldDelete = false
if (isExclusionMode) {
// Exclusion mode: delete only fields explicitly set to 0
shouldDelete = hasKey && projValue === 0
} else {
// Inclusion mode: delete fields not in projection (unless nested projection)
shouldDelete = !hasKey || projValue === 0
}
if (shouldDelete) {
// eslint-disable-next-line @typescript-eslint/no-dynamic-delete
delete data[key]
}
+4 -2
View File
@@ -2,6 +2,8 @@ module.exports = {
preset: 'ts-jest',
testEnvironment: 'node',
testMatch: ['**/?(*.)+(spec|test).[jt]s?(x)'],
roots: ["./src"],
coverageReporters: ["text-summary", "html"]
roots: ['./src'],
collectCoverage: true,
coverageReporters: ['text-summary', 'html', 'lcov'],
coverageDirectory: 'coverage'
}
+4 -2
View File
@@ -2,6 +2,8 @@ module.exports = {
preset: 'ts-jest',
testEnvironment: 'node',
testMatch: ['**/?(*.)+(spec|test).[jt]s?(x)'],
roots: ["./src"],
coverageReporters: ["text-summary", "html"]
roots: ['./src'],
collectCoverage: true,
coverageReporters: ['text-summary', 'html', 'lcov'],
coverageDirectory: 'coverage'
}
+2 -3
View File
@@ -30,9 +30,9 @@
"build": "compile",
"build:watch": "compile",
"format": "format src",
"test": "jest --passWithNoTests --silent",
"test": "jest --passWithNoTests --silent --forceExit",
"_phase:build": "compile transpile src",
"_phase:test": "jest --passWithNoTests --silent",
"_phase:test": "jest --passWithNoTests --silent --forceExit",
"_phase:format": "format src",
"_phase:validate": "compile validate"
},
@@ -58,7 +58,6 @@
"@hcengineering/core": "^0.7.3",
"@hcengineering/platform": "^0.7.3",
"@hcengineering/server-core": "workspace:0.7.0",
"@hcengineering/mongo": "workspace:0.7.0",
"@hcengineering/minio": "workspace:0.7.0",
"@hcengineering/s3": "workspace:0.7.0",
"@hcengineering/datalake": "workspace:0.7.0",
+4 -2
View File
@@ -2,6 +2,8 @@ module.exports = {
preset: 'ts-jest',
testEnvironment: 'node',
testMatch: ['**/?(*.)+(spec|test).[jt]s?(x)'],
roots: ["./src"],
coverageReporters: ["text-summary", "html"]
roots: ['./src'],
collectCoverage: true,
coverageReporters: ['text-summary', 'html', 'lcov'],
coverageDirectory: 'coverage'
}
+1 -1
View File
@@ -3,7 +3,7 @@ DATALAKE_STORAGE_CONFIG="datalake|http://datalake:4031"
BACKUP_STORAGE_CONFIG="minio|minio?accessKey=minioadmin&secretKey=minioadmin"
BACKUP_BUCKET_NAME=dev-backups
DB_URL=mongodb://mongodb:27018
DB_URL=postgresql://root@cockroach:26258/defaultdb?sslmode=disable
MONGO_URL=mongodb://mongodb:27018
DB_PG_URL=postgresql://root@cockroach:26257/defaultdb?sslmode=disable
QUEUE_CONFIG='redpanda:9093;-staging'