Compare commits

...

29 Commits

Author SHA1 Message Date
samunohito 1d9fa528e9 正しくREDACTEDにされるかのテストを追加 2026-08-01 09:33:15 +09:00
samunohito adc1ec2319 test(backend): strengthen queue trace context coverage 2026-08-01 08:12:54 +09:00
samunohito 5b153839bd fix(backend): address OpenTelemetry review feedback 2026-08-01 07:59:31 +09:00
kakkokari-gtyih d8e22cdda2 Update Changelog 2026-07-31 19:12:34 +09:00
kakkokari-gtyih 38fd1b4ebd Merge branch 'develop' into revert-17812-fix/revert-otel 2026-07-31 19:11:26 +09:00
github-actions[bot] d54a9dcd3e [skip ci] Update CHANGELOG.md (prepend template) 2026-07-31 07:48:04 +00:00
github-actions[bot] 8ea4a0ecac Release: 2026.7.0 2026-07-31 07:47:55 +00:00
おさむのひと 8dd59a8c06 fix: RSS系ウィジェットにフィードURL検査機構を追加 (#17831) 2026-07-31 16:11:39 +09:00
おさむのひと f74a303556 fix: sensitive-detectorへのリクエスト時もHttpRequestServiceを使うように (#17828) 2026-07-31 13:43:51 +09:00
samunohito 33afa9876f fix CodeRabbit review 2026-07-31 10:52:00 +09:00
おさむのひと 589d43b38b Update packages/backend/src/logging/PrettyConsoleBackend.ts
Co-authored-by: anatawa12 <anatawa12@icloud.com>
2026-07-30 21:31:42 +09:00
かっこかり bcc3bc4fd7 fix(frontend): MkLightboxの動画の縦横比を事前に確定させるように (#17822)
* fix(frontend): MkLightboxの動画の縦横比を事前に確定させるように

* fix

* fix
2026-07-29 14:47:58 +09:00
かっこかり 038623325c Revert "fix: OTel除去 (#17812)"
This reverts commit 6dac6f7928.
2026-07-29 09:40:14 +09:00
おさむのひと 6f579b6854 chore: docker_example.ymlのsentryForBackend更新が漏れていたのを追記 (#17817) 2026-07-28 21:25:08 +09:00
github-actions[bot] d600525bb2 Bump version to 2026.7.0-rc.0 2026-07-28 10:21:11 +00:00
おさむのひと 6dac6f7928 fix: OTel除去 (#17812) 2026-07-28 19:15:44 +09:00
かっこかり 318af2e1ff fix(frontend): follow-up of #17811 (#17814) 2026-07-28 17:35:33 +09:00
かっこかり a44e419d5a fix(frontend): follow-up of 2509a28 (#17815) 2026-07-28 17:35:18 +09:00
かっこかり 2509a28813 Merge commit from fork 2026-07-28 11:11:56 +09:00
おさむのひと a12ee2e5d7 Merge commit from fork 2026-07-28 11:09:37 +09:00
かっこかり 703aa2360b enhance(frontend): UploaderItemsの詳細メニューからもプレビューを起動できるように (#17811)
* enhance(frontend): UploaderItemsの詳細メニューからもプレビューを起動できるように

* clean up
2026-07-27 18:34:31 +09:00
かっこかり cb58ff32a4 fix(gh): check-misskey-js-autogenが失敗する問題を修正 (#17809)
* fix(gh): check-misskey-js-autogenが失敗する問題を修正

* fix comment

* fix
2026-07-27 16:48:36 +09:00
かっこかり f6979333d4 docs: Update CHANGELOG.md to include about js-yaml changes (#17805) 2026-07-27 00:14:34 +09:00
かっこかり e897255e71 fix 2026-07-25 21:43:41 +09:00
syuilo daf2a85539 Update CHANGELOG.md 2026-07-25 21:37:20 +09:00
chocolate-pie 1a59ec20e3 Merge commit from fork
* fix: Fix improper authorization in `admin/reset-password`

* Apply suggestions from code review

Co-authored-by: かっこかり <67428053+kakkokari-gtyih@users.noreply.github.com>

* fix: improper authorization in `admin/unset-mfa`

* fix

---------

Co-authored-by: かっこかり <67428053+kakkokari-gtyih@users.noreply.github.com>
2026-07-25 21:34:46 +09:00
4ster1sk 3e31e59bcd fix(backend): フォロー中のチャンネルを再度フォローした際にALREADY_FOLLOWINGエラーを返すように (#17802)
* test(e2e): フォロー中のチャンネルに再度フォローAPIを叩いた場合のテストを追加

* fix(backend): フォロー中のチャンネルを再度フォローした際にALREADY_FOLLOWINGエラーを返すように

* docs(changelog): update changelog
2026-07-25 21:09:12 +09:00
anatawa12 982d4905c0 use UPSERT instead of select for hashtag statistics update (#17795)
* feat: set defaults for hashtag table

* chore: use upsert to update hashtag table

* fix: query runner not cleaned up

* fix: query runner not cleaned up correctly

* chore: replace await using => try-finally
2026-07-25 19:48:20 +09:00
renovate[bot] 7ecce0901c fix(deps): update dependency js-yaml to v5.2.2 [security] (#17797)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-07-25 16:07:49 +09:00
46 changed files with 1474 additions and 383 deletions
+23
View File
@@ -173,8 +173,31 @@ id: 'aidx'
#sentryForBackend:
# enableNodeProfiling: true
# # Specify Sentry integration names to disable individual auto-instrumentation.
# # The names are integration .name values, not factory function names.
# # To check enabled names, set `options.debug: true` and see
# # "Integration installed: <name>" in the Sentry logs.
# #
# # As of 2026-07-07 / @sentry/node 10.62.0, useful names for Misskey include:
# # Postgres ... DB queries (when pg is externalized)
# # Redis ... ioredis commands
# # Fastify ... inbound HTTP routes
# # Http ... inbound/outbound HTTP; disabling this can also affect request isolation
# # NodeFetch ... fetch/undici requests
# disabledIntegrations: ['Postgres']
# options:
# dsn: 'https://examplePublicKey@o0.ingest.sentry.io/0'
# # By default, Misskey prevents Sentry trace headers (`sentry-trace` and
# # `baggage`) from being sent to remote ActivityPub/Webhook/etc. hosts.
# # To intentionally propagate distributed traces to trusted internal services,
# # list only those internal URL patterns here.
# # Avoid broad patterns that match remote servers unless you intentionally
# # want to restore Sentry's legacy behavior of propagating traces to all
# # outbound HTTP requests.
# #tracePropagationTargets: []
# # Internal allowlist example:
# #tracePropagationTargets:
# # - 'internal-service.example'
#sentryForFrontend:
# vueIntegration:
@@ -0,0 +1,73 @@
name: Check Misskey JS autogen (comment)
on:
workflow_run:
types: [completed]
workflows:
- Check Misskey JS autogen # check-misskey-js-autogen.yml
jobs:
comment-misskey-js-autogen:
runs-on: ubuntu-latest
# No code is cloned or executed here, so it is safe to hold pull-requests: write
if: ${{ github.event.workflow_run.event == 'pull_request' }}
permissions:
actions: read
pull-requests: write
steps:
# The result is missing when the run failed before the check job (e.g. the PR does not build), so do not fail here
- name: Download result
id: download-result
continue-on-error: true
uses: actions/download-artifact@v8
with:
name: misskey-js-autogen-result
path: result
github-token: ${{ github.token }}
repository: ${{ github.repository }}
run-id: ${{ github.event.workflow_run.id }}
# The artifact comes from a run that built fork code, so validate it before using
- name: Load result
id: load-result
if: steps.download-result.outcome == 'success'
run: |
pr_number="$(cat result/pr-number)"
changes="$(cat result/changes)"
case "$pr_number" in
''|*[!0-9]*) echo "invalid pr number"; exit 1;;
esac
case "$changes" in
true|false) ;;
*) echo "invalid result"; exit 1;;
esac
echo "pr-number=$pr_number" >> "$GITHUB_OUTPUT"
echo "changes=$changes" >> "$GITHUB_OUTPUT"
- name: send message
if: steps.load-result.outputs.changes == 'true'
uses: thollander/actions-comment-pull-request@v3
with:
pr-number: ${{ steps.load-result.outputs.pr-number }}
comment-tag: check-misskey-js-autogen
message: |-
Thank you for sending us a great Pull Request! 👍
Please regenerate misskey-js type definitions! 🙏
example:
```sh
pnpm run build-misskey-js-with-types
```
- name: send message
if: steps.load-result.outputs.changes == 'false'
uses: thollander/actions-comment-pull-request@v3
with:
pr-number: ${{ steps.load-result.outputs.pr-number }}
comment-tag: check-misskey-js-autogen
mode: delete
message: "Thank you!"
create_if_not_exists: false
+17 -26
View File
@@ -1,7 +1,8 @@
# this name is used in check-misskey-js-autogen.comment.yml so be careful when change name
name: Check Misskey JS autogen
on:
pull_request_target:
pull_request:
branches:
- master
- develop
@@ -10,7 +11,6 @@ on:
- packages/backend/**
jobs:
# pull_request_target safety: permissions: read-all, and there are no secrets used in this job
generate-misskey-js:
runs-on: ubuntu-latest
permissions:
@@ -58,7 +58,6 @@ jobs:
name: generated-misskey-js
path: packages/misskey-js/generator/built/autogen
# pull_request_target safety: permissions: read-all, and no user codes are executed
get-actual-misskey-js:
runs-on: ubuntu-latest
permissions:
@@ -78,12 +77,11 @@ jobs:
name: actual-misskey-js
path: packages/misskey-js/src/autogen
# pull_request_target safety: nothing is cloned from repository
comment-misskey-js-autogen:
check-misskey-js-autogen:
runs-on: ubuntu-latest
needs: [generate-misskey-js, get-actual-misskey-js]
permissions:
pull-requests: write
contents: read
steps:
- name: download generated-misskey-js
uses: actions/download-artifact@v8
@@ -111,28 +109,21 @@ jobs:
- name: Print full diff
run: cat ./misskey-js.diff
- name: send message
if: steps.check-changes.outputs.changes == 'true'
uses: thollander/actions-comment-pull-request@v3
with:
comment-tag: check-misskey-js-autogen
message: |-
Thank you for sending us a great Pull Request! 👍
Please regenerate misskey-js type definitions! 🙏
# A fork's pull_request token cannot comment, so hand the result to check-misskey-js-autogen.comment.yml
- name: Save result
env:
PR_NUMBER: ${{ github.event.pull_request.number }}
CHANGES: ${{ steps.check-changes.outputs.changes }}
run: |
mkdir -p result
echo "$PR_NUMBER" > result/pr-number
echo "$CHANGES" > result/changes
example:
```sh
pnpm run build-misskey-js-with-types
```
- name: send message
if: steps.check-changes.outputs.changes == 'false'
uses: thollander/actions-comment-pull-request@v3
- name: Upload result
uses: actions/upload-artifact@v7
with:
comment-tag: check-misskey-js-autogen
mode: delete
message: "Thank you!"
create_if_not_exists: false
name: misskey-js-autogen-result
path: result
- name: Make failure if changes are detected
if: steps.check-changes.outputs.changes == 'true'
+27 -11
View File
@@ -1,3 +1,22 @@
## Unreleased
### General
-
### Client
-
### Server
- Feat: OpenTelemetryサポート
- 詳細な設定はconfigファイルを参照してください。
- Sentryとの併用も可能です。Sentry併用時は、PostgreSQL Query と Redis command は Sentry で計装されます。
- 以下の自動計装をサポートしています。(計装対象にする項目は設定可能)
- PostgreSQL query
- Redis command
- 全ての受信HTTPリクエスト
- 全ての送信HTTPリクエスト
- ジョブキュー(エンキュー元のトレースを含む)
## 2026.7.0
### Note
@@ -12,11 +31,14 @@
- Node.js のセキュリティアップデートに伴い、最低動作バージョンを 22.22.2 / 24.17.0 / 26.4.0 に引き上げました。
- Docker Image は Node.js 26.4.0-trixie に更新されています。
- バックエンドで画像処理に用いているライブラリ sharp のシステム要件の変更により、**SSE4.2 命令セットをサポートしていない x86_64 CPU では Misskey が正しく動作しなくなります**。仮想マシンに Misskey をデプロイしている場合や、古いハードウェアをお使いの場合は、アップデート前にお使いの環境をご確認ください。なお、ARM64 など x86_64 ではない環境においてはこの変更による影響はありません。
- YAMLパーサーをアップデートし、より厳格なチェックが行われるようになりました。起動時にconfigの読み取りで構文エラーが発生する可能性があります。\
例えば `allowPrivateNetworks` を 2026.6.0 までの `example.yml` の構文を元に記述している場合は、配列の閉じ括弧のインデントを上げるか、リスト表示に書き換える必要があります。詳しくは https://github.com/misskey-dev/misskey/pull/17701 をご覧ください。
### General
- Feat: コントロールパネルから二要素認証を解除できるように
- Feat: 条件に一致したURLプレビューのサムネイルを隠すことができるように
(Based on https://github.com/MisskeyIO/misskey/pull/214)
- Enhance: 依存関係の更新
### Client
- 2025.4.0 以前の設定情報の移行処理が削除されました
@@ -31,6 +53,7 @@
- Fix: 一部の画像をビューワーで読み込んだ際に正しく表示されない問題を修正
- Enhance: ファイルアップロード前にプレビューできるように
- Enhance: タブがバックグラウンドの間は必要ない定期更新処理を停止するように
- Enhance: 翻訳の更新
- Fix: 「画像を新しいタブで開く」が機能しなくなっていた問題を修正
- Fix: デバイスタイプをスマートフォンに固定している状態で画面幅が広いとき、画面左上のアイコンが表示されない問題を修正
- Fix: チャットでIMEの変換を確定するEnterでメッセージが送信されてしまうことがある問題を修正
@@ -44,21 +67,12 @@
- Fix: ノートの詳細表示で削除された引用元が表示されない問題を修正
### Server
- Feat: OpenTelemetryサポート
- 詳細な設定はconfigファイルを参照してください。
- Sentryとの併用も可能です。Sentry併用時は、PostgreSQL Query と Redis command は Sentry で計装されます。
- 以下の自動計装をサポートしています。(計装対象にする項目は設定可能)
- PostgreSQL query
- Redis command
- 全ての受信HTTPリクエスト
- 全ての送信HTTPリクエスト
- ジョブキュー(エンキュー元のトレースを含む)
- Feat: ログ基盤の刷新
- API内部エラーのログに構造化属性と正規化したエラー情報を付与し、認証情報を自動的に秘匿するように(従来形式の表示は維持)
- ログ全体の既定出力レベルとドメインごとの出力レベルを設定できるように
- バックエンドのログを1行JSON形式で出力できるように
- OpenTelemetryのTrace ContextをJSON形式のログへ関連付けられるように
- HTTPのAccess logをstatus class単位で出力できるように(開発時のリクエスト・レスポンス本文、OpenTelemetryのTrace Contextにも対応)
- SentryのTrace ContextをJSON形式のログへ関連付けられるように
- HTTPのAccess logをstatus class単位で出力できるように(開発時のリクエスト・レスポンス本文、SentryのTrace Contextにも対応)
- Enhance: Sentry バックエンドの自動計装を `sentryForBackend.disabledIntegrations` で個別に無効化できるように
- Enhance: センシティブメディアの判定を外部サービス ([sensitive-detector](https://github.com/misskey-dev/sensitive-detector)) に分離し、`nsfwjs` / `@tensorflow/tfjs(-node)` の同梱と NSFW 判定モデルを廃止 (#16804)
- Enhance: Node.js 22.22.2以降、24.17.0以降、26.4.0以降をサポートするように
@@ -71,6 +85,8 @@
- Fix: フォロワー限定投稿へのリプライをホーム投稿に出来る問題を修正
- Fix: ファイルをアップロードするAPIにて、処理終了後に一時ファイルが削除されないことがある問題を修正
- Fix: 初期設定で作成したアカウント以外でアカウント作成APIが使用できない問題を修正
- Fix: フォロー中のチャンネルを再度フォローした際にALREADY_FOLLOWINGエラーを返すように
- Fix: セキュリティに関する修正
## 2026.6.0
+2 -2
View File
@@ -1,6 +1,6 @@
{
"name": "misskey",
"version": "2026.7.0-beta.6",
"version": "2026.7.0",
"codename": "nasubi",
"repository": {
"type": "git",
@@ -57,7 +57,7 @@
"esbuild": "0.28.1",
"execa": "9.6.1",
"ignore-walk": "9.0.0",
"js-yaml": "5.2.1",
"js-yaml": "5.2.2",
"tar": "7.5.21"
},
"devDependencies": {
@@ -0,0 +1,26 @@
/*
* SPDX-FileCopyrightText: syuilo and misskey-project
* SPDX-License-Identifier: AGPL-3.0-only
*/
export class HashtagTableDefaults1784899839024 {
name = 'HashtagTableDefaults1784899839024'
async up(queryRunner) {
await queryRunner.query(`ALTER TABLE "hashtag" ALTER COLUMN "mentionedUserIds" SET DEFAULT '{}'`);
await queryRunner.query(`ALTER TABLE "hashtag" ALTER COLUMN "mentionedLocalUserIds" SET DEFAULT '{}'`);
await queryRunner.query(`ALTER TABLE "hashtag" ALTER COLUMN "mentionedRemoteUserIds" SET DEFAULT '{}'`);
await queryRunner.query(`ALTER TABLE "hashtag" ALTER COLUMN "attachedUserIds" SET DEFAULT '{}'`);
await queryRunner.query(`ALTER TABLE "hashtag" ALTER COLUMN "attachedLocalUserIds" SET DEFAULT '{}'`);
await queryRunner.query(`ALTER TABLE "hashtag" ALTER COLUMN "attachedRemoteUserIds" SET DEFAULT '{}'`);
}
async down(queryRunner) {
await queryRunner.query(`ALTER TABLE "hashtag" ALTER COLUMN "attachedRemoteUserIds" DROP DEFAULT`);
await queryRunner.query(`ALTER TABLE "hashtag" ALTER COLUMN "attachedLocalUserIds" DROP DEFAULT`);
await queryRunner.query(`ALTER TABLE "hashtag" ALTER COLUMN "attachedUserIds" DROP DEFAULT`);
await queryRunner.query(`ALTER TABLE "hashtag" ALTER COLUMN "mentionedRemoteUserIds" DROP DEFAULT`);
await queryRunner.query(`ALTER TABLE "hashtag" ALTER COLUMN "mentionedLocalUserIds" DROP DEFAULT`);
await queryRunner.query(`ALTER TABLE "hashtag" ALTER COLUMN "mentionedUserIds" DROP DEFAULT`);
}
}
+2 -2
View File
@@ -71,7 +71,7 @@
"@opentelemetry/api": "1.9.1",
"@opentelemetry/core": "2.9.0",
"@opentelemetry/exporter-trace-otlp-proto": "0.220.0",
"@opentelemetry/instrumentation": "0.219.0",
"@opentelemetry/instrumentation": "0.220.0",
"@opentelemetry/instrumentation-pg": "0.72.0",
"@opentelemetry/resources": "2.9.0",
"@opentelemetry/sdk-trace-base": "2.9.0",
@@ -193,7 +193,7 @@
"eslint-plugin-import": "2.32.0",
"execa": "9.6.1",
"fkill": "10.0.3",
"js-yaml": "5.2.1",
"js-yaml": "5.2.2",
"pid-port": "2.1.1",
"rolldown": "1.1.5",
"simple-oauth2": "5.1.0",
+13 -2
View File
@@ -27,6 +27,10 @@ type SentryBackendConfig = {
disabledIntegrations?: string[];
};
type SentryBackendConfigSource = Omit<SentryBackendConfig, 'options'> & {
options?: Partial<Sentry.NodeOptions>;
};
type OtelBackendConfig = {
endpoint?: string;
headers?: Record<string, string>;
@@ -86,7 +90,7 @@ type Source = {
index: string;
scope?: 'local' | 'global' | string[];
};
sentryForBackend?: SentryBackendConfig;
sentryForBackend?: SentryBackendConfigSource;
otelForBackend?: OtelBackendConfig;
sentryForFrontend?: {
options: Partial<SentryVue.BrowserOptions> & { dsn: string };
@@ -338,7 +342,7 @@ export function loadConfig(): Config {
redisForJobQueue: config.redisForJobQueue ? convertRedisOptions(config.redisForJobQueue, host) : redis,
redisForTimelines: config.redisForTimelines ? convertRedisOptions(config.redisForTimelines, host) : redis,
redisForReactions: config.redisForReactions ? convertRedisOptions(config.redisForReactions, host) : redis,
sentryForBackend: config.sentryForBackend,
sentryForBackend: config.sentryForBackend == null ? undefined : normalizeSentryBackendConfig(config.sentryForBackend),
otelForBackend: config.otelForBackend,
sentryForFrontend: config.sentryForFrontend,
id: config.id,
@@ -376,6 +380,13 @@ export function loadConfig(): Config {
};
}
export function normalizeSentryBackendConfig(config: SentryBackendConfigSource): SentryBackendConfig {
return {
...config,
options: config.options ?? {},
};
}
function tryCreateUrl(url: string) {
try {
return new URL(url);
+4 -11
View File
@@ -4,7 +4,6 @@
*/
import { Injectable, Inject } from '@nestjs/common';
import fetch from 'node-fetch';
import { DI } from '@/di-symbols.js';
import { bindThis } from '@/decorators.js';
import { HttpRequestService } from '@/core/HttpRequestService.js';
@@ -131,9 +130,6 @@ export class AiService {
@bindThis
private async detectChunk(url: string, apiKey: string | null, timeout: number, chunk: Buffer[]): Promise<(Prediction[] | null)[]> {
const controller = new AbortController();
const timer = setTimeout(() => controller.abort(), timeout);
try {
const form = new FormData();
for (let i = 0; i < chunk.length; i++) {
@@ -148,14 +144,13 @@ export class AiService {
headers['Authorization'] = `Bearer ${apiKey}`;
}
const res = await fetch(url, {
const res = await this.httpRequestService.send(url, {
method: 'POST',
headers,
body: form,
// 外部サービスとして通常の proxy / private address 制限を適用する。
// サイドカーへの private network 接続は allowedPrivateNetworks 等で明示的に許可する。
agent: (u) => this.httpRequestService.getAgentByUrl(u),
signal: controller.signal,
timeout,
}, {
throwErrorWhenResponseNotOk: false,
});
if (!res.ok) {
@@ -181,8 +176,6 @@ export class AiService {
} catch (err) {
this.logger.warn(`sensitive detection error: ${err instanceof Error ? err.message : String(err)}`);
return chunk.map(() => null);
} finally {
clearTimeout(timer);
}
}
}
@@ -13,6 +13,8 @@ import { GlobalEvents, GlobalEventService } from '@/core/GlobalEventService.js';
import { bindThis } from '@/decorators.js';
import type { MiLocalUser } from '@/models/User.js';
import { RedisKVCache } from '@/misc/cache.js';
import { IdentifiableError } from '@/misc/identifiable-error.js';
import { isDuplicateKeyValueError } from '@/misc/is-duplicate-key-value-error.js';
@Injectable()
export class ChannelFollowingService implements OnModuleInit {
@@ -96,11 +98,18 @@ export class ChannelFollowingService implements OnModuleInit {
requestUser: MiLocalUser,
targetChannel: MiChannel,
): Promise<void> {
await this.channelFollowingsRepository.insert({
id: this.idService.gen(),
followerId: requestUser.id,
followeeId: targetChannel.id,
});
try {
await this.channelFollowingsRepository.insert({
id: this.idService.gen(),
followerId: requestUser.id,
followeeId: targetChannel.id,
});
} catch (e) {
if (isDuplicateKeyValueError(e)) {
throw new IdentifiableError('6e335e39-0203-4418-a936-b3f2dc987845', 'already following');
}
throw e;
}
this.globalEventService.publishInternalEvent('followChannel', {
userId: requestUser.id,
+90 -114
View File
@@ -16,11 +16,20 @@ import { UserEntityService } from '@/core/entities/UserEntityService.js';
import { bindThis } from '@/decorators.js';
import { FeaturedService } from '@/core/FeaturedService.js';
import { UtilityService } from '@/core/UtilityService.js';
import { isDuplicateKeyValueError } from '@/misc/is-duplicate-key-value-error.js';
import Logger from '../logger.js';
const logger = new Logger('hashtag/create');
type AttachedOrMentioned = 'attached' | 'mentioned';
type UpdatingHashtagColumn = {
totalUserIds: keyof MiHashtag & `${AttachedOrMentioned}UserIds`,
totalUsersCount: keyof MiHashtag & `${AttachedOrMentioned}UsersCount`,
localUserIds: keyof MiHashtag & `${AttachedOrMentioned}LocalUserIds`,
localUsersCount: keyof MiHashtag & `${AttachedOrMentioned}LocalUsersCount`,
remoteUserIds: keyof MiHashtag & `${AttachedOrMentioned}RemoteUserIds`,
remoteUsersCount: keyof MiHashtag & `${AttachedOrMentioned}RemoteUsersCount`,
};
@Injectable()
export class HashtagService {
constructor(
@@ -68,126 +77,93 @@ export class HashtagService {
// TODO: サンプリング
this.updateHashtagsRanking(tag, user.id);
{
const index = await this.hashtagsRepository.findOneBy({ name: tag });
const column: UpdatingHashtagColumn = isUserAttached ? {
totalUserIds: 'attachedUserIds',
totalUsersCount: 'attachedUsersCount',
localUserIds: 'attachedLocalUserIds',
localUsersCount: 'attachedLocalUsersCount',
remoteUserIds: 'attachedRemoteUserIds',
remoteUsersCount: 'attachedRemoteUsersCount',
} : {
totalUserIds: 'mentionedUserIds',
totalUsersCount: 'mentionedUsersCount',
localUserIds: 'mentionedLocalUserIds',
localUsersCount: 'mentionedLocalUsersCount',
remoteUserIds: 'mentionedRemoteUserIds',
remoteUsersCount: 'mentionedRemoteUsersCount',
};
if (index == null && inc) {
try {
if (isUserAttached) {
await this.hashtagsRepository.insert({
id: this.idService.gen(),
name: tag,
mentionedUserIds: [],
mentionedUsersCount: 0,
mentionedLocalUserIds: [],
mentionedLocalUsersCount: 0,
mentionedRemoteUserIds: [],
mentionedRemoteUsersCount: 0,
attachedUserIds: [user.id],
attachedUsersCount: 1,
attachedLocalUserIds: this.userEntityService.isLocalUser(user) ? [user.id] : [],
attachedLocalUsersCount: this.userEntityService.isLocalUser(user) ? 1 : 0,
attachedRemoteUserIds: this.userEntityService.isRemoteUser(user) ? [user.id] : [],
attachedRemoteUsersCount: this.userEntityService.isRemoteUser(user) ? 1 : 0,
} as MiHashtag);
} else {
await this.hashtagsRepository.insert({
id: this.idService.gen(),
name: tag,
mentionedUserIds: [user.id],
mentionedUsersCount: 1,
mentionedLocalUserIds: this.userEntityService.isLocalUser(user) ? [user.id] : [],
mentionedLocalUsersCount: this.userEntityService.isLocalUser(user) ? 1 : 0,
mentionedRemoteUserIds: this.userEntityService.isRemoteUser(user) ? [user.id] : [],
mentionedRemoteUsersCount: this.userEntityService.isRemoteUser(user) ? 1 : 0,
attachedUserIds: [],
attachedUsersCount: 0,
attachedLocalUserIds: [],
attachedLocalUsersCount: 0,
attachedRemoteUserIds: [],
attachedRemoteUsersCount: 0,
} as MiHashtag);
}
return;
} catch (err) {
if (isDuplicateKeyValueError(err)) {
logger.info(`Duplicate insertion detected. Falling back to update. #${tag}`);
} else {
throw err;
}
}
}
if (inc) {
await this.#incrementHashTag(user, tag, column);
} else {
await this.#decrementHashTag(user, tag, column);
}
}
async #incrementHashTag(
user: { id: MiUser['id']; host: MiUser['host']; },
tag: string,
columns: UpdatingHashtagColumn,
) {
const isLocal = this.userEntityService.isLocalUser(user);
const { totalUserIds, totalUsersCount } = columns;
const localOrRemoteUserIds = isLocal ? columns.localUserIds : columns.remoteUserIds;
const localOrRemoteUserCount = isLocal ? columns.localUsersCount : columns.remoteUsersCount;
const runner = this.db.createQueryRunner('master');
try {
await runner.query(
`INSERT into "hashtag"("id", "name", "${totalUserIds}", "${totalUsersCount}", "${localOrRemoteUserIds}",
"${localOrRemoteUserCount}")
VALUES ($3, $1, ARRAY [$2], 1, ARRAY [$2], 1)
ON CONFLICT ("name")
DO UPDATE SET "${totalUserIds}" = ${appendUserIdIfNotExists(totalUserIds)},
"${totalUsersCount}" = ${incrementCountIfNotExists(totalUserIds, totalUsersCount)},
"${localOrRemoteUserIds}" = ${appendUserIdIfNotExists(localOrRemoteUserIds)},
"${localOrRemoteUserCount}" = ${incrementCountIfNotExists(localOrRemoteUserIds, localOrRemoteUserCount)}`,
[tag, user.id, this.idService.gen()],
);
} finally {
await runner.release();
}
await this.db.transaction(async transactionalEntityManager => {
const transactionalHashtagRepository = transactionalEntityManager
.getRepository(MiHashtag);
function appendUserIdIfNotExists(userIds: keyof MiHashtag & `${string}UserIds`): string {
return `CASE WHEN NOT ("hashtag"."${userIds}" @> ARRAY[$2 ::varchar]) THEN array_append("hashtag"."${userIds}", $2) ELSE "hashtag"."${userIds}" END`;
}
const index = await transactionalHashtagRepository
.createQueryBuilder()
.setLock('pessimistic_write')
.where('name = :name', { name: tag })
.getOne();
function incrementCountIfNotExists(userIds: keyof MiHashtag & `${string}UserIds`, userCount: keyof MiHashtag & `${string}UsersCount`): string {
return `CASE WHEN NOT ("hashtag"."${userIds}" @> ARRAY[$2 ::varchar]) THEN "hashtag"."${userCount}" + 1 ELSE "hashtag"."${userCount}" END`;
}
}
if (index == null) return;
async #decrementHashTag(
user: { id: MiUser['id']; host: MiUser['host']; },
tag: string,
columns: UpdatingHashtagColumn,
) {
const isLocal = this.userEntityService.isLocalUser(user);
const { totalUserIds, totalUsersCount } = columns;
const localOrRemoteUserIds = isLocal ? columns.localUserIds : columns.remoteUserIds;
const localOrRemoteUserCount = isLocal ? columns.localUsersCount : columns.remoteUsersCount;
const set = {} as any;
const runner = this.db.createQueryRunner('master');
try {
await runner.query(
`UPDATE "hashtag"
SET "${totalUserIds}" = array_remove("${totalUserIds}", $2),
"${totalUsersCount}" = ${decrementIfExists(totalUserIds, totalUsersCount)},
"${localOrRemoteUserIds}" = array_remove("${localOrRemoteUserIds}", $2),
"${localOrRemoteUserCount}" = ${decrementIfExists(localOrRemoteUserIds, localOrRemoteUserCount)}
WHERE "name" = $1`,
[tag, user.id],
);
} finally {
await runner.release();
}
if (isUserAttached) {
if (inc) {
// 自分が初めてこのタグを使ったなら
if (!index.attachedUserIds.some(id => id === user.id)) {
set.attachedUserIds = () => `array_append("attachedUserIds", '${user.id}')`;
set.attachedUsersCount = () => '"attachedUsersCount" + 1';
}
// 自分が(ローカル内で)初めてこのタグを使ったなら
if (this.userEntityService.isLocalUser(user) && !index.attachedLocalUserIds.some(id => id === user.id)) {
set.attachedLocalUserIds = () => `array_append("attachedLocalUserIds", '${user.id}')`;
set.attachedLocalUsersCount = () => '"attachedLocalUsersCount" + 1';
}
// 自分が(リモートで)初めてこのタグを使ったなら
if (this.userEntityService.isRemoteUser(user) && !index.attachedRemoteUserIds.some(id => id === user.id)) {
set.attachedRemoteUserIds = () => `array_append("attachedRemoteUserIds", '${user.id}')`;
set.attachedRemoteUsersCount = () => '"attachedRemoteUsersCount" + 1';
}
} else {
set.attachedUserIds = () => `array_remove("attachedUserIds", '${user.id}')`;
set.attachedUsersCount = () => '"attachedUsersCount" - 1';
if (this.userEntityService.isLocalUser(user)) {
set.attachedLocalUserIds = () => `array_remove("attachedLocalUserIds", '${user.id}')`;
set.attachedLocalUsersCount = () => '"attachedLocalUsersCount" - 1';
} else {
set.attachedRemoteUserIds = () => `array_remove("attachedRemoteUserIds", '${user.id}')`;
set.attachedRemoteUsersCount = () => '"attachedRemoteUsersCount" - 1';
}
}
} else {
// 自分が初めてこのタグを使ったなら
if (!index.mentionedUserIds.some(id => id === user.id)) {
set.mentionedUserIds = () => `array_append("mentionedUserIds", '${user.id}')`;
set.mentionedUsersCount = () => '"mentionedUsersCount" + 1';
}
// 自分が(ローカル内で)初めてこのタグを使ったなら
if (this.userEntityService.isLocalUser(user) && !index.mentionedLocalUserIds.some(id => id === user.id)) {
set.mentionedLocalUserIds = () => `array_append("mentionedLocalUserIds", '${user.id}')`;
set.mentionedLocalUsersCount = () => '"mentionedLocalUsersCount" + 1';
}
// 自分が(リモートで)初めてこのタグを使ったなら
if (this.userEntityService.isRemoteUser(user) && !index.mentionedRemoteUserIds.some(id => id === user.id)) {
set.mentionedRemoteUserIds = () => `array_append("mentionedRemoteUserIds", '${user.id}')`;
set.mentionedRemoteUsersCount = () => '"mentionedRemoteUsersCount" + 1';
}
}
if (Object.keys(set).length > 0) {
await transactionalHashtagRepository
.createQueryBuilder()
.update()
.where('id = :id', { id: index.id })
.set(set)
.execute();
}
});
function decrementIfExists(userIds: keyof MiHashtag & `${string}UserIds`, userCount: keyof MiHashtag & `${string}UsersCount`): string {
return `CASE WHEN ("${userIds}" @> ARRAY[$2]) THEN "${userCount}" - 1 ELSE "${userCount}" END`;
}
}
@bindThis
@@ -19,7 +19,7 @@ import { bindThis } from '@/decorators.js';
import { validateContentTypeSetAsActivityPub } from '@/core/activitypub/misc/validator.js';
import { assertActivityMatchesUrl, FetchAllowSoftFailMask } from '@/core/activitypub/misc/check-against-url.js';
import type { IObject } from '@/core/activitypub/type.js';
import type { Response } from 'node-fetch';
import type { BodyInit, Response } from 'node-fetch';
import type { URL } from 'node:url';
export type HttpRequestSendOptions = {
@@ -311,7 +311,7 @@ export class HttpRequestService {
url: string,
args: {
method?: string,
body?: string,
body?: BodyInit,
headers?: Record<string, string>,
timeout?: number,
size?: number,
@@ -50,6 +50,8 @@ type CreateResourceDeps = {
serviceVersion: string;
};
type InstrumentationInstaller = () => (() => void) | Promise<() => void>;
export class OpenTelemetryAdapter implements TelemetryAdapter {
public constructor(
private readonly deps: OpenTelemetryAdapterDeps,
@@ -111,39 +113,46 @@ export class OpenTelemetryAdapter implements TelemetryAdapter {
// provider操作をdepsに閉じ込め、span wrapper本体をユニットテストしやすくする。
const tracer = provider.getTracer('misskey-backend');
return new OpenTelemetryAdapter({
tracer,
provider,
getActiveSpan: () => trace.getActiveSpan(),
spanStatusCodeError: SpanStatusCode.ERROR,
shutdownTimeout: DEFAULT_SHUTDOWN_TIMEOUT,
shutdownHttpClientInstrumentation: installHttpClientInstrumentation({
return installInstrumentationsWithCleanup([
() => installHttpClientInstrumentation({
tracer,
spanKindClient: SpanKind.CLIENT,
spanStatusCodeError: SpanStatusCode.ERROR,
}),
// pg のrequire hookとioredis diagnostics channelは、Nest moduleの動的importより前に有効化する。
shutdownDatabaseInstrumentation: await installDatabaseInstrumentation(provider, {
() => installDatabaseInstrumentation(provider, {
capturePgSpans: config.capturePgSpans === true,
capturePgStatement: config.capturePgStatement === true,
capturePgConnectionSpans: config.capturePgConnectionSpans === true,
}),
shutdownRedisInstrumentation: installRedisInstrumentation(tracer, SpanKind.CLIENT, SpanStatusCode.ERROR, {
() => installRedisInstrumentation(tracer, SpanKind.CLIENT, SpanStatusCode.ERROR, {
captureConnectionSpans: config.captureRedisConnectionSpans === true,
captureCommandSpans: config.captureRedisCommandSpans === true,
requireParentSpan: config.captureRedisRootSpans !== true,
}),
queueTraceContext: {
], ([
shutdownHttpClientInstrumentation,
shutdownDatabaseInstrumentation,
shutdownRedisInstrumentation,
]) => new OpenTelemetryAdapter({
tracer,
propagation,
trace,
getActiveContext: () => context.active(),
rootContext: ROOT_CONTEXT,
mode: getQueueTraceContextMode(config.jobTraceContextMode),
provider,
getActiveSpan: () => trace.getActiveSpan(),
spanStatusCodeError: SpanStatusCode.ERROR,
},
});
shutdownTimeout: DEFAULT_SHUTDOWN_TIMEOUT,
shutdownHttpClientInstrumentation,
shutdownDatabaseInstrumentation,
shutdownRedisInstrumentation,
queueTraceContext: {
tracer,
propagation,
trace,
getActiveContext: () => context.active(),
rootContext: ROOT_CONTEXT,
mode: getQueueTraceContextMode(config.jobTraceContextMode),
spanStatusCodeError: SpanStatusCode.ERROR,
},
}));
}
public captureMessage(message: string, _opts: TelemetryCaptureMessageOptions): void {
@@ -192,9 +201,11 @@ export class OpenTelemetryAdapter implements TelemetryAdapter {
}
public async shutdown(): Promise<void> {
this.deps.shutdownHttpClientInstrumentation?.();
this.deps.shutdownDatabaseInstrumentation?.();
this.deps.shutdownRedisInstrumentation?.();
shutdownInstrumentations([
this.deps.shutdownHttpClientInstrumentation,
this.deps.shutdownDatabaseInstrumentation,
this.deps.shutdownRedisInstrumentation,
]);
// BatchSpanProcessorのflushが詰まってもプロセス終了を妨げないよう、上限時間を設ける。
// タイムアウト側のtimerは、flushが先に終わった場合にイベントループを無駄に引き留めないようclearする。
let timer: NodeJS.Timeout | undefined;
@@ -209,6 +220,32 @@ export class OpenTelemetryAdapter implements TelemetryAdapter {
}
}
export async function installInstrumentationsWithCleanup<T>(
installers: readonly InstrumentationInstaller[],
create: (shutdowns: ReadonlyArray<() => void>) => T,
): Promise<T> {
const shutdowns: Array<() => void> = [];
try {
for (const install of installers) {
shutdowns.push(await install());
}
return create(shutdowns);
} catch (error) {
shutdownInstrumentations(shutdowns.reverse());
throw error;
}
}
function shutdownInstrumentations(shutdowns: ReadonlyArray<(() => void) | undefined>): void {
for (const shutdown of shutdowns) {
try {
shutdown?.();
} catch {
// instrumentation の停止失敗で、残りの停止処理や provider の flush を妨げない。
}
}
}
export function createResource(config: OtelBackendRuntimeConfig, deps: CreateResourceDeps): Resource {
// resourceを明示指定するとSDKのdefaultResource()は自動付与されなくなる(マージではなく上書き)ため、
// telemetry.sdk.*等の標準属性を失わないよう明示的にmergeする。
@@ -4,6 +4,7 @@
*/
import { channel } from 'node:diagnostics_channel';
import { diag } from '@opentelemetry/api';
import type { ClientRequest, IncomingMessage } from 'node:http';
import type { Span, SpanOptions, SpanStatusCode, Tracer } from '@opentelemetry/api';
@@ -18,6 +19,7 @@ type HttpClientInstrumentationDeps = {
spanKindClient: SpanOptions['kind'];
spanStatusCodeError: SpanStatusCode;
subscribe: (name: string, listener: (message: unknown) => void) => () => void;
reportError?: (error: unknown) => void;
};
type RequestCreatedMessage = { request: ClientRequest };
@@ -31,10 +33,10 @@ type RequestErrorMessage = { request: ClientRequest; error: Error };
export function createHttpClientInstrumentation(deps: HttpClientInstrumentationDeps): () => void {
const spans = new WeakMap<ClientRequest, HttpClientSpan>();
const unsubscribeCreated = deps.subscribe(HTTP_CLIENT_REQUEST_CREATED, (message: unknown) => {
const unsubscribeCreated = deps.subscribe(HTTP_CLIENT_REQUEST_CREATED, guardDiagnosticsListener(deps, (message: unknown) => {
const { request } = message as RequestCreatedMessage;
const { url, host, port } = getRequestDetails(request);
const method = request.method ?? 'GET';
const method = request.method;
const span = deps.tracer.startSpan(method, {
kind: deps.spanKindClient,
attributes: {
@@ -45,9 +47,9 @@ export function createHttpClientInstrumentation(deps: HttpClientInstrumentationD
},
});
spans.set(request, span);
});
}));
const unsubscribeResponseFinish = deps.subscribe(HTTP_CLIENT_RESPONSE_FINISH, (message: unknown) => {
const unsubscribeResponseFinish = deps.subscribe(HTTP_CLIENT_RESPONSE_FINISH, guardDiagnosticsListener(deps, (message: unknown) => {
const { request, response } = message as ResponseFinishMessage;
const span = spans.get(request);
if (span == null) return;
@@ -56,18 +58,16 @@ export function createHttpClientInstrumentation(deps: HttpClientInstrumentationD
if (statusCode != null) {
span.setAttribute('http.response.status_code', statusCode);
}
if (response.httpVersion != null) {
span.setAttribute('network.protocol.version', response.httpVersion);
}
span.setAttribute('network.protocol.version', response.httpVersion);
if (statusCode != null && statusCode >= 400) {
span.setAttribute('error.type', String(statusCode));
span.setStatus({ code: deps.spanStatusCodeError });
}
span.end();
spans.delete(request);
});
}));
const unsubscribeRequestError = deps.subscribe(HTTP_CLIENT_REQUEST_ERROR, (message: unknown) => {
const unsubscribeRequestError = deps.subscribe(HTTP_CLIENT_REQUEST_ERROR, guardDiagnosticsListener(deps, (message: unknown) => {
const { request, error } = message as RequestErrorMessage;
const span = spans.get(request);
if (span == null) return;
@@ -77,7 +77,7 @@ export function createHttpClientInstrumentation(deps: HttpClientInstrumentationD
span.setStatus({ code: deps.spanStatusCodeError });
span.end();
spans.delete(request);
});
}));
return () => {
unsubscribeCreated();
@@ -89,6 +89,7 @@ export function createHttpClientInstrumentation(deps: HttpClientInstrumentationD
export function installHttpClientInstrumentation(deps: Omit<HttpClientInstrumentationDeps, 'subscribe'>): () => void {
return createHttpClientInstrumentation({
...deps,
reportError: deps.reportError ?? (error => diag.error('Failed to process HTTP client diagnostics event.', error)),
subscribe: (name, listener) => {
const diagnosticChannel = channel(name);
diagnosticChannel.subscribe(listener);
@@ -98,20 +99,47 @@ export function installHttpClientInstrumentation(deps: Omit<HttpClientInstrument
}
function getRequestDetails(request: ClientRequest): { url: string; host: string; port: number } {
const protocol = request.protocol ?? 'http:';
const host = request.getHeader('host')?.toString() ?? request.host ?? 'localhost';
const url = new URL(request.path || '/', `${protocol}//${host}`);
// URL 属性には認証情報やクエリ文字列を含めない。
url.username = '';
url.password = '';
url.search = '';
url.hash = '';
let protocol: 'http:' | 'https:' = 'http:';
return {
url: url.toString(),
host: url.hostname,
// URL.port は既定ポートでは空文字列になるため、スキームから補う。
port: url.port === '' ? (url.protocol === 'https:' ? 443 : 80) : Number(url.port),
try {
protocol = request.protocol === 'https:' ? 'https:' : 'http:';
const host = request.getHeader('host')?.toString() ?? request.host;
const url = new URL(request.path || '/', `${protocol}//${host}`);
// URL 属性には認証情報やクエリ文字列を含めない。
url.username = '';
url.password = '';
url.search = '';
url.hash = '';
return {
url: url.toString(),
host: url.hostname,
// URL.port は既定ポートでは空文字列になるため、スキームから補う。
port: url.port === '' ? (url.protocol === 'https:' ? 443 : 80) : Number(url.port),
};
} catch {
return {
url: `${protocol}//localhost/`,
host: 'localhost',
port: protocol === 'https:' ? 443 : 80,
};
}
}
function guardDiagnosticsListener(
deps: Pick<HttpClientInstrumentationDeps, 'reportError'>,
listener: (message: unknown) => void,
): (message: unknown) => void {
return (message) => {
try {
listener(message);
} catch (error) {
try {
deps.reportError?.(error);
} catch {
// diagnostics listener の失敗をアプリケーションへ伝播させない。
}
}
};
}
@@ -4,13 +4,13 @@
*/
import { tracingChannel } from 'node:diagnostics_channel';
import { context, trace } from '@opentelemetry/api';
import { context, diag, trace } from '@opentelemetry/api';
import type { Span, SpanKind, SpanStatusCode, Tracer } from '@opentelemetry/api';
type IORedisCommandContext = {
command: string;
args: string[];
database: number;
database: number | undefined;
serverAddress: string;
serverPort: number | undefined;
};
@@ -39,6 +39,7 @@ type RedisInstrumentationDeps = {
getActiveSpan(): Span | undefined;
spanKindClient: SpanKind;
spanStatusCodeError: SpanStatusCode;
reportError?: (error: unknown) => void;
};
type RedisInstrumentationOptions = {
@@ -65,6 +66,7 @@ export function installRedisInstrumentation(
getActiveSpan: () => trace.getSpan(context.active()),
spanKindClient,
spanStatusCodeError,
reportError: error => diag.error('Failed to process Redis diagnostics event.', error),
}, options);
}
@@ -77,9 +79,9 @@ export function createRedisInstrumentation(deps: RedisInstrumentationDeps, optio
name: message.command,
attributes: {
'db.system.name': 'redis',
'db.namespace': message.database.toString(10),
'db.operation.name': message.command,
'server.address': message.serverAddress,
...(message.database != null ? { 'db.namespace': message.database.toString(10) } : {}),
...(message.serverPort != null ? { 'server.port': message.serverPort } : {}),
},
}));
@@ -138,7 +140,7 @@ function createTracingChannelSubscribers<T extends object>(
};
const subscribers: TracingChannelSubscribers<T> = {
start: (message) => {
start: guardTracingSubscriber(deps, (message: T) => {
if (requireParentSpan && deps.getActiveSpan() == null) return;
const options = getSpanOptions(message);
@@ -147,24 +149,41 @@ function createTracingChannelSubscribers<T extends object>(
attributes: options.attributes,
});
spans.set(message, { span, recordedError: false });
},
}),
// Promiseを返すコマンドでは完了前にもendイベントが来るため、同期例外だけここで閉じる。
end: (message) => {
end: guardTracingSubscriber(deps, (message: T & { error?: unknown }) => {
if (message.error != null) finish(message);
},
}),
asyncStart: () => {},
asyncEnd: finish,
error: (message) => {
asyncEnd: guardTracingSubscriber(deps, finish),
error: guardTracingSubscriber(deps, (message: T & { error: unknown }) => {
const state = spans.get(message);
if (state == null) return;
recordError(state, message.error);
},
}),
};
channel.subscribe(subscribers);
return subscribers;
}
function guardTracingSubscriber<T>(
deps: Pick<RedisInstrumentationDeps, 'reportError'>,
subscriber: (message: T) => void,
): (message: T) => void {
return (message) => {
try {
subscriber(message);
} catch (error) {
try {
deps.reportError?.(error);
} catch {
// diagnostics subscriber の失敗をアプリケーションへ伝播させない。
}
}
};
}
function toError(value: unknown): Error {
return value instanceof Error ? value : new Error(String(value));
}
@@ -73,7 +73,10 @@ export function startSpanWithTraceContext<T>(name: string, jobData: object, fn:
// Queue context を解釈できる adapter に span 作成を任せる。
// 対応 adapter が無い構成では通常の startSpan へフォールバックする。
const adapter = adapters.find(adapter => adapter.startSpanWithTraceContext != null);
return adapter?.startSpanWithTraceContext?.(name, jobData, fn) ?? startSpan(name, fn);
if (adapter?.startSpanWithTraceContext != null) {
return adapter.startSpanWithTraceContext(name, jobData, fn);
}
return startSpan(name, fn);
}
export async function shutdownTelemetry(): Promise<void> {
@@ -73,7 +73,7 @@ export class PrettyConsoleBackend implements LogBackend {
presentationLevel === 'debug' ? chalk.gray(record.message) :
presentationLevel === 'info' ? record.message :
null;
// 主プロセスは従来どおり「*」、子プロセスはワーカー番号で識別します。
// 主プロセスは「*」で示し、子プロセスはワーカー番号で識別します。
const worker = record.isPrimary ? '*' : record.workerId;
let log = `${label} ${worker}\t[${contexts.join(' ')}]\t${message}`;
+6
View File
@@ -20,6 +20,7 @@ export class MiHashtag {
@Column({
...id(),
default: [],
array: true,
})
public mentionedUserIds: MiUser['id'][];
@@ -32,6 +33,7 @@ export class MiHashtag {
@Column({
...id(),
default: [],
array: true,
})
public mentionedLocalUserIds: MiUser['id'][];
@@ -44,6 +46,7 @@ export class MiHashtag {
@Column({
...id(),
default: [],
array: true,
})
public mentionedRemoteUserIds: MiUser['id'][];
@@ -56,6 +59,7 @@ export class MiHashtag {
@Column({
...id(),
default: [],
array: true,
})
public attachedUserIds: MiUser['id'][];
@@ -68,6 +72,7 @@ export class MiHashtag {
@Column({
...id(),
default: [],
array: true,
})
public attachedLocalUserIds: MiUser['id'][];
@@ -80,6 +85,7 @@ export class MiHashtag {
@Column({
...id(),
default: [],
array: true,
})
public attachedRemoteUserIds: MiUser['id'][];
@@ -10,6 +10,7 @@ import { ApiError } from '@/server/api/error.js';
import type { UsersRepository, UserProfilesRepository, MiMeta } from '@/models/_.js';
import { DI } from '@/di-symbols.js';
import { secureRndstr } from '@/misc/secure-rndstr.js';
import { RoleService } from '@/core/RoleService.js';
import { ModerationLogService } from '@/core/ModerationLogService.js';
export const meta = {
@@ -25,10 +26,10 @@ export const meta = {
code: 'NO_SUCH_USER',
id: 'ccafc7fe-5074-4edd-9dc0-8ef9ef6a701d',
},
cannotResetPasswordOfRootUser: {
message: 'Cannot reset password of the root user.',
code: 'CANNOT_RESET_PASSWORD_OF_ROOT_USER',
id: 'f28fc207-42ca-44c7-a577-44b4f0ec5999',
accessDenied: {
message: 'Access denied.',
code: 'ACCESS_DENIED',
id: 'cda8f8ce-89a6-4f92-8055-33bbe0c1464d',
},
},
@@ -66,6 +67,7 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint-
@Inject(DI.userProfilesRepository)
private userProfilesRepository: UserProfilesRepository,
private roleService: RoleService,
private moderationLogService: ModerationLogService,
) {
super(meta, paramDef, async (ps, me) => {
@@ -75,8 +77,8 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint-
throw new ApiError(meta.errors.noSuchUser);
}
if (this.serverSettings.rootUserId === user.id) {
throw new ApiError(meta.errors.cannotResetPasswordOfRootUser);
if (await this.roleService.isAdministrator(user) && me.id !== user.id) {
throw new ApiError(meta.errors.accessDenied);
}
const passwd = secureRndstr(8);
@@ -11,6 +11,7 @@ import { MiUserProfile } from '@/models/UserProfile.js';
import { MiUserSecurityKey } from '@/models/UserSecurityKey.js';
import type { UsersRepository } from '@/models/_.js';
import { DI } from '@/di-symbols.js';
import { RoleService } from '@/core/RoleService.js';
import { ModerationLogService } from '@/core/ModerationLogService.js';
export const meta = {
@@ -26,6 +27,11 @@ export const meta = {
code: 'NO_SUCH_USER',
id: 'ccafc7fe-5074-4edd-9dc0-8ef9ef6a701d',
},
accessDenied: {
message: 'Access denied.',
code: 'ACCESS_DENIED',
id: 'cda8f8ce-89a6-4f92-8055-33bbe0c1464d',
},
},
} as const;
@@ -46,6 +52,7 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint-
@Inject(DI.usersRepository)
private usersRepository: UsersRepository,
private roleService: RoleService,
private moderationLogService: ModerationLogService,
) {
super(meta, paramDef, async (ps, me) => {
@@ -55,6 +62,10 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint-
throw new ApiError(meta.errors.noSuchUser);
}
if (await this.roleService.isAdministrator(user) && me.id !== user.id) {
throw new ApiError(meta.errors.accessDenied);
}
await this.db.transaction(async (transactionalEntityManager) => {
// パスキーを全て削除
await transactionalEntityManager.delete(MiUserSecurityKey, { userId: user.id });
@@ -8,6 +8,7 @@ import { Endpoint } from '@/server/api/endpoint-base.js';
import type { ChannelsRepository } from '@/models/_.js';
import { DI } from '@/di-symbols.js';
import { ChannelFollowingService } from '@/core/ChannelFollowingService.js';
import { IdentifiableError } from '@/misc/identifiable-error.js';
import { ApiError } from '../../error.js';
export const meta = {
@@ -25,6 +26,11 @@ export const meta = {
code: 'NO_SUCH_CHANNEL',
id: 'c0031718-d573-4e85-928e-10039f1fbb68',
},
alreadyFollowing: {
message: 'You are already following that channel.',
code: 'ALREADY_FOLLOWING',
id: '7db31665-651e-40c1-8e6e-28e9ad829a2d',
},
},
} as const;
@@ -52,7 +58,14 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint-
throw new ApiError(meta.errors.noSuchChannel);
}
await this.channelFollowingService.follow(me, channel);
try {
await this.channelFollowingService.follow(me, channel);
} catch (e) {
if (e instanceof IdentifiableError) {
if (e.id === '6e335e39-0203-4418-a936-b3f2dc987845') throw new ApiError(meta.errors.alreadyFollowing);
}
throw e;
}
});
}
}
@@ -7,8 +7,11 @@ import Parser from 'rss-parser';
import { Injectable } from '@nestjs/common';
import { Endpoint } from '@/server/api/endpoint-base.js';
import { HttpRequestService } from '@/core/HttpRequestService.js';
import { ApiError } from '../error.js';
const rssParser = new Parser();
const MAX_URL_LENGTH = 8192;
const MAX_RESPONSE_SIZE = 1024 * 1024;
const MAX_CONCURRENT_REQUESTS = 32;
export const meta = {
tags: ['meta'],
@@ -17,6 +20,34 @@ export const meta = {
allowGet: true,
cacheSec: 60 * 3,
limit: {
duration: 60 * 1000,
max: 300,
},
errors: {
invalidUrl: {
message: 'Invalid URL.',
code: 'INVALID_URL',
id: '89b7ee05-ccfc-4bdd-9b13-61172fd1e06c',
httpStatusCode: 400,
},
fetchRssFailed: {
message: 'Failed to fetch RSS.',
code: 'FETCH_RSS_FAILED',
id: '8db5d3d8-31d7-452f-b0cc-ca3b8925de12',
kind: 'server',
httpStatusCode: 422,
},
fetchRssUnavailable: {
message: 'RSS fetching is temporarily unavailable.',
code: 'FETCH_RSS_UNAVAILABLE',
id: '91e6ff44-c63f-4725-9ad0-b7a40d7f7655',
kind: 'server',
httpStatusCode: 503,
},
},
res: {
type: 'object',
properties: {
@@ -215,21 +246,84 @@ export const paramDef = {
@Injectable()
export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint-disable-line import/no-default-export
private readonly inFlightRequests = new Map<string, Promise<Awaited<ReturnType<Parser['parseString']>>>>();
private activeRequestCount = 0;
constructor(
private httpRequestService: HttpRequestService,
) {
super(meta, paramDef, async (ps, me) => {
const res = await this.httpRequestService.send(ps.url, {
method: 'GET',
headers: {
Accept: 'application/rss+xml, */*',
},
timeout: 5000,
});
super(meta, paramDef, async (ps) => {
const url = this.normalizeUrl(ps.url);
const inFlightRequest = this.inFlightRequests.get(url);
if (inFlightRequest != null) {
return await inFlightRequest;
}
const text = await res.text();
if (this.activeRequestCount >= MAX_CONCURRENT_REQUESTS) {
throw new ApiError(meta.errors.fetchRssUnavailable);
}
return rssParser.parseString(text);
this.activeRequestCount++;
const request = this.fetchRss(url)
.catch(() => {
throw new ApiError(meta.errors.fetchRssFailed);
})
.finally(() => {
this.inFlightRequests.delete(url);
this.activeRequestCount--;
});
this.inFlightRequests.set(url, request);
return await request;
});
}
private normalizeUrl(input: string): string {
if (input.length === 0 || input.length > MAX_URL_LENGTH) {
throw new ApiError(meta.errors.invalidUrl);
}
let url: URL;
try {
url = new URL(input);
} catch {
throw new ApiError(meta.errors.invalidUrl);
}
if (
(url.protocol !== 'http:' && url.protocol !== 'https:') ||
url.username !== '' ||
url.password !== ''
) {
throw new ApiError(meta.errors.invalidUrl);
}
url.hash = '';
return url.href;
}
private async fetchRss(url: string): Promise<Awaited<ReturnType<Parser['parseString']>>> {
const res = await this.httpRequestService.send(url, {
method: 'GET',
headers: {
Accept: 'application/rss+xml, */*',
},
timeout: 5000,
size: MAX_RESPONSE_SIZE,
});
const finalUrl = new URL(res.url);
if (finalUrl.protocol !== 'http:' && finalUrl.protocol !== 'https:') {
throw new Error('Invalid final URL protocol');
}
const text = await res.text();
const rssParser = new Parser({
xml2js: {
async: true,
},
});
return await rssParser.parseString(text);
}
}
+36
View File
@@ -0,0 +1,36 @@
/*
* SPDX-FileCopyrightText: syuilo and misskey-project
* SPDX-License-Identifier: AGPL-3.0-only
*/
process.env.NODE_ENV = 'test';
import * as assert from 'assert';
import { describe, beforeAll, test } from 'vitest';
import { api, castAsError, signup } from '../utils.js';
import type * as misskey from 'misskey-js';
describe('Channel', () => {
let alice: misskey.entities.SignupResponse;
beforeAll(async () => {
alice = await signup({ username: 'alice' });
});
describe('Follow', () => {
let channel: misskey.entities.ChannelsCreateResponse;
beforeAll(async () => {
const res = await api('channels/create', { name: 'follow-test-channel' }, alice);
channel = res.body;
});
test('フォローしているチャンネルを再度フォローするとALREADY_FOLLOWINGエラーになる', async () => {
const res1 = await api('channels/follow', { channelId: channel.id }, alice);
assert.strictEqual(res1.status, 204);
const res2 = await api('channels/follow', { channelId: channel.id }, alice);
assert.strictEqual(res2.status, 400);
assert.strictEqual(castAsError(res2.body as any).error.code, 'ALREADY_FOLLOWING');
});
});
});
@@ -55,6 +55,7 @@ describe('Redis telemetry instrumentation', () => {
}, { connection, prefix });
worker.once('completed', () => resolve());
worker.once('failed', (_job, error) => reject(error));
worker.on('error', reject);
});
await tracer.startActiveSpan('HTTP POST /telemetry-test', async httpSpan => {
@@ -78,7 +79,7 @@ describe('Redis telemetry instrumentation', () => {
expect(Object.values(span.attributes)).not.toContain(secret);
}
} finally {
await worker?.close();
await worker?.close(true);
await queue.obliterate({ force: true });
await queue.close();
uninstall();
+32 -33
View File
@@ -4,19 +4,12 @@
*/
import { describe, test, expect, vi, beforeEach } from 'vitest';
import type { AiService as AiServiceType, Prediction } from '@/core/AiService.js';
import type { MiMeta } from '@/models/_.js';
import type { HttpRequestService } from '@/core/HttpRequestService.js';
import type { LoggerService } from '@/core/LoggerService.js';
import { AiService, type Prediction } from '@/core/AiService.js';
// AiService が直接 import している node-fetch をモックして、外部サービスへの送信内容と
// レスポンス解釈を検証する。
const { fetchMock } = vi.hoisted(() => ({ fetchMock: vi.fn() }));
vi.mock('node-fetch', () => ({ default: fetchMock }));
const { AiService } = await import('@/core/AiService.js');
let getAgentByUrlMock: ReturnType<typeof vi.fn>;
const sendMock = vi.fn();
const DEFAULT_META = {
sensitiveMediaDetectionApiUrl: 'http://localhost:3009' as string | null,
@@ -25,10 +18,9 @@ const DEFAULT_META = {
sensitiveMediaDetectionMaxImagesPerRequest: 4,
};
function makeService(metaOverrides: Partial<typeof DEFAULT_META> = {}): AiServiceType {
function makeService(metaOverrides: Partial<typeof DEFAULT_META> = {}): AiService {
const meta = { ...DEFAULT_META, ...metaOverrides } as unknown as MiMeta;
getAgentByUrlMock = vi.fn(() => undefined);
const httpRequestService = { getAgentByUrl: getAgentByUrlMock } as unknown as HttpRequestService;
const httpRequestService = { send: sendMock } as unknown as HttpRequestService;
const loggerService = {
getLogger: () => ({ warn: () => {}, error: () => {}, info: () => {} }),
} as unknown as LoggerService;
@@ -52,11 +44,11 @@ const buf = (s: string) => Buffer.from(s);
describe('AiService', () => {
beforeEach(() => {
fetchMock.mockReset();
sendMock.mockReset();
});
test('正常: 送信順を保った予測値配列を返す', async () => {
fetchMock.mockResolvedValue(okResponse([
sendMock.mockResolvedValue(okResponse([
{ success: true, predictions: neutral() },
{ success: true, predictions: [{ className: 'Porn', probability: 0.8 }] },
]));
@@ -66,30 +58,35 @@ describe('AiService', () => {
[{ className: 'Neutral', probability: 0.99 }],
[{ className: 'Porn', probability: 0.8 }],
]);
expect(fetchMock).toHaveBeenCalledTimes(1);
expect(fetchMock.mock.calls[0][0]).toBe('http://localhost:3009/v1/detect-images');
expect(sendMock).toHaveBeenCalledTimes(1);
expect(sendMock.mock.calls[0][0]).toBe('http://localhost:3009/v1/detect-images');
});
test('外部サービス: 通常の outbound agent を使用する', async () => {
fetchMock.mockResolvedValue(okResponse([{ success: true, predictions: neutral() }]));
test('外部サービス: HttpRequestService を使用する', async () => {
sendMock.mockResolvedValue(okResponse([{ success: true, predictions: neutral() }]));
const svc = makeService({ sensitiveMediaDetectionApiUrl: 'https://detector.example.com' });
await svc.detectSensitiveMany([buf('a')]);
const requestOptions = fetchMock.mock.calls[0][1] as { agent: (url: URL) => unknown };
requestOptions.agent(new URL('https://detector.example.com/v1/detect-images'));
expect(getAgentByUrlMock).toHaveBeenCalledWith(expect.any(URL));
expect(sendMock).toHaveBeenCalledWith('https://detector.example.com/v1/detect-images', {
method: 'POST',
headers: {},
body: expect.any(FormData),
timeout: 5000,
}, {
throwErrorWhenResponseNotOk: false,
});
});
test('detectSensitive: 単一画像はバッチの先頭を返す', async () => {
fetchMock.mockResolvedValue(okResponse([{ success: true, predictions: neutral() }]));
sendMock.mockResolvedValue(okResponse([{ success: true, predictions: neutral() }]));
const svc = makeService();
const res = await svc.detectSensitive(buf('a'));
expect(res).toEqual(neutral());
});
test('部分失敗: 失敗パーツのみ null になる', async () => {
fetchMock.mockResolvedValue(okResponse([
sendMock.mockResolvedValue(okResponse([
{ success: true, predictions: neutral() },
{ success: false, error: { code: 'IMAGE_DECODE_FAILED', message: 'x' } },
]));
@@ -100,14 +97,14 @@ describe('AiService', () => {
});
test('非200: チャンク全件 null(例外を投げない)', async () => {
fetchMock.mockResolvedValue({ ok: false, status: 503, statusText: 'Service Unavailable', json: async () => ({}) });
sendMock.mockResolvedValue({ ok: false, status: 503, statusText: 'Service Unavailable', json: async () => ({}) });
const svc = makeService();
const res = await svc.detectSensitiveMany([buf('a'), buf('b')]);
expect(res).toEqual([null, null]);
});
test('通信エラー: チャンク全件 null(例外を投げない)', async () => {
fetchMock.mockRejectedValue(new Error('network down'));
sendMock.mockRejectedValue(new Error('network down'));
const svc = makeService();
const res = await svc.detectSensitiveMany([buf('a')]);
expect(res).toEqual([null]);
@@ -117,11 +114,11 @@ describe('AiService', () => {
const svc = makeService({ sensitiveMediaDetectionApiUrl: null });
const res = await svc.detectSensitiveMany([buf('a'), buf('b')]);
expect(res).toEqual([null, null]);
expect(fetchMock).not.toHaveBeenCalled();
expect(sendMock).not.toHaveBeenCalled();
});
test('チャンク分割: maxImagesPerRequest ごとに順次送信する', async () => {
fetchMock.mockResolvedValue(okResponse([
sendMock.mockResolvedValue(okResponse([
{ success: true, predictions: neutral() },
{ success: true, predictions: neutral() },
{ success: true, predictions: neutral() },
@@ -130,22 +127,24 @@ describe('AiService', () => {
const svc = makeService({ sensitiveMediaDetectionMaxImagesPerRequest: 2 });
const res = await svc.detectSensitiveMany([buf('a'), buf('b'), buf('c'), buf('d'), buf('e')]);
// 5 枚を 2 枚ずつ → 3 リクエスト、結果は順序を保って 5 件。
expect(fetchMock).toHaveBeenCalledTimes(3);
expect(sendMock).toHaveBeenCalledTimes(3);
expect(res).toHaveLength(5);
expect(res.every(x => x != null)).toBe(true);
});
test('APIキー設定時のみ Authorization: Bearer を付与する', async () => {
fetchMock.mockResolvedValue(okResponse([{ success: true, predictions: neutral() }]));
sendMock.mockResolvedValue(okResponse([{ success: true, predictions: neutral() }]));
const withKey = makeService({ sensitiveMediaDetectionApiKey: 'secret' });
await withKey.detectSensitiveMany([buf('a')]);
expect((fetchMock.mock.calls[0][1] as any).headers.Authorization).toBe('Bearer secret');
const withKeyHeaders = (sendMock.mock.calls[0][1] as { headers: Record<string, string> }).headers;
expect(withKeyHeaders.Authorization).toBe('Bearer secret');
fetchMock.mockClear();
fetchMock.mockResolvedValue(okResponse([{ success: true, predictions: neutral() }]));
sendMock.mockClear();
sendMock.mockResolvedValue(okResponse([{ success: true, predictions: neutral() }]));
const withoutKey = makeService();
await withoutKey.detectSensitiveMany([buf('a')]);
expect((fetchMock.mock.calls[0][1] as any).headers.Authorization).toBeUndefined();
const withoutKeyHeaders = (sendMock.mock.calls[0][1] as { headers: Record<string, string> }).headers;
expect(withoutKeyHeaders.Authorization).toBeUndefined();
});
});
@@ -3,13 +3,13 @@
* SPDX-License-Identifier: AGPL-3.0-only
*/
import { beforeEach, describe, expect, test, vi } from 'vitest';
import { afterEach, beforeEach, describe, expect, test, vi } from 'vitest';
import { SpanStatusCode } from '@opentelemetry/api';
import { defaultResource, detectResources, envDetector, resourceFromAttributes } from '@opentelemetry/resources';
import { ParentBasedSampler, TraceIdRatioBasedSampler } from '@opentelemetry/sdk-trace-base';
import { ATTR_SERVICE_INSTANCE_ID, ATTR_SERVICE_NAME, ATTR_SERVICE_VERSION } from '@opentelemetry/semantic-conventions';
import type { Context, SpanContext } from '@opentelemetry/api';
import { OpenTelemetryAdapter, createResource, createSampler, getMisskeyProcessRole } from '@/core/telemetry/adapters/OpenTelemetryAdapter.js';
import { OpenTelemetryAdapter, createResource, createSampler, getMisskeyProcessRole, installInstrumentationsWithCleanup } from '@/core/telemetry/adapters/OpenTelemetryAdapter.js';
const mocks = vi.hoisted(() => {
return {
@@ -40,6 +40,10 @@ const samplerDeps = {
};
describe('OpenTelemetryAdapter', () => {
afterEach(() => {
vi.useRealTimers();
});
test('wraps async work in an active span and ends it after success', async () => {
const span = {
end: vi.fn(),
@@ -216,7 +220,6 @@ describe('OpenTelemetryAdapter', () => {
await vi.advanceTimersByTimeAsync(50);
await expect(shutdown).resolves.toBeUndefined();
vi.useRealTimers();
});
test('clears the shutdown timeout timer once provider.shutdown() resolves first', async () => {
@@ -234,7 +237,30 @@ describe('OpenTelemetryAdapter', () => {
expect(clearTimeoutSpy).toHaveBeenCalled();
clearTimeoutSpy.mockRestore();
vi.useRealTimers();
});
test('continues shutdown and flushes the provider when instrumentation cleanup fails', async () => {
const shutdownHttpClientInstrumentation = vi.fn(() => { throw new Error('failed'); });
const shutdownDatabaseInstrumentation = vi.fn();
const shutdownRedisInstrumentation = vi.fn();
const provider = { shutdown: vi.fn().mockResolvedValue(undefined) };
const adapter = new OpenTelemetryAdapter({
tracer: { startActiveSpan: vi.fn() },
provider,
getActiveSpan: () => undefined,
spanStatusCodeError: SpanStatusCode.ERROR,
shutdownTimeout: 10,
shutdownHttpClientInstrumentation,
shutdownDatabaseInstrumentation,
shutdownRedisInstrumentation,
});
await expect(adapter.shutdown()).resolves.toBeUndefined();
expect(shutdownHttpClientInstrumentation).toHaveBeenCalledOnce();
expect(shutdownDatabaseInstrumentation).toHaveBeenCalledOnce();
expect(shutdownRedisInstrumentation).toHaveBeenCalledOnce();
expect(provider.shutdown).toHaveBeenCalledOnce();
});
test('captureMessage starts a standalone span to report the error when there is no active span', () => {
@@ -271,6 +297,38 @@ describe('OpenTelemetryAdapter', () => {
});
});
describe('installInstrumentationsWithCleanup', () => {
test('cleans up acquired instrumentations in reverse order and preserves the installation error', async () => {
const installationError = new Error('failed to install');
const shutdownHttpClientInstrumentation = vi.fn();
const shutdownDatabaseInstrumentation = vi.fn(() => { throw new Error('failed to clean up'); });
await expect(installInstrumentationsWithCleanup([
() => shutdownHttpClientInstrumentation,
() => shutdownDatabaseInstrumentation,
() => { throw installationError; },
], () => { throw new Error('should not create'); })).rejects.toBe(installationError);
expect(shutdownDatabaseInstrumentation).toHaveBeenCalledOnce();
expect(shutdownHttpClientInstrumentation).toHaveBeenCalledOnce();
expect(shutdownDatabaseInstrumentation.mock.invocationCallOrder[0]).toBeLessThan(shutdownHttpClientInstrumentation.mock.invocationCallOrder[0]);
});
test('cleans up all instrumentations when adapter construction fails', async () => {
const constructionError = new Error('failed to create adapter');
const shutdownHttpClientInstrumentation = vi.fn();
const shutdownDatabaseInstrumentation = vi.fn();
await expect(installInstrumentationsWithCleanup([
() => shutdownHttpClientInstrumentation,
() => shutdownDatabaseInstrumentation,
], () => { throw constructionError; })).rejects.toBe(constructionError);
expect(shutdownDatabaseInstrumentation).toHaveBeenCalledOnce();
expect(shutdownHttpClientInstrumentation).toHaveBeenCalledOnce();
});
});
describe('createSampler', () => {
test('accepts sample rates within [0, 1]', () => {
expect(() => createSampler(0, samplerDeps)).not.toThrow();
@@ -4,9 +4,11 @@
*/
import { describe, expect, test, vi } from 'vitest';
import { normalizeSentryBackendConfig } from '@/config.js';
import { SentryTelemetryAdapter, buildSentryIntegrations, buildSentryNodeOptions, buildSentryOtlpInitOptions } from '@/core/telemetry/adapters/SentryTelemetryAdapter.js';
type TestIntegration = Parameters<ReturnType<typeof buildSentryIntegrations>>[0][number];
type TestSpanProcessor = NonNullable<ReturnType<typeof buildSentryOtlpInitOptions>['openTelemetrySpanProcessors']>[number];
function testIntegration(name: string): TestIntegration {
return { name };
@@ -79,43 +81,59 @@ describe('SentryTelemetryAdapter', () => {
});
test('builds Sentry options that export spans to both Sentry and OTLP', () => {
const existingProcessor = { name: 'existingProcessor' };
const existingProcessor = { name: 'existingProcessor' } as unknown as TestSpanProcessor;
const otlpProcessor = { name: 'otlpProcessor' };
const result = buildSentryOtlpInitOptions({
sentryConfig: {
enableNodeProfiling: false,
disabledIntegrations: ['Redis'],
const result = buildSentryOtlpInitOptions({
sentryConfig: {
enableNodeProfiling: false,
disabledIntegrations: ['Redis'],
options: {
openTelemetrySpanProcessors: [existingProcessor as any],
openTelemetrySpanProcessors: [existingProcessor],
tracesSampleRate: 0.25,
},
},
otelConfig: { serviceVersion: '2026.1.0' },
otlpProcessor,
});
},
otelConfig: { serviceVersion: '2026.1.0' },
otlpProcessor,
});
expect(result.tracesSampleRate).toBe(0.25);
expect(result.openTelemetrySpanProcessors).toEqual([existingProcessor, otlpProcessor]);
// OTel併存時もremoteへtrace headerを漏らさないデフォルトはSentry単体時と揃える。
expect(result.tracePropagationTargets).toEqual([]);
expect((result.integrations as any)([
expect(result.integrations).toBeTypeOf('function');
if (typeof result.integrations !== 'function') throw new Error('Expected integrations to be a function');
expect(result.integrations([
testIntegration('Http'),
testIntegration('Redis'),
testIntegration('Postgres'),
]).map((integration: TestIntegration) => integration.name)).toEqual(['Http', 'Postgres']);
});
test('uses safe defaults when Sentry options are omitted', () => {
const otlpProcessor = { name: 'otlpProcessor' };
const result = buildSentryOtlpInitOptions({
sentryConfig: normalizeSentryBackendConfig({
enableNodeProfiling: false,
}),
otelConfig: { serviceVersion: '2026.1.0' },
otlpProcessor,
});
expect(result.tracePropagationTargets).toEqual([]);
expect(result.openTelemetrySpanProcessors).toEqual([otlpProcessor]);
});
test('does not disable Sentry trace propagation when explicitly enabled for OTel coexistence', () => {
const result = buildSentryOtlpInitOptions({
sentryConfig: {
enableNodeProfiling: false,
options: {},
},
otelConfig: {
serviceVersion: '2026.1.0',
propagateTraceToRemote: true,
},
otelConfig: {
serviceVersion: '2026.1.0',
propagateTraceToRemote: true,
},
otlpProcessor: { name: 'otlpProcessor' },
});
@@ -130,9 +148,9 @@ describe('SentryTelemetryAdapter', () => {
tracePropagationTargets: ['^https://internal\\.example/'],
},
},
otelConfig: { serviceVersion: '2026.1.0' },
otlpProcessor: { name: 'otlpProcessor' },
});
otelConfig: { serviceVersion: '2026.1.0' },
otlpProcessor: { name: 'otlpProcessor' },
});
expect(result.tracePropagationTargets).toEqual(['^https://internal\\.example/']);
});
@@ -57,7 +57,7 @@ describe('http-client-instrumentation', () => {
expect(span.setAttribute).toHaveBeenCalledWith('network.protocol.version', '1.1');
expect(span.end).toHaveBeenCalledTimes(1);
unsubscribe();
expect(listeners).toHaveLength(0);
expect(listeners.size).toBe(0);
});
test('records a request error and ends the span once', () => {
@@ -105,4 +105,84 @@ describe('http-client-instrumentation', () => {
expect(span.setAttribute).toHaveBeenCalledWith('error.type', '502');
expect(span.setStatus).toHaveBeenCalledWith({ code: SpanStatusCode.ERROR });
});
test('uses safe request details when the request URL cannot be constructed', () => {
const listeners = new Map<string, (message: unknown) => void>();
const span = { end: vi.fn(), recordException: vi.fn(), setAttribute: vi.fn(), setStatus: vi.fn() };
const tracer = { startSpan: vi.fn(() => span) };
createHttpClientInstrumentation({
tracer: tracer as any,
spanKindClient: SpanKind.CLIENT,
spanStatusCodeError: SpanStatusCode.ERROR,
subscribe: (name, listener) => {
listeners.set(name, listener);
return () => listeners.delete(name);
},
});
const clientRequest = {
...request(),
host: '::1',
getHeader: vi.fn(() => undefined),
};
listeners.get('http.client.request.created')!({ request: clientRequest });
expect(tracer.startSpan).toHaveBeenCalledWith('POST', {
kind: SpanKind.CLIENT,
attributes: {
'http.request.method': 'POST',
'url.full': 'https://localhost/',
'server.address': 'localhost',
'server.port': 443,
},
});
});
test('does not propagate errors from diagnostics listeners', () => {
const listeners = new Map<string, (message: unknown) => void>();
const reportError = vi.fn();
const responseError = new Error('response instrumentation failed');
const requestError = new Error('request instrumentation failed');
const startError = new Error('span creation failed');
const span = {
end: vi.fn(),
recordException: vi.fn(() => { throw requestError; }),
setAttribute: vi.fn(() => { throw responseError; }),
setStatus: vi.fn(),
};
const tracer = {
startSpan: vi.fn()
.mockReturnValueOnce(span)
.mockReturnValueOnce(span)
.mockImplementationOnce(() => { throw startError; }),
};
createHttpClientInstrumentation({
tracer: tracer as any,
spanKindClient: SpanKind.CLIENT,
spanStatusCodeError: SpanStatusCode.ERROR,
subscribe: (name, listener) => {
listeners.set(name, listener);
return () => listeners.delete(name);
},
reportError,
});
const responseRequest = request();
const errorRequest = request();
listeners.get('http.client.request.created')!({ request: responseRequest });
expect(() => listeners.get('http.client.response.finish')!({
request: responseRequest,
response: { statusCode: 200 },
})).not.toThrow();
listeners.get('http.client.request.created')!({ request: errorRequest });
expect(() => listeners.get('http.client.request.error')!({
request: errorRequest,
error: new Error('connection failed'),
})).not.toThrow();
expect(() => listeners.get('http.client.request.created')!({ request: request() })).not.toThrow();
expect(reportError).toHaveBeenCalledWith(responseError);
expect(reportError).toHaveBeenCalledWith(requestError);
expect(reportError).toHaveBeenCalledWith(startError);
});
});
@@ -4,11 +4,12 @@
*/
import { describe, expect, test, vi } from 'vitest';
import { SpanStatusCode } from '@opentelemetry/api';
import type { Context, SpanContext } from '@opentelemetry/api';
import { getQueueSpanContext, getQueueTraceContextMode, injectActiveTraceContext, injectQueueTraceContext } from '@/core/telemetry/queue-trace-context.js';
import { executeSpan, getQueueSpanContext, getQueueTraceContextMode, injectActiveTraceContext, injectQueueTraceContext, recordSpanError, startSpanWithQueueTraceContext } from '@/core/telemetry/queue-trace-context.js';
const rootContext = {} as Context;
const extractedContext = {} as Context;
const rootContext = { kind: 'root' } as unknown as Context;
const extractedContext = { kind: 'extracted' } as unknown as Context;
const sourceSpanContext: SpanContext = {
traceId: '0123456789abcdef0123456789abcdef',
spanId: '0123456789abcdef',
@@ -94,6 +95,8 @@ describe('queue-trace-context', () => {
},
parentContext: rootContext,
});
expect(result?.parentContext).toBe(rootContext);
expect(getSpanContext).toHaveBeenCalledWith(extractedContext);
});
test('uses the extracted context as the parent when parent mode is selected', () => {
@@ -108,6 +111,7 @@ describe('queue-trace-context', () => {
options: {},
parentContext: extractedContext,
});
expect(result?.parentContext).toBe(extractedContext);
});
test('ignores malformed or missing carriers', () => {
@@ -129,4 +133,101 @@ describe('queue-trace-context', () => {
expect(getQueueTraceContextMode('parent')).toBe('parent');
expect(() => getQueueTraceContextMode('children')).toThrow('otelForBackend.jobTraceContextMode');
});
test('executes synchronous work and ends the span once', () => {
const span = { end: vi.fn(), recordException: vi.fn(), setStatus: vi.fn() };
expect(executeSpan(span as any, () => 'ok', SpanStatusCode.ERROR)).toBe('ok');
expect(span.end).toHaveBeenCalledOnce();
expect(span.recordException).not.toHaveBeenCalled();
});
test('waits for resolved Promise work before ending the span', async () => {
const span = { end: vi.fn(), recordException: vi.fn(), setStatus: vi.fn() };
let resolveWork: ((value: string) => void) | undefined;
const work = new Promise<string>(resolve => {
resolveWork = resolve;
});
const result = executeSpan(span as any, () => work, SpanStatusCode.ERROR);
expect(span.end).not.toHaveBeenCalled();
if (resolveWork == null) throw new Error('work resolver was not initialized');
resolveWork('ok');
await expect(result).resolves.toBe('ok');
expect(span.end).toHaveBeenCalledOnce();
expect(span.recordException).not.toHaveBeenCalled();
});
test('records and propagates synchronous failures', () => {
const span = { end: vi.fn(), recordException: vi.fn(), setStatus: vi.fn() };
const error = new Error('boom');
expect(() => executeSpan(span as any, () => { throw error; }, SpanStatusCode.ERROR)).toThrow(error);
expect(span.recordException).toHaveBeenCalledWith(error);
expect(span.setStatus).toHaveBeenCalledWith({ code: SpanStatusCode.ERROR, message: error.message });
expect(span.end).toHaveBeenCalledOnce();
});
test('records and propagates rejected Promise work', async () => {
const span = { end: vi.fn(), recordException: vi.fn(), setStatus: vi.fn() };
const error = new Error('boom');
await expect(executeSpan(span as any, () => Promise.reject(error), SpanStatusCode.ERROR)).rejects.toBe(error);
expect(span.recordException).toHaveBeenCalledWith(error);
expect(span.setStatus).toHaveBeenCalledWith({ code: SpanStatusCode.ERROR, message: error.message });
expect(span.end).toHaveBeenCalledOnce();
});
test('normalizes non-Error failures before recording them', () => {
const span = { recordException: vi.fn(), setStatus: vi.fn() };
recordSpanError(span as any, 'boom', SpanStatusCode.ERROR);
expect(span.recordException).toHaveBeenCalledWith(expect.any(Error));
expect(span.recordException).toHaveBeenCalledWith(expect.objectContaining({ message: 'boom' }));
expect(span.setStatus).toHaveBeenCalledWith({ code: SpanStatusCode.ERROR, message: 'boom' });
});
test('starts a span with the extracted queue context', () => {
const span = { end: vi.fn(), recordException: vi.fn(), setStatus: vi.fn() };
const startActiveSpan = vi.fn((_name: string, _options: unknown, _context: unknown, fn: (spanArg: typeof span) => string) => fn(span));
const getSpanContext = vi.fn(() => sourceSpanContext);
const deps = {
tracer: { startActiveSpan } as any,
rootContext,
propagation: { inject: vi.fn(), extract: () => extractedContext } as any,
trace: { getSpanContext },
getActiveContext: () => rootContext,
mode: 'link' as const,
spanStatusCodeError: SpanStatusCode.ERROR,
};
expect(startSpanWithQueueTraceContext(deps, 'Queue: Deliver', jobData(), () => 'ok', () => 'fallback')).toBe('ok');
expect(startActiveSpan).toHaveBeenCalledWith('Queue: Deliver', {
root: true,
links: [{ context: sourceSpanContext }],
}, rootContext, expect.any(Function));
expect(getSpanContext).toHaveBeenCalledOnce();
expect(getSpanContext).toHaveBeenCalledWith(extractedContext);
expect(span.end).toHaveBeenCalledOnce();
});
test('uses the fallback without starting a span when queue context is missing', () => {
const startActiveSpan = vi.fn();
const fallback = vi.fn(() => 'fallback');
const deps = {
tracer: { startActiveSpan } as any,
rootContext,
propagation: { inject: vi.fn(), extract: vi.fn() } as any,
trace: { getSpanContext: vi.fn() },
getActiveContext: () => rootContext,
mode: 'link' as const,
spanStatusCodeError: SpanStatusCode.ERROR,
};
expect(startSpanWithQueueTraceContext(deps, 'Queue: Deliver', {}, () => 'ok', fallback)).toBe('fallback');
expect(fallback).toHaveBeenCalledOnce();
expect(startActiveSpan).not.toHaveBeenCalled();
});
});
@@ -0,0 +1,192 @@
/*
* SPDX-FileCopyrightText: syuilo and misskey-project
* SPDX-License-Identifier: AGPL-3.0-only
*/
import { beforeEach, describe, expect, test, vi } from 'vitest';
import { HttpRequestService } from '@/core/HttpRequestService.js';
import FetchRssEndpoint, { meta } from '@/server/api/endpoints/fetch-rss.js';
import { ApiError } from '@/server/api/error.js';
import type { Mocked } from 'vitest';
import type { Response } from 'node-fetch';
const rssParserMocks = vi.hoisted(() => ({
constructor: vi.fn(),
parseString: vi.fn(),
}));
vi.mock('rss-parser', () => ({
default: class {
constructor(options: unknown) {
rssParserMocks.constructor(options);
}
public parseString(input: string) {
return rssParserMocks.parseString(input);
}
},
}));
const RSS = '<?xml version="1.0"?><rss version="2.0"><channel><title>Test</title></channel></rss>';
function deferred<T>() {
let resolve!: (value: T) => void;
let reject!: (reason?: unknown) => void;
const promise = new Promise<T>((res, rej) => {
resolve = res;
reject = rej;
});
return { promise, resolve, reject };
}
function response(url: string, text = RSS): Response {
return {
url,
text: vi.fn().mockResolvedValue(text),
} as unknown as Response;
}
describe('fetch-rss endpoint', () => {
let httpRequestService: Mocked<HttpRequestService>;
let endpoint: FetchRssEndpoint;
beforeEach(() => {
rssParserMocks.constructor.mockReset();
rssParserMocks.parseString.mockReset();
rssParserMocks.parseString.mockResolvedValue({ items: [] });
httpRequestService = {
send: vi.fn(),
} as unknown as Mocked<HttpRequestService>;
endpoint = new FetchRssEndpoint(httpRequestService);
});
async function exec(url: string) {
return await endpoint.exec({ url }, null, null);
}
async function expectApiError(promise: Promise<unknown>, code: string, status: number) {
await expect(promise).rejects.toMatchObject({
code,
httpStatusCode: status,
info: undefined,
});
}
test.each([
'',
'not a URL',
'ftp://example.com/feed.xml',
`https://example.com/${'a'.repeat(8192)}`,
])('rejects invalid URL: %s', async (url) => {
await expectApiError(exec(url), 'INVALID_URL', 400);
expect(httpRequestService.send).not.toHaveBeenCalled();
});
test.each([
'https://user@example.com/feed.xml',
'https://user:password@example.com/feed.xml',
])('rejects URL containing credentials: %s', async (url) => {
await expectApiError(exec(url), 'INVALID_URL', 400);
expect(httpRequestService.send).not.toHaveBeenCalled();
});
test('does not expose details from internal errors', async () => {
httpRequestService.send.mockRejectedValue(new Error('secret upstream details'));
const promise = exec('https://example.com/feed.xml');
await expectApiError(promise, 'FETCH_RSS_FAILED', 422);
await expect(promise).rejects.not.toMatchObject({
message: expect.stringContaining('secret upstream details'),
});
});
test('passes the 1 MiB response limit to HttpRequestService', async () => {
httpRequestService.send.mockResolvedValue(response('https://example.com/feed.xml'));
await exec('https://example.com/feed.xml');
expect(httpRequestService.send).toHaveBeenCalledWith('https://example.com/feed.xml', expect.objectContaining({
size: 1024 * 1024,
}));
});
test('creates an asynchronous RSS parser for every request', async () => {
httpRequestService.send
.mockResolvedValueOnce(response('https://example.com/first.xml'))
.mockResolvedValueOnce(response('https://example.com/second.xml'));
await exec('https://example.com/first.xml');
await exec('https://example.com/second.xml');
expect(rssParserMocks.constructor).toHaveBeenCalledTimes(2);
expect(rssParserMocks.constructor).toHaveBeenNthCalledWith(1, { xml2js: { async: true } });
expect(rssParserMocks.constructor).toHaveBeenNthCalledWith(2, { xml2js: { async: true } });
});
test('rejects a non-HTTP final URL without exposing it', async () => {
httpRequestService.send.mockResolvedValue(response('file:///secret/feed.xml'));
await expectApiError(exec('https://example.com/feed.xml'), 'FETCH_RSS_FAILED', 422);
});
test('shares an in-flight request for the same normalized URL', async () => {
const pending = deferred<Response>();
httpRequestService.send.mockReturnValue(pending.promise);
const first = exec('HTTPS://EXAMPLE.COM:443/feed.xml#first');
const second = exec('https://example.com/feed.xml#second');
expect(httpRequestService.send).toHaveBeenCalledTimes(1);
pending.resolve(response('https://example.com/feed.xml'));
await expect(Promise.all([first, second])).resolves.toHaveLength(2);
expect(httpRequestService.send).toHaveBeenCalledTimes(1);
});
test('limits concurrent requests for different URLs to 32', async () => {
const pending = deferred<Response>();
httpRequestService.send.mockReturnValue(pending.promise);
const requests = Array.from({ length: 32 }, (_, i) => exec(`https://example.com/${i}.xml`));
expect(httpRequestService.send).toHaveBeenCalledTimes(32);
await expectApiError(exec('https://example.com/overflow.xml'), 'FETCH_RSS_UNAVAILABLE', 503);
expect(httpRequestService.send).toHaveBeenCalledTimes(32);
pending.resolve(response('https://example.com/feed.xml'));
await Promise.all(requests);
httpRequestService.send.mockResolvedValue(response('https://example.com/after.xml'));
await expect(exec('https://example.com/after.xml')).resolves.toBeDefined();
});
test('cleans up the in-flight request and concurrency slot after success', async () => {
httpRequestService.send.mockResolvedValue(response('https://example.com/feed.xml'));
await exec('https://example.com/feed.xml');
await exec('https://example.com/feed.xml');
expect(httpRequestService.send).toHaveBeenCalledTimes(2);
});
test('cleans up the in-flight request and concurrency slot after failure', async () => {
httpRequestService.send.mockRejectedValue(new Error('upstream failed'));
const requests = Array.from({ length: 32 }, (_, i) => exec(`https://example.com/${i}.xml`));
await Promise.allSettled(requests);
httpRequestService.send.mockResolvedValue(response('https://example.com/0.xml'));
await expect(exec('https://example.com/0.xml')).resolves.toBeDefined();
expect(httpRequestService.send).toHaveBeenCalledTimes(33);
});
test('has the expected rate limit metadata', () => {
expect(meta.limit).toEqual({
duration: 60 * 1000,
max: 300,
});
});
test('uses only the declared structured API errors', () => {
expect(new ApiError(meta.errors.invalidUrl)).toMatchObject({ code: 'INVALID_URL', httpStatusCode: 400 });
expect(new ApiError(meta.errors.fetchRssFailed)).toMatchObject({ code: 'FETCH_RSS_FAILED', httpStatusCode: 422 });
expect(new ApiError(meta.errors.fetchRssUnavailable)).toMatchObject({ code: 'FETCH_RSS_UNAVAILABLE', httpStatusCode: 503 });
});
});
@@ -3,9 +3,15 @@
* SPDX-License-Identifier: AGPL-3.0-only
*/
import { createRequire } from 'node:module';
import { context, propagation, trace } from '@opentelemetry/api';
import { NodeTracerProvider } from '@opentelemetry/sdk-trace-node';
import { describe, expect, test, vi } from 'vitest';
import type { Span as SdkSpan, SpanProcessor } from '@opentelemetry/sdk-trace-base';
import { installDatabaseInstrumentation, installInstrumentation } from '@/core/telemetry/database-instrumentation.js';
const backendRequire = createRequire(import.meta.url);
describe('database-instrumentation', () => {
test('does not install PostgreSQL instrumentation when disabled', async () => {
const uninstall = await installDatabaseInstrumentation({} as any, {
@@ -55,6 +61,76 @@ describe('database-instrumentation', () => {
expect(pg.disable).toHaveBeenCalledOnce();
});
test('redacts SQL attributes after pg instrumentation records the statement', async () => {
const statement = 'SELECT very_secret_literal';
const attributeWrites: Array<{ key: string; value: unknown }> = [];
let attributesAtStart: Record<string, unknown> | undefined;
let pgSpan: SdkSpan | undefined;
const spanProcessor: SpanProcessor = {
forceFlush: async () => {},
onStart: span => {
if (span.instrumentationScope.name !== '@opentelemetry/instrumentation-pg') return;
pgSpan = span;
attributesAtStart = { ...span.attributes };
const setAttribute = span.setAttribute.bind(span);
span.setAttribute = (key, value) => {
attributeWrites.push({ key, value });
return setAttribute(key, value);
};
},
onEnd: () => {},
shutdown: async () => {},
};
const provider = new NodeTracerProvider({ spanProcessors: [spanProcessor] });
provider.register();
let uninstall: (() => void) | undefined;
try {
uninstall = await installDatabaseInstrumentation(provider, {
capturePgSpans: true,
capturePgStatement: false,
capturePgConnectionSpans: false,
});
const { Client } = backendRequire('pg') as typeof import('pg');
const tracer = provider.getTracer('database-instrumentation-test');
tracer.startActiveSpan('parent', parentSpan => {
try {
// pg instrumentation runs before an unconnected client queues the query,
// so no database is required.
new Client().query(statement, () => {});
} finally {
parentSpan.end();
}
});
const recordedPgSpan = pgSpan;
expect(recordedPgSpan).toBeDefined();
if (recordedPgSpan == null) throw new Error('PostgreSQL span was not started.');
expect(attributesAtStart).not.toHaveProperty('db.statement');
expect(attributesAtStart).not.toHaveProperty('db.query.text');
const rawSqlIndex = attributeWrites.findIndex(({ key, value }) =>
(key === 'db.statement' || key === 'db.query.text') && value === statement);
expect(rawSqlIndex).toBeGreaterThanOrEqual(0);
const rawSqlKey = attributeWrites[rawSqlIndex].key;
expect(attributeWrites.findIndex(({ key, value }, index) =>
index > rawSqlIndex && key === rawSqlKey && value === '[REDACTED]')).toBeGreaterThan(rawSqlIndex);
expect(recordedPgSpan.attributes['db.statement']).toBe('[REDACTED]');
expect(recordedPgSpan.attributes['db.query.text']).toBe('[REDACTED]');
expect(Object.values(recordedPgSpan.attributes)).not.toContain(statement);
} finally {
pgSpan?.end();
uninstall?.();
await provider.shutdown();
trace.disable();
context.disable();
propagation.disable();
}
});
test('keeps SQL statement attributes when explicitly enabled', () => {
const pg = { setTracerProvider: vi.fn(), enable: vi.fn(), disable: vi.fn() };
const config = vi.fn();
@@ -95,7 +171,7 @@ describe('database-instrumentation', () => {
}));
});
test('cleans up both instrumentations when initialization fails', () => {
test('cleans up the pg instrumentation when initialization fails', () => {
const pg = { setTracerProvider: vi.fn(), enable: vi.fn(), disable: vi.fn() };
pg.enable.mockImplementation(() => { throw new Error('failed'); });
@@ -145,4 +145,65 @@ describe('redis-instrumentation', () => {
expect(tracingChannel).not.toHaveBeenCalled();
});
test('omits the database attribute when diagnostics context has no database', () => {
let subscribers: any;
const span = { end: vi.fn(), recordException: vi.fn(), setStatus: vi.fn(), setAttribute: vi.fn() };
const tracer = { startSpan: vi.fn(() => span) };
createRedisInstrumentation({
tracingChannel: () => ({ subscribe: (value) => { subscribers = value; }, unsubscribe: vi.fn() }),
tracer: tracer as any,
getActiveSpan: () => ({}) as any,
spanKindClient: SpanKind.CLIENT,
spanStatusCodeError: SpanStatusCode.ERROR,
}, { captureCommandSpans: true });
subscribers.start({ command: 'get', args: ['key'], database: undefined, serverAddress: 'redis', serverPort: 6379 });
expect(tracer.startSpan).toHaveBeenCalledWith('get', expect.objectContaining({
attributes: expect.not.objectContaining({
'db.namespace': expect.anything(),
}),
}));
});
test('does not propagate errors from Redis tracing subscribers', () => {
const subscribers = new Map<string, any>();
const reportError = vi.fn();
const endError = new Error('span end failed');
const startError = new Error('span creation failed');
const span = {
end: vi.fn(() => { throw endError; }),
recordException: vi.fn(),
setStatus: vi.fn(),
setAttribute: vi.fn(),
};
const tracer = {
startSpan: vi.fn()
.mockReturnValueOnce(span)
.mockImplementationOnce(() => { throw startError; }),
};
createRedisInstrumentation({
tracingChannel: (name) => ({
subscribe: (value) => { subscribers.set(name, value); },
unsubscribe: vi.fn(),
}),
tracer: tracer as any,
getActiveSpan: () => ({}) as any,
spanKindClient: SpanKind.CLIENT,
spanStatusCodeError: SpanStatusCode.ERROR,
reportError,
}, {
captureCommandSpans: true,
captureConnectionSpans: true,
});
const command = { command: 'get', args: ['key'], database: 0, serverAddress: 'redis', serverPort: 6379 };
subscribers.get('ioredis:command').start(command);
expect(() => subscribers.get('ioredis:command').asyncEnd(command)).not.toThrow();
expect(() => subscribers.get('ioredis:connect').start({ serverAddress: 'redis', serverPort: 6379 })).not.toThrow();
expect(reportError).toHaveBeenCalledWith(endError);
expect(reportError).toHaveBeenCalledWith(startError);
});
});
@@ -127,6 +127,28 @@ describe('telemetry-registry', () => {
expect(adapterStartSpan).toHaveBeenCalledWith('test', fn);
});
test('startSpanWithTraceContext executes an undefined-returning callback only once', async () => {
const { initTelemetry, startSpanWithTraceContext } = await import('@/core/telemetry/telemetry-registry.js');
const otelForBackend = { endpoint: 'http://collector:4318/v1/traces' };
const adapterStartSpan = vi.fn((_name: string, fn: () => undefined) => fn());
const adapterStartSpanWithTraceContext = vi.fn((_name: string, _jobData: object, fn: () => undefined) => fn());
mocks.otelCreate.mockResolvedValue({
shutdown: vi.fn(),
captureMessage: vi.fn(),
startSpan: adapterStartSpan,
startSpanWithTraceContext: adapterStartSpanWithTraceContext,
});
await initTelemetry(config({ otelForBackend }));
const jobData = { id: 'job' };
const fn = vi.fn(() => undefined);
expect(startSpanWithTraceContext('test', jobData, fn)).toBeUndefined();
expect(adapterStartSpanWithTraceContext).toHaveBeenCalledWith('test', jobData, fn);
expect(adapterStartSpan).not.toHaveBeenCalled();
expect(fn).toHaveBeenCalledTimes(1);
});
test('startSpan wraps work through multiple registered adapters in order for future adapter combinations', async () => {
const { initTelemetry, startSpan } = await import('@/core/telemetry/telemetry-registry.js');
const calls: string[] = [];
+8
View File
@@ -27,6 +27,14 @@ export function extractDomain(url: string) {
return match ? match[1] : null;
}
export function tryParseUrl(url: string | URL, base?: string | URL): URL | null {
try {
return new URL(url, base);
} catch {
return null;
}
}
export function maybeMakeRelative(urlStr: string, baseStr: string): string {
try {
const baseObj = new URL(baseStr);
@@ -83,16 +83,16 @@ SPDX-License-Identifier: AGPL-3.0-only
v-else-if="content.type === 'video'"
ref="videoEl"
data-gallery-click-action="video"
:class="$style.content"
:class="[$style.video, { [$style.videoSized]: videoAspectRatio != null }]"
:src="content.url"
:alt="content.file?.comment ?? undefined"
draggable="false"
:controls="prefer.s.useNativeUiForVideoAudioPlayer"
playsinline
@loadedmetadata="originalContentLoaded = true"
@loadedmetadata="onVideoLoadedMetadata"
@click.stop="onVideoClick"
></video>
<div v-if="content.type === 'video' && !prefer.s.useNativeUiForVideoAudioPlayer && !isVideoPlaying" data-gallery-click-action="video" :class="$style.playIconWrapper">
<div v-if="content.type === 'video' && !prefer.s.useNativeUiForVideoAudioPlayer && !isVideoPlaying" :class="$style.playIconWrapper">
<div :class="$style.playIcon">
<i class="ti ti-player-play"></i>
</div>
@@ -224,6 +224,21 @@ const isVideoPlaying = computed(() => videoControl.value?.isPlaying ?? false);
const isVideoActuallyPlaying = computed(() => videoControl.value?.isActuallyPlaying ?? false);
let canOpenAnimation = false;
const videoAspectRatio = ref<number | null>(
props.content.width != null && props.content.height != null && props.content.width > 0 && props.content.height > 0
? props.content.width / props.content.height
: null
);
function onVideoLoadedMetadata() {
originalContentLoaded.value = true;
//
if (videoAspectRatio.value != null) return;
if (videoEl.value == null || videoEl.value.videoWidth === 0 || videoEl.value.videoHeight === 0) return;
videoAspectRatio.value = videoEl.value.videoWidth / videoEl.value.videoHeight;
}
const headerSize = 30;
const footerSize = props.content.type === 'video' && !prefer.s.useNativeUiForVideoAudioPlayer ? 80 : 0;
@@ -941,6 +956,25 @@ defineExpose({
object-fit: contain;
}
.video {
display: block;
user-select: none;
position: absolute;
top: 50%;
left: 50%;
translate: -50% -50%;
width: 100%;
height: 100%;
object-fit: contain;
}
.videoSized {
width: min(100cqw, calc(100cqh * v-bind("videoAspectRatio ?? 16 / 9")));
height: auto;
background-color: #000;
aspect-ratio: v-bind("videoAspectRatio ?? 16 / 9");
}
.loading {
position: absolute;
top: 0;
@@ -969,6 +1003,8 @@ defineExpose({
position: relative;
width: 100%;
height: 100%;
// .videoSized使100cqw / 100cqh (= padding)
container-type: size;
transition: scale 200ms ease, opacity 200ms ease !important;
}
@@ -985,6 +1021,7 @@ defineExpose({
height: 100%;
display: grid;
place-items: center;
pointer-events: none;
}
.playIcon {
@@ -1000,8 +1037,8 @@ defineExpose({
transition: scale 100ms ease;
}
.playIconWrapper:hover .playIcon,
.playIcon:hover {
// hovervideo
.video:hover ~ .playIconWrapper .playIcon {
scale: 1.2;
}
@@ -49,7 +49,7 @@ SPDX-License-Identifier: AGPL-3.0-only
</template>
<script lang="ts" setup>
import { computed, onUnmounted, watch } from 'vue';
import { computed } from 'vue';
import { isLink } from '@@/js/is-link.js';
import type { UploaderItem } from '@/composables/use-uploader.js';
import { getUploadName } from '@/composables/use-uploader.js';
@@ -57,6 +57,7 @@ import { i18n } from '@/i18n.js';
import MkButton from '@/components/MkButton.vue';
import bytes from '@/filters/bytes.js';
import * as os from '@/os.js';
import type { Content } from '@/components/MkLightbox.item.vue';
const props = defineProps<{
items: UploaderItem[];
@@ -72,32 +73,6 @@ const emit = defineEmits<{
(ev: 'showMenuViaContextmenu', item: UploaderItem, event: PointerEvent): void;
}>();
//#region objectUrlMap
const objectUrlMap = new Map<UploaderItem, string>();
watch(() => props.items, () => {
for (const item of props.items) {
if (!objectUrlMap.has(item)) {
objectUrlMap.set(item, URL.createObjectURL(item.file));
}
}
for (const [item, url] of objectUrlMap.entries()) {
if (!props.items.includes(item)) {
URL.revokeObjectURL(url);
objectUrlMap.delete(item);
}
}
}, { immediate: true, deep: true });
onUnmounted(() => {
for (const url of objectUrlMap.values()) {
URL.revokeObjectURL(url);
}
objectUrlMap.clear();
});
//#endregion
function getUploadNameParts(item: UploaderItem): {
baseName: string;
extension: string | null;
@@ -127,13 +102,15 @@ function onContextmenu(item: UploaderItem, ev: PointerEvent) {
async function onThumbnailClick(item: UploaderItem, ev: PointerEvent) {
if (item.file.type.startsWith('image') || item.file.type.startsWith('video')) {
const contents = props.items.filter(item => item.file.type.startsWith('image') || item.file.type.startsWith('video')).map(item => ({
id: item.id,
type: (item.file.type.startsWith('video') ? 'video' as const : 'image' as const),
url: objectUrlMap.get(item)!,
thumbnailUrl: item.thumbnail,
filename: getUploadName(item),
}));
const contents = props.items
.filter(item => item.file.type.startsWith('image') || item.file.type.startsWith('video'))
.map<Content>(item => ({
id: item.id,
type: (item.file.type.startsWith('video') ? 'video' : 'image'),
url: item.objectUrl,
thumbnailUrl: item.thumbnail,
filename: getUploadName(item),
}));
const { dispose } = await os.popupAsyncWithDialog(import('@/components/MkLightbox.vue').then(x => x.default), {
defaultIndex: contents.findIndex(content => content.id === item.id),
@@ -16,6 +16,7 @@ import { i18n } from '@/i18n.js';
import { prefer } from '@/preferences.js';
import { isWebpSupported } from '@/utility/isWebpSupported.js';
import { uploadFile, UploadAbortedError } from '@/utility/drive.js';
import type { Content } from '@/components/MkLightbox.item.vue';
import * as os from '@/os.js';
import { ensureSignin } from '@/i.js';
@@ -74,6 +75,7 @@ export type UploaderItem = {
compressedSize?: number | null;
preprocessedFile?: Blob | null;
file: File;
objectUrl: string;
watermarkPreset: WatermarkPreset | null;
watermarkLayers: WatermarkLayers | null;
imageFrameParams: ImageFrameParams | null;
@@ -133,12 +135,13 @@ export function useUploader(options: {
const filename = file.name ?? 'untitled';
const extension = filename.split('.').length > 1 ? '.' + filename.split('.').pop() : '';
const watermarkPreset = uploaderFeatures.value.watermark && $i.policies.watermarkAvailable ? (prefer.s.watermarkPresets.find(p => p.id === prefer.s.defaultWatermarkPresetId) ?? null) : null;
const objectUrl = window.URL.createObjectURL(file);
items.value.push({
id,
name: prefer.s.keepOriginalFilename ? filename : id + extension,
suffix: '',
progress: null,
thumbnail: THUMBNAIL_SUPPORTED_TYPES.includes(file.type) ? window.URL.createObjectURL(file) : null,
thumbnail: THUMBNAIL_SUPPORTED_TYPES.includes(file.type) ? objectUrl : null,
preprocessing: false,
preprocessProgress: null,
uploading: false,
@@ -150,6 +153,7 @@ export function useUploader(options: {
watermarkLayers: watermarkPreset?.layers ?? null,
imageFrameParams: null,
file: markRaw(file),
objectUrl,
});
const reactiveItem = items.value.at(-1)!;
preprocess(reactiveItem).then(() => {
@@ -163,8 +167,24 @@ export function useUploader(options: {
}
}
function removeItem(item: UploaderItem) {
function revokeItemObjectUrls(item: UploaderItem) {
if (item.thumbnail != null) URL.revokeObjectURL(item.thumbnail);
URL.revokeObjectURL(item.objectUrl);
}
function createItemObjectUrl(item: UploaderItem, file: Blob | File): string {
revokeItemObjectUrls(item);
return window.URL.createObjectURL(file);
}
function updateItemObjectUrls(item: UploaderItem, file: Blob | File) {
const newObjectUrl = createItemObjectUrl(item, file);
item.objectUrl = newObjectUrl;
item.thumbnail = THUMBNAIL_SUPPORTED_TYPES.includes(file.type) ? newObjectUrl : null;
}
function removeItem(item: UploaderItem) {
revokeItemObjectUrls(item);
items.value.splice(items.value.indexOf(item), 1);
}
@@ -214,7 +234,35 @@ export function useUploader(options: {
closed: () => dispose(),
});
},
}, {
});
if (item.file.type.startsWith('image/') || item.file.type.startsWith('video/')) {
menu.push({
text: i18n.ts.preview,
icon: 'ti ti-photo-search',
action: async () => {
const contents = items.value
.filter(item => item.file.type.startsWith('image/') || item.file.type.startsWith('video/'))
.map<Content>(item => ({
id: item.id,
type: item.file.type.startsWith('video/') ? 'video' : 'image',
url: item.objectUrl,
thumbnail: item.thumbnail,
filename: getUploadName(item),
caption: item.caption ?? null,
}));
const { dispose } = await os.popupAsyncWithDialog(import('@/components/MkLightbox.vue').then(x => x.default), {
defaultIndex: contents.findIndex(x => x.id === item.id),
contents,
}, {
closed: () => dispose(),
});
},
});
}
menu.push({
type: 'divider',
});
}
@@ -235,11 +283,12 @@ export function useUploader(options: {
text: i18n.ts.cropImage,
action: async () => {
const cropped = await os.cropImageFile(item.file, { aspectRatio: null });
if (item.thumbnail != null) URL.revokeObjectURL(item.thumbnail);
const newObjectUrl = createItemObjectUrl(item, cropped);
items.value.splice(items.value.indexOf(item), 1, {
...item,
file: markRaw(cropped),
thumbnail: window.URL.createObjectURL(cropped),
thumbnail: THUMBNAIL_SUPPORTED_TYPES.includes(cropped.type) ? newObjectUrl : null,
objectUrl: newObjectUrl,
});
const reactiveItem = items.value.find(x => x.id === item.id)!;
preprocess(reactiveItem).then(() => {
@@ -260,11 +309,12 @@ export function useUploader(options: {
image: item.file,
}, {
ok: (file) => {
if (item.thumbnail != null) URL.revokeObjectURL(item.thumbnail);
const newObjectUrl = createItemObjectUrl(item, file);
items.value.splice(items.value.indexOf(item), 1, {
...item,
file: markRaw(file),
thumbnail: window.URL.createObjectURL(file),
thumbnail: THUMBNAIL_SUPPORTED_TYPES.includes(file.type) ? newObjectUrl : null,
objectUrl: newObjectUrl,
});
const reactiveItem = items.value.find(x => x.id === item.id)!;
preprocess(reactiveItem).then(() => {
@@ -694,8 +744,7 @@ export function useUploader(options: {
imageBitmap.close();
if (item.thumbnail != null) URL.revokeObjectURL(item.thumbnail);
item.thumbnail = THUMBNAIL_SUPPORTED_TYPES.includes(preprocessedFile.type) ? window.URL.createObjectURL(preprocessedFile) : null;
updateItemObjectUrls(item, preprocessedFile);
item.preprocessedFile = markRaw(preprocessedFile);
}
@@ -754,14 +803,13 @@ export function useUploader(options: {
item.suffix = '';
}
if (item.thumbnail != null) URL.revokeObjectURL(item.thumbnail);
item.thumbnail = THUMBNAIL_SUPPORTED_TYPES.includes(preprocessedFile.type) ? window.URL.createObjectURL(preprocessedFile) : null;
updateItemObjectUrls(item, preprocessedFile);
item.preprocessedFile = markRaw(preprocessedFile);
}
function reset() {
for (const item of items.value) {
if (item.thumbnail != null) URL.revokeObjectURL(item.thumbnail);
revokeItemObjectUrls(item);
}
abortAll();
@@ -30,6 +30,8 @@ SPDX-License-Identifier: AGPL-3.0-only
import { ref } from 'vue';
import * as Misskey from 'misskey-js';
import { useInterval } from '@@/js/use-interval.js';
import { url as baseUrl } from '@@/js/config.js';
import { tryParseUrl } from '@@/js/url.js';
import MkMarqueeText from '@/components/MkMarqueeText.vue';
import { shuffle } from '@/utility/shuffle.js';
@@ -53,7 +55,11 @@ const tick = () => {
if (props.shuffle) {
shuffle(feed.items);
}
items.value = feed.items;
items.value = feed.items.filter((item) => {
if (!item.link) return false;
const itemUrl = tryParseUrl(item.link, baseUrl);
return itemUrl != null && ['http:', 'https:'].includes(itemUrl.protocol);
});
fetching.value = false;
key.value++;
});
+7 -2
View File
@@ -24,10 +24,11 @@ import { ref, watch, computed } from 'vue';
import * as Misskey from 'misskey-js';
import { url as base } from '@@/js/config.js';
import { useInterval } from '@@/js/use-interval.js';
import { tryParseUrl } from '@@/js/url.js';
import { useWidgetPropsManager } from './widget.js';
import { i18n } from '@/i18n.js';
import type { WidgetComponentEmits, WidgetComponentExpose, WidgetComponentProps } from './widget.js';
import type { FormWithDefault, GetFormResultType } from '@/utility/form.js';
import { i18n } from '@/i18n.js';
import MkContainer from '@/components/MkContainer.vue';
const name = 'rss';
@@ -81,7 +82,11 @@ const tick = () => {
window.fetch(fetchEndpoint.value, {})
.then(res => res.json())
.then((feed: Misskey.entities.FetchRssResponse) => {
rawItems.value = feed.items;
rawItems.value = feed.items.filter((item) => {
if (!item.link) return false;
const itemUrl = tryParseUrl(item.link, base);
return itemUrl != null && ['http:', 'https:'].includes(itemUrl.protocol);
});
fetching.value = false;
});
};
@@ -29,15 +29,16 @@ SPDX-License-Identifier: AGPL-3.0-only
<script lang="ts" setup>
import { ref, watch, computed } from 'vue';
import * as Misskey from 'misskey-js';
import { url as base } from '@@/js/config.js';
import { useInterval } from '@@/js/use-interval.js';
import { tryParseUrl } from '@@/js/url.js';
import { useWidgetPropsManager } from './widget.js';
import type { WidgetComponentEmits, WidgetComponentExpose, WidgetComponentProps } from './widget.js';
import MkMarqueeText from '@/components/MkMarqueeText.vue';
import type { FormWithDefault, GetFormResultType } from '@/utility/form.js';
import MkMarqueeText from '@/components/MkMarqueeText.vue';
import MkContainer from '@/components/MkContainer.vue';
import { shuffle } from '@/utility/shuffle.js';
import { i18n } from '@/i18n.js';
import { url as base } from '@@/js/config.js';
import { useInterval } from '@@/js/use-interval.js';
const name = 'rssTicker';
@@ -121,7 +122,11 @@ const tick = () => {
window.fetch(fetchEndpoint.value, {})
.then(res => res.json())
.then((feed: Misskey.entities.FetchRssResponse) => {
rawItems.value = feed.items;
rawItems.value = feed.items.filter((item) => {
if (!item.link) return false;
const itemUrl = tryParseUrl(item.link, base);
return itemUrl != null && ['http:', 'https:'].includes(itemUrl.protocol);
});
fetching.value = false;
key.value++;
});
+21
View File
@@ -0,0 +1,21 @@
/*
* SPDX-FileCopyrightText: syuilo and misskey-project
* SPDX-License-Identifier: AGPL-3.0-only
*/
import { describe, expect, test } from 'vitest';
import { tryParseUrl } from '@@/js/url.js';
describe('tryParseUrl', () => {
test('parses an absolute URL', () => {
expect(tryParseUrl('https://example.com/path')?.href).toBe('https://example.com/path');
});
test('parses a relative URL against a base URL', () => {
expect(tryParseUrl('/path', 'https://example.com/base')?.href).toBe('https://example.com/path');
});
test('returns null for an invalid URL', () => {
expect(tryParseUrl('https://[invalid')).toBeNull();
});
});
+1 -1
View File
@@ -38,6 +38,6 @@
"tsx": "4.23.0"
},
"dependencies": {
"js-yaml": "5.2.1"
"js-yaml": "5.2.2"
}
}
+1 -1
View File
@@ -1,7 +1,7 @@
{
"type": "module",
"name": "misskey-js",
"version": "2026.7.0-beta.6",
"version": "2026.7.0",
"description": "Misskey SDK for JavaScript",
"license": "MIT",
"main": "./built/index.js",
+9
View File
@@ -21879,6 +21879,15 @@ export interface operations {
'application/json': components['schemas']['Error'];
};
};
/** @description Too many requests */
429: {
headers: {
[name: string]: unknown;
};
content: {
'application/json': components['schemas']['Error'];
};
};
/** @description Internal server error */
500: {
headers: {
+11 -11
View File
@@ -24,8 +24,8 @@ importers:
specifier: 9.0.0
version: 9.0.0
js-yaml:
specifier: 5.2.1
version: 5.2.1
specifier: 5.2.2
version: 5.2.2
tar:
specifier: 7.5.21
version: 7.5.21
@@ -231,8 +231,8 @@ importers:
specifier: 0.220.0
version: 0.220.0(@opentelemetry/api@1.9.1)
'@opentelemetry/instrumentation':
specifier: 0.219.0
version: 0.219.0(@opentelemetry/api@1.9.1)
specifier: 0.220.0
version: 0.220.0(@opentelemetry/api@1.9.1)
'@opentelemetry/instrumentation-pg':
specifier: 0.72.0
version: 0.72.0(@opentelemetry/api@1.9.1)
@@ -592,8 +592,8 @@ importers:
specifier: 10.0.3
version: 10.0.3
js-yaml:
specifier: 5.2.1
version: 5.2.1
specifier: 5.2.2
version: 5.2.2
pid-port:
specifier: 2.1.1
version: 2.1.1
@@ -1230,8 +1230,8 @@ importers:
packages/i18n:
dependencies:
js-yaml:
specifier: 5.2.1
version: 5.2.1
specifier: 5.2.2
version: 5.2.2
devDependencies:
'@types/node':
specifier: 26.1.1
@@ -6500,8 +6500,8 @@ packages:
resolution: {integrity: sha512-1td788aAnnZ5qs7V2QIRl1owjtYpbKt749Y3xauqQgwIIGF/xXWz1wMTEBx5O3LK3lXLVuqXPdPxj2BoFHaW9Q==}
hasBin: true
js-yaml@5.2.1:
resolution: {integrity: sha512-zfLtNfQqxVqq3uaTqSkh4x4hZw3KHobGUA0fJUj4wawW8bsQLTVqpHdXSIzidh7o+4lEW36tANuAGdaFx6Zgnw==}
js-yaml@5.2.2:
resolution: {integrity: sha512-dayzUzKkJ1MkuUtZglSebU43utNXH0OWQByK9rKOOuYIO8M5TV1y+n8ALMdG0rdzBnfNkOmZEqrURepb0ejqBw==}
hasBin: true
jsbn@0.1.1:
@@ -15042,7 +15042,7 @@ snapshots:
dependencies:
argparse: 2.0.1
js-yaml@5.2.1:
js-yaml@5.2.2:
dependencies:
argparse: 2.0.1
+2 -2
View File
@@ -54,14 +54,14 @@ minimumReleaseAgeExclude:
- slacc-win32-x64-msvc
- '@typescript/native-preview*'
- vite # そのうち消す
# Renovate security update: @opentelemetry/core@2.8.0
- "@opentelemetry/core@2.8.0"
# Renovate security update: typeorm@1.1.0
- typeorm@1.1.0
# Renovate security update: @fastify/static@10.1.2
- "@fastify/static@10.1.2"
# Renovate security update: tar@7.5.21
- tar@7.5.21
# Renovate security update: js-yaml@5.2.2
- js-yaml@5.2.2
overrides:
'@aiscript-dev/aiscript-languageserver': '-'
'bullmq>ioredis': 5.11.1