import type { WorkflowDefinition } from "@codemation/core"; import { access, stat } from "node:fs/promises"; import path from "node:path"; import { pathToFileURL } from "node:url"; import type { NamespacedUnregister } from "tsx/esm/api"; import type { CodemationConfig } from "../config/CodemationConfig"; import { CodemationConfigNormalizer } from "../config/CodemationConfigNormalizer"; import type { NormalizedCodemationConfig } from "../config/CodemationConfigNormalizer"; import { BootTimer } from "../../bootstrap/perf/BootTimer"; import { logLevelPolicyFactory } from "../../infrastructure/logging/LogLevelPolicyFactory"; import { ServerLoggerFactory } from "../../infrastructure/logging/ServerLoggerFactory"; import { DiscoveredWorkflowsEmptyMessageFactory } from "./DiscoveredWorkflowsEmptyMessageFactory"; import { CodemationConsumerConfigExportsResolver } from "./CodemationConsumerConfigExportsResolver"; import { WorkflowDefinitionExportsResolver } from "./WorkflowDefinitionExportsResolver"; import { WorkflowDiscoveryPathSegmentsComputer } from "./WorkflowDiscoveryPathSegmentsComputer"; import { WorkflowModulePathFinder } from "./WorkflowModulePathFinder"; export type CodemationConsumerConfigResolution = Readonly<{ config: NormalizedCodemationConfig; bootstrapSource: string | null; workflowSources: ReadonlyArray; }>; type ConsumerImportSession = Readonly<{ shouldResetImporter: boolean; resetCacheKeys: Set; }>; export class CodemationConsumerConfigLoader { private static readonly importerRegistrationsByTsconfig = new Map(); private static readonly importerNamespaceVersionByTsconfig = new Map(); private readonly configExportsResolver = new CodemationConsumerConfigExportsResolver(); private readonly configNormalizer = new CodemationConfigNormalizer(); private readonly workflowModulePathFinder = new WorkflowModulePathFinder(); private readonly workflowDefinitionExportsResolver = new WorkflowDefinitionExportsResolver(); private readonly discoveredWorkflowsEmptyMessageFactory = new DiscoveredWorkflowsEmptyMessageFactory(); private readonly pathSegmentsComputer = new WorkflowDiscoveryPathSegmentsComputer(); private readonly performanceDiagnosticsLogger = new ServerLoggerFactory( logLevelPolicyFactory, ).createPerformanceDiagnostics("codemation-config-loader.timing"); private readonly bootLogger = new ServerLoggerFactory(logLevelPolicyFactory).create("codemation.boot"); private static readonly resolutionCache = new Map>(); static invalidateAll(): void { this.resolutionCache.clear(); } async load( args: Readonly<{ consumerRoot: string; configPathOverride?: string }>, ): Promise { const cacheKey = `${args.consumerRoot}|${args.configPathOverride ?? ""}`; const cached = CodemationConsumerConfigLoader.resolutionCache.get(cacheKey); if (cached) { return cached; } const promise = this.loadUncached(args); CodemationConsumerConfigLoader.resolutionCache.set(cacheKey, promise); try { return await promise; } catch (error) { CodemationConsumerConfigLoader.resolutionCache.delete(cacheKey); throw error; } } private async loadUncached( args: Readonly<{ consumerRoot: string; configPathOverride?: string }>, ): Promise { const loadStarted = performance.now(); let mark = loadStarted; const importSession = this.createImportSession(); const phaseDurations = new Map(); const phaseMs = (label: string): void => { const now = performance.now(); const delta = now - mark; mark = now; phaseDurations.set(label, delta); this.performanceDiagnosticsLogger.info( `load.${label} +${delta.toFixed(1)}ms (cumulative ${(now - loadStarted).toFixed(1)}ms)`, ); }; const bootstrapSource = await BootTimer.measureAsync("config.resolveConfigPath", () => this.resolveConfigPath(args.consumerRoot, args.configPathOverride), ); phaseMs("resolveConfigPath"); if (!bootstrapSource) { throw new Error( 'Codemation config not found. Expected "codemation.config.ts" in the consumer project root or "src/".', ); } const moduleExports = await BootTimer.measureAsync("config.importConfigModule", () => this.importModule(bootstrapSource, importSession), ); phaseMs("importConfigModule"); const rawConfig = this.configExportsResolver.resolveConfig(moduleExports); if (!rawConfig) { throw new Error(`Config file does not export a Codemation config object: ${bootstrapSource}`); } const config = this.configNormalizer.normalize(rawConfig); if (rawConfig.codemationVersion) { this.bootLogger.info(`codemationVersion: ${rawConfig.codemationVersion}`); } const workflowSources = await BootTimer.measureAsync("config.resolveWorkflowSources", () => this.resolveWorkflowSources(args.consumerRoot, config), ); phaseMs("resolveWorkflowSources"); const workflows = await BootTimer.measureAsync("config.loadDiscoveredWorkflows", async () => this.mergeWorkflows( config.workflows ?? [], await this.loadDiscoveredWorkflows(args.consumerRoot, config, workflowSources, importSession), ), ); phaseMs("loadDiscoveredWorkflows"); const resolvedConfig: NormalizedCodemationConfig = { ...config, workflows, }; logLevelPolicyFactory.create().applyCodemationLogConfig(resolvedConfig.log); return { config: resolvedConfig, bootstrapSource, workflowSources, }; } private async resolveConfigPath( consumerRoot: string, configPathOverride: string | undefined, ): Promise { if (configPathOverride) { const explicitPath = path.isAbsolute(configPathOverride) ? configPathOverride : path.resolve(consumerRoot, configPathOverride); if (!(await this.exists(explicitPath))) { throw new Error(`Config file not found: ${explicitPath}`); } return explicitPath; } for (const candidate of this.getConventionCandidates(consumerRoot)) { if (await this.exists(candidate)) { return candidate; } } return null; } private getConventionCandidates(consumerRoot: string): ReadonlyArray { return [ path.resolve(consumerRoot, "codemation.config.ts"), path.resolve(consumerRoot, "codemation.config.js"), path.resolve(consumerRoot, "src", "codemation.config.ts"), path.resolve(consumerRoot, "src", "codemation.config.js"), ]; } private async resolveWorkflowSources(consumerRoot: string, config: CodemationConfig): Promise> { if ((config.workflowDiscovery?.directories?.length ?? 0) === 0) { return []; } const discoveredPaths = await this.workflowModulePathFinder.discoverModulePaths({ consumerRoot, workflowDirectories: config.workflowDiscovery?.directories, exists: (absolutePath) => this.exists(absolutePath), }); return [...discoveredPaths].sort((left: string, right: string) => left.localeCompare(right)); } private async loadDiscoveredWorkflows( consumerRoot: string, config: CodemationConfig, workflowSources: ReadonlyArray, importSession: ConsumerImportSession, ): Promise> { const workflowDiscoveryDirectories = config.workflowDiscovery?.directories ?? []; const workflowsById = new Map(); let skippedImportCount = 0; const loadResults = await Promise.allSettled( workflowSources.map(async (workflowSource: string) => ({ workflowSource, segments: this.pathSegmentsComputer.compute({ consumerRoot, workflowDiscoveryDirectories, absoluteWorkflowModulePath: workflowSource, }), moduleExports: await BootTimer.measureAsync( `workflow.${path.basename(workflowSource).replace(/\.tsx?$/, "")}`, () => this.importModule(workflowSource, importSession), ), })), ); for (let i = 0; i < loadResults.length; i += 1) { const result = loadResults[i]; if (result === undefined) { continue; } if (result.status === "rejected") { const workflowSource = workflowSources[i] ?? ""; const message = result.reason instanceof Error ? result.reason.message : String(result.reason); this.bootLogger.warn(`Skipping workflow file that failed to import: ${workflowSource} — ${message}`); skippedImportCount += 1; continue; } const loadedWorkflowModule = result.value; for (const workflow of this.workflowDefinitionExportsResolver.resolve(loadedWorkflowModule.moduleExports)) { const enriched = loadedWorkflowModule.segments && loadedWorkflowModule.segments.length > 0 ? ({ ...workflow, discoveryPathSegments: loadedWorkflowModule.segments } satisfies WorkflowDefinition) : workflow; workflowsById.set(workflow.id, enriched); } } if (workflowsById.size === 0 && workflowSources.length > 0) { if (skippedImportCount > 0) { this.bootLogger.warn( `Booting with no discovered workflows: all ${skippedImportCount} discovered workflow file(s) failed to import (see warnings above). The host will serve an empty workflow set until they are fixed.`, ); return []; } throw new Error(this.discoveredWorkflowsEmptyMessageFactory.create(workflowSources)); } return [...workflowsById.values()]; } private mergeWorkflows( configuredWorkflows: ReadonlyArray, discoveredWorkflows: ReadonlyArray, ): ReadonlyArray { const workflowsById = new Map(); for (const workflow of discoveredWorkflows) { workflowsById.set(workflow.id, workflow); } for (const workflow of configuredWorkflows) { workflowsById.set(workflow.id, workflow); } return [...workflowsById.values()]; } private async importModule( modulePath: string, importSession: ConsumerImportSession, ): Promise> { if (this.shouldUseNativeRuntimeImport()) { return await this.importModuleWithNativeRuntime(modulePath); } const tsconfigPath = await this.resolveTsconfigPath(modulePath); const cacheKey = tsconfigPath || "default"; const shouldResetImporter = importSession.shouldResetImporter; const didResetImporterForThisImport = shouldResetImporter && !importSession.resetCacheKeys.has(cacheKey); if (didResetImporterForThisImport) { await this.resetImporter(tsconfigPath); importSession.resetCacheKeys.add(cacheKey); } const importSpecifier = await this.createImportSpecifier(modulePath); for (let attempt = 0; attempt < 3; attempt += 1) { try { const importedModule = await ( await this.getOrCreateImporter(tsconfigPath) ).import(importSpecifier, import.meta.url); return importedModule as Record; } catch (error) { if (!this.isStoppedTransformServiceError(error) || attempt === 2) { throw error; } await this.resetImporter(tsconfigPath); } } throw new Error(`Failed to import consumer module after retries: ${modulePath}`); } private async importModuleWithNativeRuntime(modulePath: string): Promise> { const importedModule = await import(await this.createImportSpecifier(modulePath)); return importedModule as Record; } private async resolveTsconfigPath(modulePath: string): Promise { const overridePath = process.env.CODEMATION_TSCONFIG_PATH; if (overridePath && (await this.exists(overridePath))) { return overridePath; } const discoveredPath = await this.findNearestTsconfig(modulePath); return discoveredPath ?? false; } private async getOrCreateImporter(tsconfigPath: string | false): Promise { const cacheKey = tsconfigPath || "default"; const existingImporter = CodemationConsumerConfigLoader.importerRegistrationsByTsconfig.get(cacheKey); if (existingImporter) { return existingImporter; } const { register } = await import(/* webpackIgnore: true */ this.resolveTsxImporterModuleSpecifier()); const namespaceVersion = this.nextNamespaceVersion(cacheKey); const nextImporter = register({ namespace: this.toNamespace(cacheKey, namespaceVersion), tsconfig: tsconfigPath, }); CodemationConsumerConfigLoader.importerRegistrationsByTsconfig.set(cacheKey, nextImporter); return nextImporter; } private async resetImporter(tsconfigPath: string | false): Promise { const cacheKey = tsconfigPath || "default"; const existingImporter = CodemationConsumerConfigLoader.importerRegistrationsByTsconfig.get(cacheKey); if (!existingImporter) { return; } CodemationConsumerConfigLoader.importerRegistrationsByTsconfig.delete(cacheKey); await existingImporter.unregister().catch(() => null); } private nextNamespaceVersion(cacheKey: string): number { const nextVersion = (CodemationConsumerConfigLoader.importerNamespaceVersionByTsconfig.get(cacheKey) ?? 0) + 1; CodemationConsumerConfigLoader.importerNamespaceVersionByTsconfig.set(cacheKey, nextVersion); return nextVersion; } private toNamespace(cacheKey: string, namespaceVersion: number): string { return `codemation_consumer_${cacheKey.replace(/[^a-zA-Z0-9_-]+/g, "_")}_${namespaceVersion}`; } private resolveTsxImporterModuleSpecifier(): string { return ["tsx", "esm", "api"].join("/"); } private async findNearestTsconfig(modulePath: string): Promise { let currentDirectory = path.dirname(modulePath); while (true) { const candidate = path.resolve(currentDirectory, "tsconfig.json"); if (await this.exists(candidate)) { return candidate; } const parentDirectory = path.dirname(currentDirectory); if (parentDirectory === currentDirectory) { return null; } currentDirectory = parentDirectory; } } private async createImportSpecifier(modulePath: string): Promise { const moduleUrl = pathToFileURL(modulePath); const moduleStats = await stat(modulePath); moduleUrl.searchParams.set("t", String(moduleStats.mtimeMs)); return moduleUrl.href; } private shouldUseNativeRuntimeImport(): boolean { return process.env.CODEMATION_TS_RUNTIME === "ts-node"; } private shouldResetImporterBeforeImport(): boolean { return (process.env.CODEMATION_DEV_SERVER_TOKEN?.trim().length ?? 0) > 0; } private createImportSession(): ConsumerImportSession { return { resetCacheKeys: new Set(), shouldResetImporter: this.shouldResetImporterBeforeImport(), }; } private isStoppedTransformServiceError(error: unknown): boolean { return error instanceof Error && error.message.includes("The service is no longer running"); } private async exists(filePath: string): Promise { try { await access(filePath); return true; } catch { return false; } } }