import * as path from 'path' import { IOptions } from '@plastichub/osrl/types' import { Engine as engine } from '@plastichub/osrl/Engine' import { parse } from '@plastichub/osrl/options' import { get_cached, set_cached } from '@plastichub/osr-cache/lib' import { OSR_CACHE } from '@plastichub/osr-cli-commons' import * as md5 from 'md5' import { files as glob, pathInfo } from '@plastichub/osr-cli-commons/glob' import { isFile, resolve, isFolder } from '@plastichub/osr-cli-commons/fs' import { logger } from '../src' import { sync as exists } from '@plastichub/fs/exists' import { sync as read } from '@plastichub/fs/read' import { sync as write } from '@plastichub/fs/write' import { sync as rm } from '@plastichub/fs/remove' import { Promise as BPromise } from 'bluebird' import { option } from './' import { MODULE_NAME } from '../src/constants' const chokidar = require("chokidar") export const fileAsBuffer = (path: string) => read(path, 'buffer') as Buffer || Buffer.from("-") export const createContent = async (file, _options) => { const parts = path.parse(file); const rel = path.relative(_options.root, file); let output = _options.output; let outputInfo = pathInfo(_options.output) const variables = { root: _options.root || '.', cwd: _options.cwd || path.resolve('.'), ..._options.variables } if (!outputInfo.FILE_EXT) { output = path.resolve(`${_options.output}/${path.parse(rel).dir}/${parts.name}.md`) } else { output = path.resolve(resolve(output, false, variables)); } const defaults: any = { language: _options.lang, debug: _options.debug, profile: _options.profile, output: output, plugins: _options.plugins, env: _options.env || 'library', cwd: _options.cwd || path.resolve('.'), source: file, variables }; const options = parse(defaults, defaults); const eOptions = { ...options, root: [ ...options.profile.includes, path.parse(file).dir ], toHTML: false, cache: false, keepOutputType: true, trimTagRight: false, trimTagLeft: false, trimOutputRight: false, trimOutputLeft: false, greedy: false } as IOptions; const Engine = new engine(eOptions); options.source = path.resolve(options.source); const osr_cache = OSR_CACHE() const cached = await get_cached(options.source, eOptions, _options.module || MODULE_NAME) if (osr_cache && cached && _options.cache !== false) { let md5Src = md5(Buffer.from(cached)); let md5Dst = md5(fileAsBuffer(options.output)); if (!exists(options.output) || md5Src !== md5Dst) { logger.info('Compile file serving from cache: ' + options.source) write(options.output, cached); } lockMap[file] = false return cached } logger.info('Compile file ' + file) let content = await Engine.render(options.source, options.variables) if (_options.onCompiled) { content = await _options.onCompiled(options.source, output, content) } if (osr_cache && _options.cache !== false) { options.debug && logger.info('Write output to cache', output) await set_cached(options.source, eOptions, _options.module || MODULE_NAME, content) } let dst = path.resolve(resolve(output, false, options.variables)) logger.info('Write output to: ', dst) write(dst, content) if (_options.onCompileDone) { await _options.onCompileDone(options.source, dst, options, content) } lockMap[file] = false return content } const lockMap = {} const watch = async (src, options) => { src = path.resolve(src) const dInfo = pathInfo(src) const dir = dInfo.IS_GLOB ? dInfo.DIR : src const debugWatch = options.debugWatch debugWatch && logger.info('Watching', dir, pathInfo(src)) const watcher = chokidar.watch(`${dir}`, { ignored: /(^|[\/\\])\../, persistent: true, ignoreInitial: false, followSymlinks: true, disableGlobbing: false, usePolling: false, interval: 1000, binaryInterval: 300, alwaysStat: false, depth: 99, awaitWriteFinish: { stabilityThreshold: 2500, pollInterval: 2000 }, ignorePermissionErrors: false, atomic: 2000 }) watcher.on('addDir', path => logger.debug(`Directory ${path} has been added`)) .on('unlinkDir', path => logger.debug(`Directory ${path} has been removed`)) .on('error', error => logger.debug(`Watcher error: ${error}`)) .on('ready', () => logger.debug('Initial scan complete. Ready for changes')) .on('raw', (event, _path, details) => { // deal with rename, delete, and new files // debugWatch && logger.debug('Raw event info:', event, _path, details); switch (event) { case 'rename': { const rel = path.relative(dir, path.parse(details.watchedPath).dir) const dst = path.resolve(`${options.output}/${rel}/${_path}`) if (exists(dst)) { debugWatch && logger.debug('Renamed:', _path, ' : deleting output : ', dst, details) rm(dst) } else { debugWatch && logger.debug('Renamed:', _path, ' : cant find output : ', dst, details) } const _src = path.resolve(`${details.watchedPath}`) if (lockMap[_src]) { logger.warn('Locked', _src) return } if (exists(_src) && isFile(_src)) { debugWatch && logger.debug('Renamed:', _path, ' : creating content', details) lockMap[_src] = true createContent(_src, { ...options }) } else { //debugWatch && logger.debug('Renamed:', _src, ' : source not found', details) } break; } case 'change': { let _src = '' if(isFile(details.watchedPath)){ _src = `${details.watchedPath}` }else{ _src = path.resolve(`${details.watchedPath}/${_path}`) } if (!exists(details.watchedPath)) { return } if (lockMap[_src]) { // logger.warn('Locked', _src) return } if (exists(_src) && isFile(_src)) { debugWatch && logger.debug('Changed:', _path, ' : creating content', details) lockMap[_src] = true createContent(_src, { ...options }) setTimeout(() => { lockMap[_src] = false }, 3000) } else { debugWatch && logger.debug('Changed:', _src, ' : source not found', details) } break; } } }) return watcher } const compileAllEx = async (files, options) => { return await BPromise.resolve(files).map((f) => { return createContent(f, options) }, { concurrency: 1 }); } export const register = (grunt) => { grunt.registerMultiTask('compile', 'watch and compile ', function () { const done = this.async() const task_options = this.data.options let debug = option('debug', task_options, grunt) const cwd = option('cwd', task_options, grunt, process.cwd()) const options = { ...this.data.options, root: option('root', task_options, grunt, path.resolve('./')), debug: option('debug', task_options, grunt), debugWatch: option('debugWatch', task_options, grunt), verbose: option('verbose', task_options, grunt), module: option('module', task_options, grunt), lang: option('lang', task_options, grunt, 'osr'), env: option('env', task_options, grunt), format: option('format', task_options, grunt, 'html'), plugins: option('plugins', task_options, grunt, ['${root}/osr/plugins']), profile: option('profile', task_options, grunt, '${OSR_ROOT}/.osrl.json'), variables: option('variables', task_options, grunt, {}), onCompiled: option('onCompiled', task_options, grunt), onCompileDone: option('onCompileDone', task_options, grunt), cache: option('cache', task_options, grunt, true) } task_options.watch && this.data.src.map(f => watch(f, options)) let _files = []; this.data.src.map(f => { _files = _files.concat(glob(cwd, f)); }); if (!_files.length) { debug && logger.error(`Have no files or glob in ${cwd}`, this.data.src); debug && logger.info(this.data.src.map(pathInfo)) return } debug && logger.info('Compile - task data ', process.cwd(), this.data, _files) debug && logger.debug("Compile - Options", options) if (options.clear !== false) { if (exists(options.output)) { debug && logger.info('Clear output', options.output) rm(options.output) } } compileAllEx(_files, options).then((f) => { !task_options.watch && done() debug && logger.debug('Done compiling all'); }) }) }