{ ".github/ISSUE_TEMPLATE/01_bug_report.md": { "content": "---\nname: 🐞 Bug Report\nabout: Describe a problem you've encountered\nlabels: \"bug\"\n---\n\n## Version Information\n\nServer Version:\nCLI Version (if applicable):\n\n## Environment\n\n\n\n## Current Behavior\n\n\n\n## Expected Behavior\n\n\n\n## Steps to Reproduce\n\n1.\n2.\n3.\n\n## Screenshots or Screencast\n\n\n\n## Logs or Traces\n\n\n\n## Possible Solutions or Workarounds\n\n\n" }, ".github/ISSUE_TEMPLATE/02_feature_request.md": { "content": "---\nname: Feature Request\nabout: Share your idea for a new feature\ntitle: \"\"\nlabels: feature\nassignees: \"\"\n---\n\n## The Problem \n\n\n\n(Explain the problem here.)\n\n## The Solution\n\n\n\n(Describe your proposed solution here.)\n\n## How will we solve\n\n\n\n(Provide an overview of how the proposed solution will be implemented.)\n\n## Test Cases\n\n\n\n(Provide a list of test cases for the new feature.)\n" }, "README.md": { "content": "# Website\n\nThis website is built using [Docusaurus 2](https://docusaurus.io/), a modern static website generator.\n\n### Installation\n\n```\n$ yarn\n```\n\n### Local Development\n\n```\n$ yarn start\n```\n\nThis command starts a local development server and opens up a browser window. Most changes are reflected live without having to restart the server.\n\n### Build\n\n```\n$ yarn build\n```\n\nThis command generates static content into the `build` directory and can be served using any static contents hosting service.\n\n### Deployment\n\nUsing SSH:\n\n```\n$ USE_SSH=true yarn deploy\n```\n\nNot using SSH:\n\n```\n$ GIT_USER= yarn deploy\n```\n\nIf you are using GitHub pages for hosting, this command is a convenient way to build the website and push to the `gh-pages` branch.\n" }, "blog/2019-05-28-first-blog-post.md": { "content": "---\nslug: first-blog-post\ntitle: First Blog Post\nauthors:\n name: Gao Wei\n title: Docusaurus Core Team\n url: https://github.com/wgao19\n image_url: https://github.com/wgao19.png\ntags: [hola, docusaurus]\ndescription: This is my first blog post.\nkeywords: [first, blog, post]\n---\n\nLorem ipsum dolor sit amet, consectetur adipiscing elit. Pellentesque elementum dignissim ultricies. Fusce rhoncus ipsum tempor eros aliquam consequat. Lorem ipsum dolor sit amet\n" }, "blog/2019-05-29-long-blog-post.md": { "content": "---\nslug: long-blog-post\ntitle: Long Blog Post\nauthors: endi\ntags: [hello, docusaurus]\n---\ndescription: This is a long blog post.\nkeywords: [long, blog, post]\n---\n\nThis is the summary of a very long blog post,\n\nUse a `` comment to limit blog post size in the list view.\n\n\n\nLorem ipsum dolor sit amet, consectetur adipiscing elit. Pellentesque elementum dignissim ultricies. Fusce rhoncus ipsum tempor eros aliquam consequat. Lorem ipsum dolor sit amet\n\nLorem ipsum dolor sit amet, consectetur adipiscing elit. Pellentesque elementum dignissim ultricies. Fusce rhoncus ipsum tempor eros aliquam consequat. Lorem ipsum dolor sit amet\n\nLorem ipsum dolor sit amet, consectetur adipiscing elit. Pellentesque elementum dignissim ultricies. Fusce rhoncus ipsum tempor eros aliquam consequat. Lorem ipsum dolor sit amet\n\nLorem ipsum dolor sit amet, consectetur adipiscing elit. Pellentesque elementum dignissim ultricies. Fusce rhoncus ipsum tempor eros aliquam consequat. Lorem ipsum dolor sit amet\n\nLorem ipsum dolor sit amet, consectetur adipiscing elit. Pellentesque elementum dignissim ultricies. Fusce rhoncus ipsum tempor eros aliquam consequat. Lorem ipsum dolor sit amet\n\nLorem ipsum dolor sit amet, consectetur adipiscing elit. Pellentesque elementum dignissim ultricies. Fusce rhoncus ipsum tempor eros aliquam consequat. Lorem ipsum dolor sit amet\n\nLorem ipsum dolor sit amet, consectetur adipiscing elit. Pellentesque elementum dignissim ultricies. Fusce rhoncus ipsum tempor eros aliquam consequat. Lorem ipsum dolor sit amet\n\nLorem ipsum dolor sit amet, consectetur adipiscing elit. Pellentesque elementum dignissim ultricies. Fusce rhoncus ipsum tempor eros aliquam consequat. Lorem ipsum dolor sit amet\n\nLorem ipsum dolor sit amet, consectetur adipiscing elit. Pellentesque elementum dignissim ultricies. Fusce rhoncus ipsum tempor eros aliquam consequat. Lorem ipsum dolor sit amet\n\nLorem ipsum dolor sit amet, consectetur adipiscing elit. Pellentesque elementum dignissim ultricies. Fusce rhoncus ipsum tempor eros aliquam consequat. Lorem ipsum dolor sit amet\n\nLorem ipsum dolor sit amet, consectetur adipiscing elit. Pellentesque elementum dignissim ultricies. Fusce rhoncus ipsum tempor eros aliquam consequat. Lorem ipsum dolor sit amet\n\nLorem ipsum dolor sit amet, consectetur adipiscing elit. Pellentesque elementum dignissim ultricies. Fusce rhoncus ipsum tempor eros aliquam consequat. Lorem ipsum dolor sit amet\n\nLorem ipsum dolor sit amet, consectetur adipiscing elit. Pellentesque elementum dignissim ultricies. Fusce rhoncus ipsum tempor eros aliquam consequat. Lorem ipsum dolor sit amet\n\nLorem ipsum dolor sit amet, consectetur adipiscing elit. Pellentesque elementum dignissim ultricies. Fusce rhoncus ipsum tempor eros aliquam consequat. Lorem ipsum dolor sit amet\n\nLorem ipsum dolor sit amet, consectetur adipiscing elit. Pellentesque elementum dignissim ultricies. Fusce rhoncus ipsum tempor eros aliquam consequat. Lorem ipsum dolor sit amet\n\nLorem ipsum dolor sit amet, consectetur adipiscing elit. Pellentesque elementum dignissim ultricies. Fusce rhoncus ipsum tempor eros aliquam consequat. Lorem ipsum dolor sit amet\n" }, "blog/2021-08-01-mdx-blog-post.mdx": { "content": "---\nslug: mdx-blog-post\ntitle: MDX Blog Post\nauthors: [slorber]\ntags: [docusaurus]\n---\ndescription: Blog posts support Docusaurus Markdown features, such as MDX.\nkeywords: [docusaurus, markdown, MDX, react]\n---\n\nBlog posts support [Docusaurus Markdown features](https://docusaurus.io/docs/markdown-features), such as [MDX](https://mdxjs.com/).\n\n:::tip\n\nUse the power of React to create interactive blog posts.\n\n```js\n\n```\n\n\n\n:::\n" }, "blog/2021-08-26-welcome/index.md": { "content": "---\nslug: welcome\ntitle: Welcome\nauthors: [slorber, yangshun]\ntags: [facebook, hello, docusaurus]\n---\ndescription: Docusaurus blogging features are powered by the blog plugin.\nkeywords: [docusaurus, blog, plugin]\n---\n\n[Docusaurus blogging features](https://docusaurus.io/docs/blog) are powered by the [blog plugin](https://docusaurus.io/docs/api/plugins/@docusaurus/plugin-content-blog).\n\nSimply add Markdown files (or folders) to the `blog` directory.\n\nRegular blog authors can be added to `authors.yml`.\n\nThe blog post date can be extracted from filenames, such as:\n\n- `2019-05-30-welcome.md`\n- `2019-05-30-welcome/index.md`\n\nA blog post folder can be convenient to co-locate blog post images:\n\n![Docusaurus Plushie](./docusaurus-plushie-banner.jpeg)\n\nThe blog supports tags as well!\n\n**And if you don't want a blog**: just delete this directory, and use `blog: false` in your Docusaurus config.\n" }, "docs/ai-modules/react-app.md": { "content": "---\ntitle: Web-UI Starter Kit\ndescription: Godspeed Web-UI Starter Kit simplifies the generation of a foundational front-end from OpenAPI/Swagger API specs. An AI agent creates the template with pre-established API connections.\nkeywords: [Godspeed, Web-UI, Starter Kit, OpenAPI, Swagger, React]\n---\n# Web-UI Starter Kit\n## Introduction\n\nGodspeed Web-UI Starter Kit simplifies the generation of a foundational front-end from OpenAPI/Swagger API specs. An AI agent creates the template with pre-established API connections.\n\n> An Engineering Powered AI Scaffolding agent to help you generate codebase from bare minimum requirements.\n\nCurrently we support [React](https://react.dev/).\n\n| Framework | Support |\n| --------- | ----------- |\n| React | ✅ |\n| Vue | Coming Soon |\n| Svelte | Coming Soon |\n| Angular | Coming Soon |\n| Vanila JS | Coming Soon |\n\n---\n\n##### Tech Stack For React\n\n* React-Query\n* React-Router\n* Tailwind\n\n---\n\n##### Essentials for Running this Package\n\n- OpenAI API/Microservice spec [Link](https://swagger.io/specification/)\n- OpenAI/ChatGPT API Secret Key [Link](https://help.openai.com/en/articles/4936850-where-do-i-find-my-secret-api-key)\n\n---\n\n\n### Steps for creating a New Project\n* Install Godspeed GS-Kit by `npm i @godspeedsystems/uikit`\n* Go to a directory /root\n* create a .env file with these keys\n\n ```\n OPENAI_API_KEY=sk-xxx\n OPENAI_ORG_KEY=org-xxx\n ```\n\n > Note: If you dont have `OPENAI_ORG_KEY` then skip it\n >\n* run -\n\n ```\n npx @godspeedsystems/uikit {app_type}\n ```\n* Enter any app type that you want, currently we are supporting 'React-Web' for react web app.\n ```\n npx @godspeedsystems/uikit react-web\n ```\n* Other tamplates are ***coming soon***. \n\n* It will ask you for the location and name of your OpenAPI file.\n```\n> npx @godspeedsystems/uikit react-web\n[WARN] Importing from 'langchain' is deprecated. See https://js.langchain.com/docs/getting-started/install#updating-from-0052 for upgrade instructions.\n┌ Generate front-end boilerplate using Godspeed\n│\n◆ What is the name of your project?\n│ gskit-app_GodspeedTrialProject\n└\n\n\n```\n* hit `Enter` after \n* Then it will ask about the where is the OpenAPI Spec file is located. Mention your file path over there.\n```\n> npx @godspeedsystems/uikit react-web\n[WARN] Importing from 'langchain' is deprecated. See https://js.langchain.com/docs/getting-started/install#updating-from-0052 for upgrade instructions.\n┌ Generate front-end boilerplate using Godspeed\n│\n◇ What is the name of your project?\n│ gskit-app-godspeedTrial\n{ name: 'gskit-app-godspeedTrial' }\n│\n◆ Where is your OpenAPI spec file located?\n│ ./api.yaml_\n└\n```\n* AI agent will read the OpenAPI spec and generate a react project with fetch hooks and tailwind configured.\n* depends on the OpenAPI file it will take few seconds to create a project. In the below case the file name is api.yaml.\n```\n> npx @godspeedsystems/uikit react-web\n\n[WARN] Importing from 'langchain' is deprecated. See https://js.langchain.com/docs/getting-started/install#updating-from-0052 for upgrade instructions.\n┌ Generate front-end boilerplate using Godspeed\n│\n◇ What is the name of your project?\n│ gskit-app-godspeedTrial\n{ name: 'gskit-app-godspeedTrial' }\n│\n◇ Where is your OpenAPI spec file located?\n│ ./api.yaml\n│\n◒ Cloning template.cloned godspeedsystems/gskit-react#main to ./gskit-app-godspeedTrial\n◇ Template cloned\n│\n◇ Pick a Data fetching library\n│ React Query\n│\n◓ Generating Fetch Hooks...\n```\n* After the Project is created it looks like...\n```\n> npx @godspeedsystems/uikit react-web\n\n[WARN] Importing from 'langchain' is deprecated. See https://js.langchain.com/docs/getting-started/install#updating-from-0052 for upgrade instructions.\n┌ Generate front-end boilerplate using Godspeed\n│\n◇ What is the name of your project?\n│ gskit-app-godspeedTrial\n{ name: 'gskit-app-godspeedTrial' }\n│\n◇ Where is your OpenAPI spec file located?\n│ ./api.yaml\n│\n◒ Cloning template.cloned godspeedsystems/gskit-react#main to ./gskit-app-godspeedTrial\n◇ Template cloned\n│\n◇ Pick a Data fetching library\n│ React Query\n│\n◇ react-query fetch hooks generated\n│\n◇ Fetch hooks written to file\n```\n* One folder with the Project name will create in the root directory. ( your react project folder )\n* Go to the new app folder and install the dependencies by `npm install`\n* Run `npm start` command to run the react project.\n* The Project will run on http://localhost:3000.\n```\nCompiled successfully!\n\nYou can now view gskit-react in the browser.\n\n Local: http://localhost:3000\n On Your Network: http://192.168.0.188:3000\n\nNote that the development build is not optimized.\nTo create a production build, use npm run build.\n\nwebpack compiled successfully\n```\n* if Port 3000 is busy it will ask to change the port.\n```\n? Something is already running on port 3000. Probably:\n node /Users/other_port_path\n in /Users/other_port_path\n\nWould you like to run the app on another port instead? › (Y/n)\n```\n* After This the project folder will created. \n\n### What you will end up with\n\n* React js application.\n* Your Tailwind gets Configured.\n* Your React Query is configured. \n* Ready for further development.\n" }, "docs/documentation_principles.md": { "content": "---\ntitle: Guiding Principles Of Documentation\ndescription: Essential guiding principles for creating effective documentation, including clarity, consistency, accuracy, relevance, and organization.\nkeywords: [documentation, principles, clarity, consistency, accuracy, relevance, organization]\n---\n# Guiding Principles Of Documentation:\n\nCreating effective documentation is essential for clear communication and knowledge transfer. Here are some guiding principles for documentation:\n\n### Clarity: \nEnsure that your documentation is easy to understand for your target audience. Use clear and concise language, avoid jargon, and define technical terms when necessary.\n\n### Consistency: \nMaintain a consistent structure and formatting throughout your documentation. This includes headings, fonts, colors, and styles.\n\n### Accuracy: \nDouble-check all information to ensure it is up-to-date and correct. Inaccurate documentation can lead to misunderstandings and errors.\n\n### Relevance: \nInclude only information that is relevant to the audience's needs. Avoid unnecessary details and tangents.\n\n### Organization: \nUse a logical and well-structured format. Group related information together, use headings and subheadings, and provide a table of contents if needed.\n\n### Accessibility:\nMake sure your documentation is easily accessible to the intended audience. This may involve using a user-friendly platform or ensuring that printed documents are readily available.\n\n### Version Control: \nIf your documentation changes over time, keep track of different versions. This is especially important for software documentation. Users should be able to access the documentation for the version they are using.\n\n### Visual Aids: \nIncorporate visuals, such as diagrams, charts, screenshots, and videos, to complement text. Visual aids can enhance understanding and retention.\n\n### Feedback: \nEncourage feedback from users. They can provide valuable insights on how to improve the documentation. Consider using surveys, comments, or a contact point for questions.\n\n### Searchability:\nImplement a search function if your documentation is digital. This makes it easier for users to find the information they need.\n\n### Context: \nProvide context for the information. Explain why something is important or how it fits into a larger process. Context helps users make better use of the information.\n\n### Use Cases: \nInclude practical examples and use cases. Demonstrating how to apply the information can be more effective than theory alone.\n\n### Conciseness:\nAvoid unnecessary verbosity. Get to the point and be concise while still providing necessary information.\n\n### Cross-Referencing:\nInclude links or references to related topics within your documentation. This helps users explore related information.\n\n### Testing:\nBefore finalizing your documentation, have a test audience review it. They can identify areas that may be unclear or missing.\n\n### Security:\nProtect sensitive information. If your documentation includes confidential data, ensure it is only accessible to authorized users.\n\n### Maintenance:\nKeep your documentation up-to-date. As systems or processes change, your documentation should reflect those changes.\n\n### User-Centered:\nAlways keep your target audience in mind. Your documentation should cater to their needs and expectations.\n\n### Simplicity:\nKeep it simple. Avoid overcomplicating your documentation. Simple, straightforward language is often more effective.\n\n### Training:\nConsider providing training on how to use your documentation effectively. Training can help users navigate and understand your documentation better.\n\nRemember that the effectiveness of documentation is measured by its ability to convey information clearly and help users achieve their goals. Tailor your documentation to the specific needs and preferences of your audience." }, "docs/microservices-framework/API & Event.md": { "content": "---\ntitle: API & Event\ndescription: In Godspeed, every API is defined as an event and handled by a corresponding function, with a unified event-driven structure for both REST and non-REST event sources.\nkeywords: [Godspeed, API, event, REST, event-driven, event schema, JWT, OAuth2, authorization]\n---\nIn Godspeed, every API is defined as an event, and handled by a corresponding function. This unified event-driven structure works for both REST and non-REST event sources (HTTP, WebSocket, Kafka, etc.).\n\nThis guide explains how to write new APIs in three clear steps:\n\n\n## Define API Schema (give all event examples)\n\nTo write an api in Godspeed, first you need to write an Event Schema. This schema is a structured YAML configuration that follows the OpenAPI specification, allowing you to define every detail of how the event should behave. All events adhere to a standard structure, which is one of the core design principles of Godspeed, regardless of their source or protocol.\n\nAn event schema specifies:\n\n- The name/topic/URL of the event\n- The event handler Workflow (fn)\n- Input and Output schema\n- [Validation error handling](/docs/microservices-framework/event-sources/validations/schema-validation)\n\n\nGodspeed follows [Schema Driven Development & Single Source of Truth](../microservices-framework/introduction/guard-rails.md#1schema-driven-development), [Configure Over Code](../microservices-framework/introduction/guard-rails.md#2configure-over-code) and [Modular Architecture](../microservices-framework/introduction/guard-rails.md#4-decoupled-architecture).\n\nThe meta-framework follows a generic schema for defining events - their inputs, outputs, swagger specs, with validations, authentication, authorization etc.\n\n```yaml\nhttp.get./greet: #The initial line depicts a fusion of the event, the employed method, and the path associated with the event.\n fn: function_greet #Required. The 'fn' key receives the function name located in 'src/functions' and forwards the accompanying parameters.\n\n #optional configurations\n #Swagger components\n body: #same as requestBody in Swagger world\n params: #same as swagger `parameters` schema\n responses: #same as swagger `responses` schema\n id: # swagger. if not provided, when generating swagger, this is generated from the URI of the event by default\n operationId: # swagger if not provided, check if `id` is set. If that is also not set, use the summary to generate the operationId\n summary:\n tags: # swagger if you give `tags` array in schema of event, framework uses that to add tags to your generated spec. Else it uses the path and name of the file containing the event as tags. For ex. _\n\n #Other non-swagger components (optional)\n authn: #custom authentication. Currently plugins support JWT. Can be customized\n authz: #your custom authz workflow\n on_request_validation_error: #when validation fails\n on_response_validation_error: #when validation fails\n log: #Open Telemetry compliant log attributes which help debug and search through logs better\n attributes:\n```\n\n`http`: Protocol http eventsource (can be any)\n\n`get` : method (depends on the eventsource used. Can be topic for Kafka)\n\n`/helloworld`: endpoint (In case of http and graphql sources. Can be groupId in case of Kafka for ex.)\n\nWe are exposing an endpoint with a `get` method on `http` protocol. This endpoint is calling an eventhandler called `helloworld` in the second line. Event handlers can be functions written in typescript, javascript or yaml workflows in Godspeed's DSL format. In the above example the helloworld function exists in `src/functions` directory.\n\nWhen switching between eventsources, the event schema undergoes significant changes.\n\n- The first line is changed for each protocol:\n\n In the case of sync events or HTTP events, the start line includes the eventsource name, method, and path.\n\n However, for async events, the start line combines the source name, topic and group ID (for Kafka), or schedule (for Cron).\n\n- Async events like Kafka do not have responses, authentication and authorization fields in schema.\n\n- Cron events do not have any input.\n\n#### Example HTTP Schema\n\n```yaml\nhttp.get./greet: #The initial line depicts a fusion of the event, the employed method, and the path associated with the event.\n fn: function_greet #The 'fn' key receives the function name located in 'src/functions' and forwards the accompanying parameters.\n on_request_validation_error: on_request_validation\n params: #It is also possible to define inputs such as 'params,' 'body,' 'headers,' and 'query parameters.'\n - name: greet_message\n in: query\n required: true\n body:\n content:\n application/json:\n schema:\n type: object\n properties:\n name:\n type: string\n responses:\n 500:\n content:\n application/json:\n schema:\n type: object\n 200:\n content:\n application/json:\n schema:\n type: object\n```\n\n## Authentication using JWT\n\n**JWT (JSON Web Token)** is a standard method for securely transmitting information between two parties as a JSON object. It’s commonly used for authentication and authorization in web applications.\nIn Godspeed, you can easily implement JWT authentication to protect routes, control access to resources and ensure secure API communication.\nCurrently supported event sources which can leverage this mechanism are:\n1. Express\n2. Apollo GraphQL\n3. Fastify\n\nTo Learn more about JWT and Payload [Click Here](/docs/microservices-framework/authentication/jwt)\n\n### How JWT Authentication Works in Godspeed\nWhen a client makes a request to a protected route, they need to include a valid JWT in the request header.\nThe server validates the token, extracts the payload, and grants access to the resource if the token is valid.\nIn the case of an expired or invalid token, the client will receive a `401 Unauthorized` response.\n\n:::tip Note\nIn our Express eventsource plugin, JWT Authentication is implemented using passport-jwt which is a strategy for authenticating with a JSON Web Token. To know more about, you can check [passport documentation](https://www.passportjs.org/)\n:::\n\n### Setup and implementation of JWT authentication in Godspeed.\n\n### Step 1: Setting up Environment\n\n**1.** In Godspeed, any configuration which includes secrets or passwords is recommended to be defined using environment variables only. For this, open custom-environment-variables.yaml file which is placed under `config/` in your root project directory and add your jwt configs there like:\n\n```\njwtSecret: JWT_SECRET\naudience: JWT_AUDIENCE\nissuer: JWT_ISSUER\n\n```\n\n**2.** Now, you need to export these variables in environment. This can be done in two ways:\n\n**(a)** Set environment variables in .env file which is under your project's root folder /.env as shown below:\n```\n JWT_SECRET= mysecret #the secret\n JWT_AUDIENCE= godspeedsystemms #aud in jwt token\n JWT_ISSUER= godspeed #iss in jwt token\n\n```\n\n**(b)** Export these variables to your environment, follow the below syntax to export, based on the shell, you are using:\n
\n For git bash \n\n ```bash\n $ export JWT_SECRET=mysecret\n $ export JWT_ISS= mycompany\n ```\n\n
\n\n
\n For windows powershell \n\n ```bash\n $env:JWT_SECRET= \"mysecret\"\n $env:JWT_ISS= \"mycompany\"\n ```\n\n
\n\nAfter exporting the environment variable, you can access these variable anywhere in your project by using inline\nscripting `<%config.issuer%>` in yaml or as `ctx.config.issuer` in js/ts workflows.\n\n:::tip Note\nIf you do not set these environment variables mentioned above, it will result in an error while running your project. And if the token values set in header differ from those specified in the configuration, the response will be 'Unauthorized.'\n:::\n\n### Step 2: Enable JWT Auth in your project's eventsource configuration file.\nJWT configuration is written under authn: in the event source's configuration file. For Express, config file name will be http.yaml. Open this file and Set up jwt authn as shown below.\n\n`src/eventsources/http.yaml`\n```\ntype: express\nport: 4000\n#auth settings to run by default on every event\nauthn:\n jwt:\n secretOrKey: <% config.jwtSecret %> # to access jwtSecret from config\n audience: <% config.audience %>\n issuer: <% config.issuer %>\n```\nOnce you have enabled it here, authentication will be true for all endpoints, unless you explicitly set authn:false in their event schema.\n\n\n### Disabling JWT Authentication at Event Level\n\nThe plugins follow zero trust policy as a first principle, so if you have setup jwt spec at event source level, authentication for all the events will be true by default, unless you explicitly set authn:false in their event schema.\nIf you don't want users to be authenticated, you can disable any end-point by writing authn: false in your event schema like this:\n\n```\nhttp.get./helloworld:\n fn: helloworld\n authn: false\n params:\n - name: name\n in: query\n required: true\n```\n\n### User Login Example using JWT Authentciation\n\n**Event**\n```yaml\n# Login with username and password\nhttp.post./login: # defines the POST request that will be triggered when a client hits /login endpoint.\n fn: verifyLogin # the workflow to handle the request\n authn: false\n body:\n content:\n application/json:\n schema:\n type: object\n properties:\n username:\n type: string\n password:\n type: string\n required:\n - username\n - password\n responses:\n '200':\n description: 'Login successful'\n content:\n application/json:\n schema:\n type: object\n '401':\n description: 'Invalid credentials'\n content:\n application/json:\n schema:\n type: string\n example: 'Invalid username or password'\n```\n**Workflow (verifyLogin.ts)**\n```ts\nimport { GSCloudEvent, GSContext, PlainObject, GSStatus, logger } from \"@godspeedsystems/core\";\nimport jwt from 'jsonwebtoken';\nexport default function (ctx: GSContext) {\n const {\n inputs: {\n data: {\n body\n }\n },\n\n }= ctx;\n logger.info(\"user info received %o\", body);\n // Dummy user validation (replace with your authentication logic)\n if (body.username === 'user' && body.password === 'password') {\n // Create a JWT token\n const token = jwt.sign(\n { sub: '1234567890', name: 'John Doe', role: 'user' }, // Payload\n ctx.config.jwtSecret, // access Secret key from config\n { expiresIn: '1h', issuer: ctx.config.issuer, audience: ctx.config.audience } // jwt Options\n );\n logger.info(\"Token generated %s\", token);\n return new GSStatus(true, 200, 'Login Successful', {JWT: token}, {});\n } else {\n return new GSStatus(true, 401, 'Failed', 'Invalid Credentials', {});\n }\n}\n```\n\n### How to access JWT payload\nYou can access the complete JWT payload in `<% inputs.user %>` in YAML workflows, and as ctx.inputs.data.user when writing JS/TS workflows.\n\nExample access from TS workflow\n```ts\nimport { GSCloudEvent, GSContext, PlainObject, GSStatus, logger } from \"@godspeedsystems/core\";\nexport default function (ctx: GSContext) {\n const {\n inputs: {\n data: {\n user\n }\n },\n }= ctx;\n\nreturn new GSStatus(true, 200, 'Ok', {'Payload user': user}, {});\n}\n\n```\n\n### Standardized JWT Configuration (For Plugin Creators)\n\nFor consistency across plugins, it's recommended to use a standardized configuration format for JWT settings (issuer, audience, secret key) in eventsources/.yaml.\n\n## Authentication using oauth2\n\n## Introduction\n OAuth2 is a widely adopted authorization framework that provides a secure and efficient way for users to grant third-party applications access to their data without revealing their credentials. By leveraging OAuth2, Godspeed simplifies the authentication process for both developers and users. OAuth2 offer the ability to connect with authentication providers such as Google, Github and Linkedin.\n\n### How It Works:\n1. **User Signup/Signin:** When a user want to signup with google/github/linkedin, express-eventsource will trigger OAuth2 authentication flow.\n2. **Redirect to Provider:** The user is redirected to the chosen authentication provider (Google, LinkedIn, or GitHub) to authorize the application.\n3. **Authorization Grant:** The provider grants the application an authorization code.\n4. **Token Exchange:** The application exchanges the authorization code for an access token.\n5. **User Information Retrieval:** The application uses the access token to retrieve the user's information from the provider's API.\n6. **Authentication Success:** Once the user's identity is verified, they are logged in to the platform.\n\n## Getting Started:\n To start using OAuth2 authentication in your Godspeed project, follow these steps:\n\n### 1. Configure Provider:\n Set up your OAuth2 application with the chosen authentication provider (Google, LinkedIn, or GitHub) to obtain the necessary client ID and client secret.\n\n### 2. Set up Environment Variables:\n Open .env file in your project's root directory.\n Add the following lines, replacing the placeholders with your actual GitHub credentials:\n ```\n GITHUB_CLIENT_ID= \"your_client_id\"\n GITHUB_CLIENT_SECRET= \"your_client_secret\"\n GITHUB_CALLBACK_URL= \"your_callback_url\" e.g http://localhost:4000/auth/github/callback\n GITHUB_AUTH_ROUTE = \"/auth/github\"\n GITHUB_CALLBACK_ROUTE = \"/auth/github/callback\"\n GITHUB_SUCCESS_REDIRECT_URL = \"/verify/user\"\n GITHUB_FAILURE_REDIRECT_URL = \"/error\"\n # Session Secret\n SESSION_SECRET = \"your_secret\"\n ```\n### 3. Configure your eventsource\n\n Here's an example of OAuth2 configuration inside (src/eventsources/http.yaml):\n ```\n authn:\n oauth2:\n github:\n client_id: <% process.env.GITHUB_CLIENT_ID %>\n client_secret: <% process.env.GITHUB_CLIENT_SECRET %>\n callback_url: <% process.env.GITHUB_CALLBACK_URL %>\n callback_route: <% process.env.GITHUB_CALLBACK_ROUTE %>\n auth_route: <% process.env.GITHUB_AUTH_ROUTE %>\n success_redirect: <% process.env.GITHUB_SUCCESS_REDIRECT_URL %>\n failure_redirect: <% process.env.GITHUB_FAILURE_REDIRECT_URL %>\n ```\n\n Let's See an example event and workflow of above provided success_redirect_url\n\n Event src/events/helloUser.yaml\n ```yaml\n http.get./verify/user:\n fn: helloUser\n authn: false\n responses:\n 200:\n content:\n application/json:\n schema:\n type: string\n ```\n\n Workflow (src/functions/helloUser.ts)\n ```ts\n import { GSCloudEvent, GSContext, PlainObject, GSStatus } from \"@godspeedsystems/core\";\n export default function (ctx: GSContext) {\n const {\n inputs: {\n data: {\n user\n }\n },\n\n }= ctx;\n const name = user.username|| user.displayName ;\n return new GSStatus(true, 200, 'OK', 'Hello ' + name, {});\n }\n ```\n If your auth_route = '/auth/github', hit localhost:4000/auth/github, it will take you to your chosen provider's authorization page i.e. Github in this case.\n After authenticating the user successfully, it will redirect to the success_redirect_url i.e '/verify/user' in this case.\n\n## Authentication using oauth2\n\n## Authorization\n\nAuthorization is a crucial component of access control, determining who can access what resources and perform specific actions.\n\n### Two types of Authorization\n### Role-Based Access Control (RBAC)\nRBAC is a widely-used authorization model where access is granted based on predefined roles. Users are assigned roles, and these roles dictate the permissions associated with accessing resources and performing actions.\n\n### Attribute-Based Access Control (ABAC)\nABAC is a dynamic authorization model that considers various attributes associated with **users, resources, actions, and context**. Policies are defined based on these attributes, allowing for more granular control over access. **ABAC is a superset of RBAC**\n\n### Key Agents in Authorization\nAuthorization involves four key agents:\n\na. **User**\nUsers are entities seeking access to resources or the ability to perform actions within a system.\n\nb. **Resource**\nResources are entities or data within a system that users may want to access or manipulate.\n\nc. **Action**\nActions define the specific operations or activities that users may want to perform on resources.\n\nd. **Context**\nContext refers to the circumstances or conditions under which a user's request for access is evaluated. This includes factors such as time, location, or any other relevant contextual information.\n\n### Workflow DSL\nYou can add authorization workflow at the task level in any workflow. The authorization workflow should return allow/deny or json output to the main worklfow.\n\n**Allow/Deny**\nIf authz workflow returns data as true/false, it means the task is allowed/denied to get executed.\n\n**JSON output**\nIf authz workflow returns JSON output then it is merged with args.data of the task for which authz is being executed.\n\nHere is the sample spec:\n**Sample workflow calling the authz workflow**\n```yaml\nsummary: Call an API\ntasks:\n - id: api_step1\n description: Hit with some dummy data. It will send back same as response\n authz:\n fn: com.jfs.authz\n args: <% inputs %>\n fn: datasource.api.post./anything\n args:\n data: <% inputs %>\n```\n\n**Sample authorization workflow `com.jfs.authz`**\n```yaml\nsummary: Authorization workflow\ntasks:\n - id: authz_step1\n description: return allow/deny based upon user\n fn: datasource.authz.post./authorize\n args:\n data: <% inputs.body.user %>\n - id: authz_step2\n description: transform response from authz api\n fn: com.gs.transform\n args: |\n \n```\n\nThe authorization workflow should return response in this format to allow/deny:\n```yaml\nsuccess: true/false\ndata: true/false/JSON output\n```\n\n> When data is returned as false i.e. deny then the framework will send `403 Unauthorized` response.\n\n### Sample DB query call authorization\nIn DB query call, authz workflow can return JSON output with where clause, include clause etc. which will be merged with the args of the main workflow which is doing DB query.\n\nHere is the sample spec:\n**Sample workflow calling the authz workflow**\n```yaml\nsummary: datastore demo\ntasks:\n - id: find_user\n description: find users\n authz:\n fn: com.jfs.authz\n args: <% inputs %>\n fn: datasource.mongo.user.findMany\n args:\n data:\n include: <% inputs.body.include %>\n where: <% inputs.body.where %>\n```\n\n**Sample authorization workflow `com.jfs.authz`**\n```yaml\nsummary: Authorization workflow\ntasks:\n - id: authz_step1\n description: return allow/deny based upon user\n fn: datasource.authz.post./authorize\n args:\n data: <% inputs.body.user %>\n\n - id: authz_step2\n description: transform response from authz api\n fn: com.gs.transform\n args: |\n \n```\n\nWhen authorization workflow `com.jfs.authz` returns `success: true` then its `data` will be merged with the main workflow which is calling the authz workflow.\nFor example, in the above authz workflow, `data` is returned as:\n```yaml\ndata:\n where:\n role: 'USER'\n```\n\nThis data will be merged with the args.data of the main workflow i.e.\n```yaml\nargs:\n data:\n include: <% inputs.body.include %>\n where: <% inputs.body.where %> # where clause from authz workflow will be merged with this\n```\n\n## Reusing Definitions\n\n## Validations\n\n## File Upload\n\n## Define Event Handler Functions (GSContext, GSCloudEvent, GSStatus, Returning from functions)\n\n## Using Datasources inside functions\n\n## Using datasources with caching" }, "docs/microservices-framework/CLI.md": { "content": "---\nsidebar_position: 1\ntitle: Godspeed CLI\ndescription: A comprehensive reference for the Godspeed Command Line Interface (CLI), covering installation, core commands for project creation, development, building, and deployment, managing plugins, interacting with Prisma, enabling observability, and troubleshooting common issues.\nkeywords: [Godspeed, CLI, command line interface, installation, create, serve, build, clean, preview, gen-crud-api, plugin, prisma, otel, observability, troubleshooting, guide, documentation]\ntoc_min_heading_level: 2\ntoc_max_heading_level: 4\n---\n\n### Overview\n\nThis document provides a detailed guide to the Godspeed Command Line Interface (CLI), the primary tool for interacting with Godspeed projects. It covers how to install the CLI and explains the usage and functionality of its various commands, including those for project creation, running the development server, building and previewing projects, generating CRUD APIs, managing plugins, working with Prisma, and enabling Open Telemetry. Troubleshooting steps for common CLI errors are also included.\n\n### Godspeed Commands\n\nOnce Godspeed CLI is installed, the `godspeed` command can be called from command line. When called without arguments, it displays its help and command usage.\n```\n$ godspeed\n```\n\n```bash\n\n ,_, ╔════════════════════════════════════╗\n (o,o) ║ Welcome to Godspeed ║\n ({___}) ║ World's First Meta Framework ║\n \" \" ╚════════════════════════════════════╝\nUsage: Godspeed CLI [options] [command]\nCLI tool for godspeed framework.\nOptions:\n -V, --version output the version number\n -h, --help display help for command\nCommands:\n create [options] create a new godspeed project.\n serve run the development server in watch mode.\n build build the godspeed project.\n clean clean the previous build.\n preview run the build in a hosted environment like production or development.\n gen-crud-api scans your prisma datasources and generate\n CRUD APIs events and workflows\n plugin manage(add, remove, update) eventsource and\n datasource plugins for godspeed.\n prisma proxy to prisma commands with some add-on\n commands to handle prisma datasources.\n help [command] display help for command\n```\n\n### Create: To create a new project\n---\nThe `create` command creates project structure for your microservice. When called without arguments, it creates project structure with default examples.\n\n```bash\n$ godspeed create my-service\n```\n\n```bash\n ,_, ╔════════════════════════════════════╗\n (o,o) ║ Welcome to Godspeed ║\n ({___}) ║ World's First Meta Framework ║\n \" \" ╚════════════════════════════════════╝\n\n… waiting Cloning project template.\n✔ success Cloning template successful.\n… waiting Generating project with default examples.\n… waiting Generating project files.\n✔ success Successfully generated godspeed project files.\n\ndependencies installed successfully!\n\nSuccessfully created the project my-service.\nUse `godspeed help` command for available commands. \n\nHappy building microservices with Godspeed! 🚀🎉\n```\n---\n### Serve: To Run the service for local development\n---\nYou can run your Godspeed project using `godspeed serve` command. This will build and run your project in auto-watch mode.\n\n```bash\n godspeed serve\n```\n\n:::tip\nIn order to run a full stack application with Godspeed service as your backend, you can use Lerna. Check the example of full stack application in [gs-node-templates repository](https://github.com/godspeedsystems/gs-node-templates).\n:::\n---\n### Build: Building the service for hosted deployment\n---\nYou can build your Godspeed project using `godspeed build` command. This will build your project and copy the contents in `/dist` folder. \n\n```bash\n godspeed build\n```\n---\n### Preview: Running the service in hosted environment\nYou can run your Godspeed project using `godspeed preview` command. This will require `godspeed build` to be run prior because preview renders the project from the `dist` folder. \n```bash\n godspeed preview\n```\n### Clean: to remove the dist/ directory\n```\ngodspeed clean\n```\nRemoves the dist/ directory and resets prior build artifacts.\n\n### Plugin: Install & Manage Plugins\n---\nGodspeed plugins are the way to extend the core Meta Framework. Currently we support adding eventsource and datasource as plugin.\n```bash\n$ godspeed help plugin\n```\n```\n ,_, ╔════════════════════════════════════╗\n (o,o) ║ Welcome to Godspeed ║\n ({___}) ║ World's First Meta Framework ║\n \" \" ╚════════════════════════════════════╝\n\nUsage: Godspeed CLI plugin [options] [command]\n\nmanage(add, remove, update) eventsource and datasource plugins for godspeed.\n\nOptions:\n -h, --help display help for command\n\nCommands:\n add Gives a list of all plugins and ask you to select \n remove Gives a list of all plugins and ask you to select \n add [pluginName] Add an eventsource/datasource plugin.\n remove [pluginName] Remove an eventsource/datasource plugin.\n update Update an eventsource/datasource plugin.\n help [command] display help for command\n\nFor detailed documentation visit https://godspeed.systems\n\n```\n---\n### Plugin Add: To install godspeed plugin\n---\nThe `godspeed plugin add` command allows you to install plugins into your Godspeed project. You can use it in two ways:\n\n#### Interactive Mode (Default)\nThis opens a CLI prompt to select from available plugins:\n```bash\n$ godspeed plugin add\n```\nYou’ll see a list of officially supported plugins like `prisma-as-datastore`, `axios-as-datasource`, `aws-as-datasource`, etc., and you can choose the ones you want using the arrow keys and spacebar.\n\n```\n ,_, ╔════════════════════════════════════╗\n (o,o) ║ Welcome to Godspeed ║\n ({___}) ║ World's First Meta Framework ║\n \" \" ╚════════════════════════════════════╝\n\n? Please select godspeed plugin to install: (Press to select, to move rows)\n┌──────┬────────────────────────────────────────┬────────────────────────────────────────────────────────────────────────────────┐\n│ │ Name │ Description │\n├──────┼────────────────────────────────────────┼────────────────────────────────────────────────────────────────────────────────┤\n│ ❯◯ │ aws-as-datasource │ aws as datasource plugin for Godspeed Framework │\n├──────┼────────────────────────────────────────┼────────────────────────────────────────────────────────────────────────────────┤\n│ ◯ │ excel-as-datasource │ excel as datasource plugin for Godspeed Framework │\n├──────┼────────────────────────────────────────┼────────────────────────────────────────────────────────────────────────────────┤\n│ ◯ │ mailer-as-datasource │ mailer as datasource plugin for Godspeed Framework │\n├──────┼────────────────────────────────────────┼────────────────────────────────────────────────────────────────────────────────┤\n│ ◯ │ kafka-as-datasource-as-eventsource │ kafka as datasource-as-eventsource plugin for Godspeed Framework │\n├──────┼────────────────────────────────────────┼────────────────────────────────────────────────────────────────────────────────┤\n│ ◯ │ cron-as-eventsource │ Cron as eventsource plugin for Godspeed Framework │\n└──────┴────────────────────────────────────────┴────────────────────────────────────────────────────────────────────────────────┘\n```\n---\n### Direct Plugin Installation by package name\n---\nYou can also directly install a plugin by specifying its NPM package name:\n\n```bash\n$ godspeed plugin add @godspeedsystems/plugins-\n```\nThis is useful when scripting setups or when you already know the exact plugin you need.\n\n#### Example:\n```bash\n$ godspeed plugin add @godspeedsystems/plugins-express-as-http\n```\n\n```bash\n$ godspeed plugin add @godspeedsystems/plugins-kafka-as-datasource-as-eventsource\n```\n---\n### Plugin Remove\n---\nThe `godspeed plugin remove` command allows the user to select a plugin from the list of available plugins and remove them from the project.\n\n```bash\n$ godspeed plugin remove\n```\n```\n ,_, ╔════════════════════════════════════╗\n (o,o) ║ Welcome to Godspeed ║\n ({___}) ║ World's First Meta Framework ║\n \" \" ╚════════════════════════════════════╝\n\n? Please select godspeed plugin to uninstall: (Press to select, to move rows)\n┌──────┬────────────────────────────────────────┬────────────────────────────────────────────────────────────────────────────────┐\n│ │ Name │ Description │\n├──────┼────────────────────────────────────────┼────────────────────────────────────────────────────────────────────────────────┤\n│ ❯◯ │ express-as-http │ Godspeed event source plugin for express as http server │\n├──────┼────────────────────────────────────────┼────────────────────────────────────────────────────────────────────────────────┤\n│ ◯ │ prisma-as-datastore │ Prisma as a datasource plugin for Godspeed Framework. │\n├──────┼────────────────────────────────────────┼────────────────────────────────────────────────────────────────────────────────┤\n│ ◯ │ axios-as-datasource │ Axios as datasource plugin for Godspeed Framework │\n└──────┴────────────────────────────────────────┴────────────────────────────────────────────────────────────────────────────────┘\n```\n### Plugin Update\n\nThe `godspeed plugin update` command allows the user to select a plugin from the list of available plugins and update them.\n\n```bash\n$ godspeed plugin update\n```\n```bash\n ,_, ╔════════════════════════════════════╗\n (o,o) ║ Welcome to Godspeed ║\n ({___}) ║ World's First Meta Framework ║\n \" \" ╚════════════════════════════════════╝\n? Please select godspeed plugin to update: (Press to select, to move rows)\n┌──────┬────────────────────────────────────────┬────────────────────────────────────────────────────────────────────────────────┐\n│ │ Name │ Description │\n├──────┼────────────────────────────────────────┼────────────────────────────────────────────────────────────────────────────────┤\n│ ❯◯ │ express-as-http │ Godspeed event source plugin for express as http server │\n├──────┼────────────────────────────────────────┼────────────────────────────────────────────────────────────────────────────────┤\n│ ◯ │ prisma-as-datastore │ Prisma as a datasource plugin for Godspeed Framework. │\n├──────┼────────────────────────────────────────┼────────────────────────────────────────────────────────────────────────────────┤\n│ ◯ │ axios-as-datasource │ Axios as datasource plugin for Godspeed Framework │\n└──────┴────────────────────────────────────────┴────────────────────────────────────────────────────────────────────────────────┘\n```\n### prisma prepare: for Prisma DB Setup\n\nGodspeed wraps Prisma CLI with additional support.\n\n```bash\ngodspeed prisma prepare\n```\nThis command:\n\n- Compiles the Prisma schema from src/datasources/``.prisma\n- Generates Prisma client\n- Syncs database schema\n- Creates client in `src/datasources/prisma-clients/`\n\nUse this after editing your *.prisma schema or modifying your .env DB credentials.\n\n\n### Otel: Enabling and disabling Open Telemetry\nCLI provides `otel` command to enable/disable observability in Godspeed.\n```bash\n$ godspeed help otel\n ,_, ╔════════════════════════════════════╗\n (o,o) ║ Welcome to Godspeed ║\n ({___}) ║ World's First Meta Framework ║\n \" \" ╚════════════════════════════════════╝\nUsage: Godspeed CLI otel [options] [command]\n\nenable/disable Observability in Godspeed.\n\nOptions:\n -h, --help display help for command\n\nCommands:\n enable enable Observability in project.\n disable disable Observability in project.\n help [command] display help for command\n\nFor detailed documentation visit https://godspeed.systems\n```\n#### otel enable\nThe `godspeed otel enable` command allows the user to enable [observability](./telemetry/overview.md) in Godspeed to collect traces, metrics and logs. \n\n```bash\n$ godspeed otel enable\n ,_, ╔════════════════════════════════════╗\n (o,o) ║ Welcome to Godspeed ║\n ({___}) ║ World's First Meta Framework ║\n \" \" ╚════════════════════════════════════╝\notel installed successfully!\nObservability has been enabled\n```\n\nThe above command performs these two functions:\n\n##### A. Installs `@godspeedsystems/tracing` package\nThis package includes auto-instrumentation of the following plugins to collect traces: \n\n**1.** [http](https://nodejs.org/api/http.html) and [https](https://nodejs.org/api/https.html) requests. \n**2.** [Prisma datasource plugin](../microservices-framework/datasources/datasource-plugins/Prisma%20Datasource.md).\n\n##### B. Sets OTEL_ENABLED env variable to true\n\nBy setting `OTEL_ENABLED` to true, the following actions are performed: \n\n**1. Traces**: starts the auto-instrumentation of traces present in the `@godspeedsystems/tracing` package. \n**2. Metrics**: starts exposing application metrics at `/metrics` endpoint of the service. Currently, the framework exposes HTTP and [Prisma datasource](../microservices-framework/datasources/datasource-plugins/Prisma%20Datasource.md) metrics. \n**3. Logs**: starts dumping the service logs in [OTEL log format](./telemetry/logging.md/#log-format) in console provided NODE_ENV is not equal to 'dev'\n\n:::info\nTelemetry data for custom plugins\n\nFollow this [Github issue](https://github.com/godspeedsystems/gs-tracing/issues/2) to know how auto-instrumentation can be enabled for the other custom [eventsource](../event-sources/event-source-plugins/) and [datasource](../microservices-framework/datasources/datasource-plugins/Overview.md) plugins. \nFollow this [Github issue](https://github.com/godspeedsystems/gs-node-service/issues/1016) to know how prometheus based metrics can be exposed for the other custom [eventsource](../event-sources/event-source-plugins/) and [datasource](../microservices-framework/datasources/datasource-plugins/Overview.md) plugins.\n:::\n\n#### otel disable\n\nThe `godspeed otel disable` command allows the user to disable [observability](./telemetry/overview.md) in Godspeed. \n```bash\n$ godspeed otel disable\n ,_, ╔════════════════════════════════════╗\n (o,o) ║ Welcome to Godspeed ║\n ({___}) ║ World's First Meta Framework ║\n \" \" ╚════════════════════════════════════╝\notel uninstalled successfully!\nObservability has been disabled in the project\n```\n\nThe above command performs these two functions:\n##### A. Uninstalls `@godspeedsystems/tracing` package from your service.\n##### B. Sets OTEL_ENABLED env variable to false\nSetting `OTEL_ENABLED` to false stops all the actions performed in [otel enable](#b-sets-otel_enabled-env-variable-to-true) command \n\n### Auto Generate GraphQL schema\n\n```\ngodspeed gen-graphql-schema\n```\n\nGenerates a GraphQL schema automatically from your existing Godspeed events and workflows." }, "docs/microservices-framework/CRUD_API.md": { "content": "---\ntitle: Generating CRUD API\ndescription: Learn how to automatically generate CRUD (Create, Read, Update, Delete) APIs in Godspeed using the gen-crud-api command and Prisma. This guide covers the steps from project creation and plugin installation to setting up your database connection, creating a Prisma schema, generating the client, and finally generating and testing the CRUD APIs.\nkeywords: [Godspeed, CRUD API, auto generate APIs, gen-crud-api, Prisma, database integration, ORM, REST API, crud api tutorial, mysql, postgresql, mongodb, sqlserver, sqllite]\n---\n\n# Generating CRUD APIs with Godspeed\n\n## Overview\n\nThe Godspeed framework provides powerful CRUD API generation capabilities through the `gen-crud-api` command. This tool automatically creates standardized Create, Read, Update, and Delete (CRUD) endpoints based on your Prisma data models, significantly accelerating API development.\n\n### Key Features\n- Automatic CRUD endpoint generation\n- Prisma ORM integration\n- Express.js-based HTTP endpoints\n- Swagger documentation generation\n- Database schema synchronization\n\n### Tutorial\nLearn how to generate CRUD APIs in Godspeed:\n\n
\n\n
\n\n## Prerequisites\n\nBefore generating CRUD APIs, ensure you have:\n- Godspeed CLI installed\n- Access to a supported database\n- Basic understanding of Prisma schema\n\n## Step-by-Step Implementation Guide\n\n### 1. Project Setup\n\n**Create a new project or skip if already created.:**\n ```bash\n godspeed create my-project\n ```\n\n**Navigate to the project directory:**\n ```bash\n cd my-project\n ```\n\nFor installation and getting started, visit the [Getting Started Guide](./guide/get-started)\n\n### 2. Prisma Plugin Installation\n\nInstall the Prisma datasource plugin:\n```bash\ngodspeed plugin add @godspeedsystems/plugins-prisma-as-datastore\n```\n\nFor plugin configuration details, refer to the [Prisma Plugin Documentation](./datasources/datasource-plugins/Prisma%20Datasource.md#how-to-add-plugin)\n\n### 3. Database Configuration\n\n#### Connection URL Setup\nAdd your database connection URL to the `.env` file:\n\n```env\n# PostgreSQL example\nDATABASE_URL=\"postgresql://username:password@localhost:5432/dbname\"\n\n# SQLite example\nDATABASE_URL=\"file:./database.db\"\n```\n\n### Supported Databases\n\nFor database-specific connection formats, see:\n- [Supported Databases List](/docs/microservices-framework/databases/Overview#list-of-currently-supported-databases)\n- [Connection URL Formats](/docs/microservices-framework/databases/MySQL#connection-url)\n\n### 4. Prisma Schema\n\nCreate a Prisma schema file in `src/datasources/`:\n:::tip\nWhen configuring the Prisma client in your Godspeed project, ensure you add the `output field` in your Prisma schema's generator block. This field should point to this location `src/datasources/prisma-clients/` where the generated prisma client files will be stored.\n:::\n\n```prisma title=src/datasources/schema.prisma\n\ndatasource db {\n provider = \"postgresql\" // or mysql, sqlite, sqlserver\n url = env(\"DATABASE_URL\")\n}\n\ngenerator client {\n provider = \"prisma-client-js\"\n output = \"./prisma-clients/schema\" //here schema is the name of prisma schema file\n previewFeatures = [\"metrics\"]\n}\n\nmodel User {\n id Int @id @default(autoincrement())\n email String @unique\n name String\n createdAt DateTime @default(now())\n updatedAt DateTime @updatedAt\n}\n```\n\n#### Important Configuration Notes\n1. Set the correct `provider` for your database\n2. Configure the `output` path for generated client\n3. Enable `metrics` in previewFeatures if needed for telemetry\n\n#### Existing Database Integration\n\nIf you already have an existing database, you can introspect it and generate the Prisma model file using prisma db pull. This will generate your .prisma file.\n\nCopy the generated file to src/datasources folder and rename it as per the name of this datasource that you want to keep. If you don't have an existing database setup with a model, then create a prisma model file from scratch.\n\n\n### 5. Client Generation and Database Sync\n\nGenerate Prisma client and sync database:\n```bash\ngodspeed prisma prepare\n```\n\nThis command:\n- Generates Prisma client\n- Syncs database schema\n- Creates client in `src/datasources/prisma-clients/`\n\n### 6. CRUD API Generation\n\nGenerate CRUD endpoints:\n```bash\ngodspeed gen-crud-api\n```\n\nSelect your schema when prompted:\n```\nSelect datasource / schema to generate CRUD APIs\n(x) schema.prisma\n( ) For all\n( ) Cancel\n```\n\n### 7. Testing Generated APIs\n\nStart the server and test APIs:\n```bash\ngodspeed serve\n```\n\nAccess Swagger documentation at:\n```\nhttp://localhost:3000/api-docs\n```\n\n## Generated Components\n\nThe generator creates:\n1. Event definitions\n2. API routes\n3. Functions\n4. Swagger documentation\n\n![Generated CRUD Components](../../static/img/generated_crud_api.png)\n\n## Troubleshooting\n\n### Common Issues\n\n1. Database Connection Errors\n - Verify connection URL format\n - Check database credentials\n - Ensure database is running\n\n2. Schema Generation Issues\n - Validate Prisma schema syntax\n - Check output directory permissions\n - Verify database provider support\n\n3. API Generation Problems\n - Confirm Prisma client generation\n - Check for schema validation errors\n - Verify event source configuration\n\nRemember to regularly update generated APIs as your data models evolve, and maintain proper version control of your schema files.\n" }, "docs/microservices-framework/authentication/jwt-auth-old.md": { "content": "---\ntitle: JWT Authentication\ndescription: Learn about JWT authentication in Godspeed, including enabling JWT, disabling authentication on endpoints, and accessing JWT payload in event handlers or workflows.\nkeywords: [Godspeed, JWT, authentication, event source, plugin, payload]\n---\n# JWT Authentication\nAll currently available _sync_ event sources support JWT authentication mechanism out of the box. For ex. Apollo Graphql, Express, Fastify.\n\n## Enabling JWT Authentication\nYou can [setup jwt configuration](./jwt-authentication.md) in the event source's configuration file, and override it in each individual event as applicable.\nThe plugins follow zero trust policy as a first principle, so if you have setup jwt spec at event source level, all endpoints will go through JWT authentication, unless you explicityly set `authn:false` in their schema.\n\n### Disabling authentication on an endpoint\n```yaml\nhttp.post./loan-application: \n fn: com.biz.kyc.ckyc.ckyc_initiate\n authn: false # explicitly disable jwt authentication on this endpoint\n \n```\n\n## For plugin creators\nFor handling JWT , it is recommended that each event source plugin adheres to a standardized JWT handling configuration. In the case of JWT, the configuration typically includes details such as the `issuer`, `audience`, and `secretOrkey`.\n\n\nYou can configure JWT settings within the `eventsources/.yaml`. Here's an example of such a configuration:\n\n### Example Configuration\n\n```yaml\ntype: express\njwt:\n issuer: <#config.issues#> # must be equal to the key iss in your jwt token\n audience: <#config.audience#> #must be equal to the key aud in your jwt token\n secretOrKey: <#config.secret#>\n```\nThe provided snippet contains payload information and a secret key. Once the above snippet is added to the `eventsources/http.yaml`, authentication for all the events will be *true* by default. \n\n:::tip Note\nThe Express plugin is implemented using passport JWT . To know more check [passport documentation](https://www.passportjs.org/)\n:::\n\nThe options supported by passport sdk are:\n\n![jwt_config_options](https://docs.godspeed.systems/assets/images/jwtconfig_options-7c650cde2021eae6cdc15d4029afe6ff.png) \n\nWhen configuring the JWT settings, if you do not provide either the `secretOrKeyProvider` or the `secretOrKey` property from the configuration options mentioned above, it will result in an error.\n\nAdditionally, if you specify an `issuer` or `audience` value in the configuration, and the token values differ from those specified in the configuration payload, the response will be 'Unauthorized.'\n\n\n## Access JWT payload in event handlers or workflows\nYou can access the complete JWT payload in `<% inputs.user %>` in YAML workflows inline scripts, and as `ctx.inputs.data.user` when writing JS/TS functions\n\n\n\n### Example JS/TS workflow\n```typescript\nexport default async function (ctx: GSContext) {\n \n //Ctx object has the basics you need to write any business logic in TS/JS\n const {\n inputs\n } = ctx;\n //Accessing deserialized inputs from the event source\n const {user, body, params, query, headers} = inputs.data;\n return {'user': inputs.data.user};\n //return new GSStatus(true, 200, 'OK', {'user': inputs.data.user});\n}\n```" }, "docs/microservices-framework/authentication/jwt-authentication.md": { "content": "---\nid: jwt-authentication\ntitle: JWT Authentication\ndescription: Learn how to configure JWT authentication in Godspeed for Express, Apollo, and Fastify event sources.\nkeywords:\n - jwt\n - authentication\n - godspeed jwt authentication\n - express jwt auth\n - GraphQL jwt\n - Authentication in fastify\n---\n\n**JWT (JSON Web Token)** is a standard method for securely transmitting information between two parties as a JSON object. It’s commonly used for authentication and authorization in web applications.\nIn Godspeed, you can easily implement JWT authentication to protect routes, control access to resources and ensure secure API communication.\nCurrently supported event sources which can leverage this mechanism are:\n1. Express\n2. Apollo GraphQL\n3. Fastify\n\nTo Learn more about JWT and Payload [Click Here](/docs/microservices-framework/authentication/jwt)\n\n\n### How JWT Authentication Works in Godspeed\nWhen a client makes a request to a protected route, they need to include a valid JWT in the request header. \nThe server validates the token, extracts the payload, and grants access to the resource if the token is valid.\nIn the case of an expired or invalid token, the client will receive a `401 Unauthorized` response.\n\n:::tip Note\nIn our Express eventsource plugin, JWT Authentication is implemented using passport-jwt which is a strategy for authenticating with a JSON Web Token. To know more about, you can check [passport documentation](https://www.passportjs.org/)\n:::\n\n### Setup and implementation of JWT authentication in Godspeed.\n\n### Step 1: Setting up Environment\n\n**1.** In Godspeed, any configuration which includes secrets or passwords is recommended to be defined using environment variables only. For this, open custom-environment-variables.yaml file which is placed under `config/` in your root project directory and add your jwt configs there like:\n\n ```\n jwtSecret: JWT_SECRET\n audience: JWT_AUDIENCE \n issuer: JWT_ISSUER\n\n ```\n\n**2.** Now, you need to export these variables in environment. This can be done in two ways:\n\n **(a)** Set environment variables in .env file which is under your project's root folder /.env as shown below:\n ```\n JWT_SECRET= mysecret #the secret\n JWT_AUDIENCE= godspeedsystemms #aud in jwt token\n JWT_ISSUER= godspeed #iss in jwt token\n\n ```\n\n **(b)** Export these variables to your environment, follow the below syntax to export, based on the shell, you are using:\n
\n For git bash \n\n ```bash\n $ export JWT_SECRET=mysecret\n $ export JWT_ISS= mycompany\n ```\n\n
\n\n
\n For windows powershell \n\n ```bash\n $env:JWT_SECRET= \"mysecret\"\n $env:JWT_ISS= \"mycompany\" \n ```\n\n
\n\n After exporting the environment variable, you can access these variable anywhere in your project by using inline\n scripting `<%config.issuer%>` in yaml or as `ctx.config.issuer` in js/ts workflows.\n\n:::tip Note \nIf you do not set these environment variables mentioned above, it will result in an error while running your project. And if the token values set in header differ from those specified in the configuration, the response will be 'Unauthorized.'\n:::\n\n### Step 2: Enable JWT Auth in your project's eventsource configuration file.\nJWT configuration is written under authn: in the event source's configuration file. For Express, config file name will be http.yaml. Open this file and Set up jwt authn as shown below.\n\n`src/eventsources/http.yaml`\n```\n type: express\n port: 4000\n #auth settings to run by default on every event\n authn:\n jwt:\t\t\t\n secretOrKey: <% config.jwtSecret %> # to access jwtSecret from config\n audience: <% config.audience %> \n issuer: <% config.issuer %> \n```\nOnce you have enabled it here, authentication will be true for all endpoints, unless you explicitly set authn:false in their event schema.\n\n\n### Disabling JWT Authentication at Event Level\n\n The plugins follow zero trust policy as a first principle, so if you have setup jwt spec at event source level, authentication for all the events will be true by default, unless you explicitly set authn:false in their event schema.\n If you don't want users to be authenticated, you can disable any end-point by writing authn: false in your event schema like this:\n\n ```\n http.get./helloworld:\n fn: helloworld\n authn: false\n params:\n - name: name\n in: query\n required: true\n ```\n\n\n### User Login Example using JWT Authentciation \n\n**Event**\n```yaml\n# Login with username and password\nhttp.post./login: # defines the POST request that will be triggered when a client hits /login endpoint.\n fn: verifyLogin # the workflow to handle the request\n authn: false\n body:\n content:\n application/json:\n schema:\n type: object\n properties:\n username:\n type: string\n password:\n type: string\n required:\n - username\n - password\n responses:\n '200':\n description: 'Login successful'\n content:\n application/json:\n schema:\n type: object\n '401':\n description: 'Invalid credentials'\n content:\n application/json:\n schema:\n type: string\n example: 'Invalid username or password'\n```\n**Workflow (verifyLogin.ts)**\n```ts\n import { GSCloudEvent, GSContext, PlainObject, GSStatus, logger } from \"@godspeedsystems/core\";\n import jwt from 'jsonwebtoken';\n export default function (ctx: GSContext) {\n const {\n inputs: {\n data: {\n body\n }\n }, \n \n }= ctx;\n logger.info(\"user info received %o\", body);\n // Dummy user validation (replace with your authentication logic)\n if (body.username === 'user' && body.password === 'password') {\n // Create a JWT token\n const token = jwt.sign(\n { sub: '1234567890', name: 'John Doe', role: 'user' }, // Payload\n ctx.config.jwtSecret, // access Secret key from config\n { expiresIn: '1h', issuer: ctx.config.issuer, audience: ctx.config.audience } // jwt Options\n );\n logger.info(\"Token generated %s\", token);\n return new GSStatus(true, 200, 'Login Successful',{JWT: token}, {}); \n } else {\n return new GSStatus(true, 401, 'Failed', 'Invalid Credentials', {}); \n }\n }\n```\n\n\n### How to access JWT payload\nYou can access the complete JWT payload in `<% inputs.user %>` in YAML workflows, and as ctx.inputs.data.user when writing JS/TS workflows.\n\nExample access from TS workflow\n```ts\nimport { GSCloudEvent, GSContext, PlainObject, GSStatus, logger } from \"@godspeedsystems/core\";\nexport default function (ctx: GSContext) {\n const {\n inputs: {\n data: {\n user\t\t\n }\n }, \n }= ctx;\n\nreturn new GSStatus(true, 200, 'OK', {'Payload user': user}, {}); \n}\n\n```\n\n### Standardized JWT Configuration (For Plugin Creators)\n\nFor consistency across plugins, it's recommended to use a standardized configuration format for JWT settings (issuer, audience, secret key) in eventsources/.yaml.\n\n" }, "docs/microservices-framework/authentication/jwt.md": { "content": "---\ntitle: What is JWT?\ndescription: Learn about JWT (JSON Web Token), its structure, authentication process, and example payload.\nkeywords: [JWT, JSON Web Token, authentication, payload, header, signature]\n---\n### What is JWT? \nA JWT consists of three parts:\n1. **Header**: Contains the signing algorithm and token type (e.g., `HS256`, `JWT`).\n2. **Payload**: Stores claims or data such as user identity, roles, and permissions.\n3. **Signature**: Ensures the integrity of the token. It’s created using the header, payload, and a secret key.\n\n \n ### Authentication Process using JWT: \n 1. Client login: The client sends credentials (e.g., username and password) to the server.\n 2. Token creation: Upon successful login, the server generates a JWT signed with a secret and sends it to the client.\n 3. Token usage: The client includes the JWT in the `Authorization` header of subsequent requests to access protected resources.\n 4. Token validation: The server validates the JWT in incoming requests, checking its signature, expiration, and payload.\n 5. Authorization: If the token is valid, the server grants access to the requested resource.\n\n\n### A typical JWT looks like this:\n```\neyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.\neyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiaWF0IjoxNTE2MjM5MDIyfQ.SflKxwRJSMeKKF2QT4fwpMeJf36POk6yJV_adQssw5c\n```\n### Example JWT Payload \n\n```\n{\n \"iss\": \"https://your-app.com\",\n \"sub\": \"user123\",\n \"aud\": \"https://your-app.com/api\",\n \"exp\": 1692425600,\n \"nbf\": 1692425000,\n \"iat\": 1692424400,\n \"jti\": \"token123\",\n \"username\": \"john.doe\",\n \"email\": \"john.doe@example.com\",\n \"roles\": [\"admin\", \"user\"]\n}\n```\n" }, "docs/microservices-framework/authentication/oauth2-authentication.md": { "content": "---\ntitle: OAuth2 Authentication\ndescription: Learn about OAuth2 authentication in Godspeed, including configuration, setup, and implementation.\nkeywords: [Godspeed, OAuth2, authentication, authorization, Google, GitHub, LinkedIn]\n---\n# OAuth2 Authentication\n\n## Introduction\n OAuth2 is a widely adopted authorization framework that provides a secure and efficient way for users to grant third-party applications access to their data without revealing their credentials. By leveraging OAuth2, Godspeed simplifies the authentication process for both developers and users. OAuth2 offer the ability to connect with authentication providers such as Google, Github and Linkedin.\n\n### How It Works:\n1. **User Signup/Signin:** When a user want to signup with google/github/linkedin, express-eventsource will trigger OAuth2 authentication flow.\n2. **Redirect to Provider:** The user is redirected to the chosen authentication provider (Google, LinkedIn, or GitHub) to authorize the application.\n3. **Authorization Grant:** The provider grants the application an authorization code.\n4. **Token Exchange:** The application exchanges the authorization code for an access token.\n5. **User Information Retrieval:** The application uses the access token to retrieve the user's information from the provider's API.\n6. **Authentication Success:** Once the user's identity is verified, they are logged in to the platform.\n\n## Getting Started: \n To start using OAuth2 authentication in your Godspeed project, follow these steps:\n\n### 1. Configure Provider:\n Set up your OAuth2 application with the chosen authentication provider (Google, LinkedIn, or GitHub) to obtain the necessary client ID and client secret.\n\n### 2. Set up Environment Variables:\n Open .env file in your project's root directory.\n Add the following lines, replacing the placeholders with your actual GitHub credentials:\n ```\n GITHUB_CLIENT_ID= \"your_client_id\"\n GITHUB_CLIENT_SECRET= \"your_client_secret\"\n GITHUB_CALLBACK_URL= \"your_callback_url\" e.g http://localhost:4000/auth/github/callback\n GITHUB_AUTH_ROUTE = \"/auth/github\"\n GITHUB_CALLBACK_ROUTE = \"/auth/github/callback\"\n GITHUB_SUCCESS_REDIRECT_URL = \"/verify/user\"\n GITHUB_FAILURE_REDIRECT_URL = \"/error\"\n # Session Secret\n SESSION_SECRET = \"your_secret\"\n ```\n### 3. Configure your eventsource\n\n Here's an example of OAuth2 configuration inside (src/eventsources/http.yaml):\n ```\n authn:\n oauth2:\n github:\n client_id: <% process.env.GITHUB_CLIENT_ID %> \n client_secret: <% process.env.GITHUB_CLIENT_SECRET %> \n callback_url: <% process.env.GITHUB_CALLBACK_URL %>\n callback_route: <% process.env.GITHUB_CALLBACK_ROUTE %>\n auth_route: <% process.env.GITHUB_AUTH_ROUTE %>\n success_redirect: <% process.env.GITHUB_SUCCESS_REDIRECT_URL %>\n failure_redirect: <% process.env.GITHUB_FAILURE_REDIRECT_URL %>\n ```\n\n Let's See an example event and workflow of above provided success_redirect_url \n \n Event src/events/helloUser.yaml\n ```yaml\n http.get./verify/user:\n fn: helloUser\n authn: false\n responses:\n 200:\n content:\n application/json:\n schema:\n type: string\n ```\n \n Workflow (src/functions/helloUser.ts)\n ```ts\n import { GSCloudEvent, GSContext, PlainObject, GSStatus } from \"@godspeedsystems/core\";\n export default function (ctx: GSContext) {\n const {\n inputs: {\n data: {\n user\n }\n }, \n \n }= ctx;\n const name = user.username|| user.displayName ;\n return new GSStatus(true, 200, 'OK', 'Hello ' + name, {}); \n }\n ```\n If your auth_route = '/auth/github', hit localhost:4000/auth/github, it will take you to your chosen provider's authorization page i.e. Github in this case. \n After authenticating the user successfully, it will redirect to the success_redirect_url i.e '/verify/user' in this case.\n\n\n### To learn more about OAuth2 Authentication in Godspeed, please watch the video provided below…\n\n
\n\n
\n\n\n\n" }, "docs/microservices-framework/authentication/overview.md": { "content": "---\ntitle: About Authentication\ndescription: Overview of authentication in Godspeed, including confirming identity and delegating authentication to individual eventsource plugins.\nkeywords: [Godspeed, authentication, identity, eventsource, plugin, JWT, OAuth2]\n---\n# About Authentication\n\nAuthentication is the process of confirming the identity of an individual, system, or entity. It involves verifying that the entity attempting to access a system or resource is indeed who or what it claims to be. In case of the API and event driven architecture realm, we need to know the user who is trying to access the system. _Who is this user?_\n\nFollowing separation of concerns (or decoupling) as a first principle, the job of authentication and loading the user information is delegated to individual eventsource plugins as part of their native `middleware` capability. Here you may use JWT, Auth0, OAuth2, Keycloak etc. as per your requirement.\n\nThe currently supported Express, Fastify and Apollo Graphql [plugins](https://github.com/godspeedsystems/gs-plugins) support JWT and OAuth2 authentication out of the box. In case you need to customize, you can copy the code from our plugins repository and modify that to suit your purpose.\n\n" }, "docs/microservices-framework/authorization/authz-usecases.md": { "content": "---\ntitle: Authorization in the framework\ndescription: Learn about authorization workflows in the framework, including setting up authorization at the eventsource, event, and task levels.\nkeywords: [authorization, workflows, eventsource, event, task, user, resource, context, action]\n---\n# Authorization in the framework\n\nThe authorization workflows in the framework can be setup at \n**i. ** The eventsouce level, applicable to all events \n**ii. ** At the event level, overriding eventsource configuration \n**iii. ** At the task level, restricting access of the user to the information stored in a datastore\n```yaml\nauthz: \n - id: authz_task_1\n summary: apply the rules over enriched context. Returns true or false or a GSStatus\n fn: | \n <% \n success: user.role === 'admin' && user.department === 'HR'\n %>\n```\n\n## What do we need for authorization\nWe need the user, resource, context and action information. This information can be accessed from `ctx.user.data.{user,headers,params,query,body}`. We also need the access policies or rules. These can exist within the microservice or API code, or they can be loaded from a rule engine or a databse, and executed here in the runtime of this event handling. \n\n### Loading the necessary user information for authorization\n**1. ** The `GSCloudEvent.data.user `object is initially populated by the middleware in the eventsource. For ex. in the JWT case. It serializes and loads the user data like role, id or any other necessary detail for the subsequent authz workflows, or the event handler workflow to use. \n**2. ** You can enrich the user in your `authz` workflows. The framework allows you to enrich user information in the authz workflows. \n**3. ** Its a matter of careful design where you choose to store and retrieve your user data and access policies. Having said that, the framework allows you full freedom to enrich user data in either the event source's middleware or in authz workflows. \n\n\n\n## Authz workflows\nThe framework allows you to specify yaml taks to run authorization workflows, at three places - event source, event or yaml task.\n\n**i. ** The approach for handling JWT or similar keys is to implement a task within the authorization (authz) workflow. \n**ii. ** The authz workflow can include a task specifically designed to process and enrich the user object with information obtained from the JWT or other keys. \n**iii. ** Finally the authz has to tell whether the user is allowed to do this action, and if yes, then what kind of data is the user allowed to retrieve in response? Hint: consider database access? Even if a user is allowed to read a table, is he allowed to read all the rows? Or is he allowed to read all the columns?\n\n### Example authorization workflows\nDevelopers can define the authorization workflow using the following method.\n\n\n\n#### Authorization tasks DSL\n\n**-** Developers can define the authorization workflow by providing a set of tasks using the DSL provided by the core framework's workflows \n**-** This approach allows for a declarative definition of tasks within the authz workflow.\n\n```yaml\n# example \nauthz: \n - id: enrich_context\n summary: |\n Enrich user information based on his parsed JWT, \n from a policy engine, DB, some datasource\n fn: com.biz.enrich_user_context \n args: \n success: true\n # if above task returns GSStatus.success==true\n - id: fetch_rules\n summary: |\n in case we need dynamically fetched and evaluated rules \n from a rule engine like Drools or JRE rule engine storing rules\n fn: com.biz.load_dynamic_rules\n # if above task returns GSStatus.success==true or just true, continue to next task and so on\n - fn: com.gs.evaluate_gre\n```\n\n**-** Think of authz instruction as a workflow. It will accept an array of task or a single task, each of which should return GSStatus or true. \n**-** These instructions may enrich the context and user data, or load and run authorization checks, or do both. \n**-** When one task fails, the whole workflow is considered to be failed. \n\n:::tip Note\n- Following Zero trust policy, when a task does not return `true` or `{success: true}` (explicitly), or in the case when `GSStatus.code === 403`, the authorization will be considered as failed. \n- Next tasks in the series of tasks will not be executed. The event or workflow task at which the user's authz has failed will exit.\n:::\n\n### Response code, message and data from authorization failure\n\n**1. If GSStatus is returned with success: false ,default code is 403, unless developer specified a 4XX or 5XX code**\n##### Sample authz task\n```yaml\n authz: \n id: task_authz\n tasks:\n - fn: com.gs.transform\n id: try_auth_3\n args:\n success: false\n code: 401\n```\n\n##### Response \n\"rule1_op\"\n\n**2. Framework won't accept 2XX and 3XX status codes when authz task is failed. It will return the default 403**\n\n##### Sample authz task\n```yaml\n authz: \n id: task_authz\n tasks:\n - fn: com.gs.transform\n id: try_auth_3\n args:\n success: false\n code: 200\n```\n##### Response \n\"rule2_op\"\n\n\n**3. If GSStatus.message is set and GSStatus.data is not set, then GSStatus.message is returned as response body**\n\n##### Sample authz task\n```yaml\n authz:\n - fn: com.gs.transform\n id: try_auth_2_authz\n args: | #assume the below if condition is true, then the control goes inside if block and prints the message\n \n\n```\n\n##### Response\n\n\n\"rule3_op\"\n\n\n**4. If GSStatus.message is set and GSStatus.data is set, but 'GSStatus.data.message' is not set, then `{...data,message}` is returned in the response body**\n\n\n##### Sample authz task\n```yaml\n authz:\n - fn: com.gs.transform\n id: try_auth_2_authz\n args: |\n \n```\n\n##### Response\n\"rule3_op\"\n\n\n**5. If GSStatus.message is set and GSStatus.data is set, and 'GSStatus.data.message' is also set, then data is returned in the response body. Meaning GSStatus.data.message has precedence over GSStatus.message**\n\n##### Sample authz task\n```yaml\n authz:\n - fn: com.gs.transform\n id: try_auth_2_authz\n args: |\n \n```\n\n##### Response\n\"rule3_op\"\n\n## Examples \nIn the Framework, authorization can be implemented at different levels: event level, task level, and even within datasource plugins. Each level offers flexibility and customization options to meet specific requirements\n\n:::tip Note\n- Following Zero Trust Policy, authz configuration can also be set at the event source level, serving as the default configuration.\n- Unless an event explicitly specifies `authz: false` or overrides its authz settings, all events will inherit the authz configuration from the event source.\n:::\n\n\n\n\n### A. Authorization at event source level\nYou can set authz workflows that apply to all events of a particular event source, unless explicitly overriden by an event definition by setting `authz: false` or `authz:` to another workflow.\n\n```yaml\ntype: express\nauthz: # enabling authz in event level\n - fn: com.gs.transform \n id: authz_task\n args: | # if this condition fails, the else gets executed\n \n```\n\n### B. Authorization at event level \n```yaml title=events/helloworld.yaml\n\"http.get./helloworld\":\n authn: true\n fn: helloworld # if the below authentication condition returns true, fn helloworld gets called\n authz: # enabling authz in event level\n - fn: com.gs.transform \n id: try_auth_2_authz\n args: | # if this condition fails, the else gets executed\n \n body:\n content:\n application/json:\n schema:\n type: object\n properties:\n name:\n type: string\n responses:\n '200':\n description: Successful response\n content:\n application/json:\n schema:\n type: object\n properties:\n message:\n type: string\n '400':\n description: Bad request response\n\n```\nWhenever an API is triggered with authorization enabled, the event source plugin parses the JWT in its request middleware and verifies user data, such as user.role in the example above. \nIf the condition evaluates to true, the corresponding workflow is executed. Otherwise, it proceeds to execute the else case, indicating \"Authorization failed.\"\n\n### C. Authorization at task level \n```yaml title=functions/helloworld.yaml\nid: helloworld_workflow\ntasks:\n - id: helloworld_workflow_first_task\n fn: com.gs.transform\n args: \n code: 200\n success: true\n data: | #sum_sumindex(4,5) is used to call the function sumindex(x,y) written in sum.js file\n \n on_error:\n continue: true\n response:\n success: true\n code: 500\n data: {}\n authz: #Enabling authz in task level\n id: task_authz\n tasks:\n - fn: com.gs.transform\n id: try_auth_3\n args:\n success: <% inputs.headers.x == \"ayush\" %>\n code: 200\n \n```\n\n\n\n\n\n### D. Restricting datastore access\n\n**1. ** Plugins can access user data through `args.meta.authzPerms` in the `GSDatasource.execute()` method. The structure of this data is defined by the plugin, following the format supported by the specific datasource plugin. \n**2. ** For example, it could include fields like `{can_access, no_access, where}`. \n```json title='Sample output of authz workflow authz_wf.yaml'\ndata: [ \n //Your access policies for further finegrained datastore access, \n // as interpreted and used by the datastore plugin to implement restricted database access over any database. \n { \n userTableName: { //to be merged in DB query by the datastore plugin \n where: { //additional filters to be applied on the DB query\n tenant_id: \"xyz\", sub_tenant_id: \"abc\" \n }, \n no_access: ['PII-A', 'sensitiveB'], //columns which should not be accessible\n can_access: ['tenant_id'] //columns which are accessible\n } \n }\n ]\n```\n\n**3. ** Subsequently, the plugin is responsible for adjusting its query to the datasource based on the information provided in args.authzPerms\n\n```ts\n async execute(ctx: GSContext, args: PlainObject): Promise {\n const { childLogger } = ctx;\n const {\n meta: { entityType, method, fnNameInWorkflow, authzPerms }, ...rest } = args as { meta: { entityType: string, method: string, fnNameInWorkflow: string, authzPerms: AuthzPerms }, rest: PlainObject };\n if (authzPerms) {\n const authzFailRes = modifyForAuthz(this.client, rest, authzPerms, entityType, method);\n if (authzFailRes) {\n return authzFailRes;\n }\n }\n // Now authz checks are set in select fields and passed in where clause\n let prismaMethod: any;\n try {\n const client = this.client;\n if (entityType && !client[entityType]) {\n logger.error('Invalid entityType %s in %s', entityType, fnNameInWorkflow);\n return new GSStatus(false, 400, 'Invalid entityType', { error: `Invalid entityType ${entityType} in ${fnNameInWorkflow}`});\n }\n prismaMethod = client[entityType][method];\n if (method && !prismaMethod) {\n logger.error('Invalid CRUD method %s in %s', method, fnNameInWorkflow);\n return new GSStatus(false, 500, 'Internal Server Error', { error: 'Internal Server Error'});\n }\n\n const prismaResponse = await prismaMethod.bind(client)(rest);\n return new GSStatus(true, responseCode(method), prismaResponse, prismaResponse);\n } catch (error: any) {\n logger.error('Error in executing Prisma query for args %o \\n Error: %o', args, error);\n return new GSStatus(false, 400, error.message, JSON.stringify(error.message));\n }\n }\n```\n\n#### Example: [prisma-as-datasource plugin](../datasources/datasource-plugins/Prisma%20Datasource.md/#database-authorization)\n
\nRestricted datastore access\nIn the below authz workflow can_acces, no_access and where conditions are provided. These conditions will be applied while fetching author details.\n\n```yaml title=authz_wf.yaml\nauthz: \n - id: authz_task_1\n summary: return access columns\n fn: com.gs.transform\n args:\n can_access: \n - col1\n - col2\n no_access:\n - col3\n where:\n tenant: <% inputs.headers.client_id %>\n```\n\n```yaml title=fetch_author.yaml\nsummary: Fetch author\ntasks:\n - id: fetch_author\n fn: datasource.mysql.author.findUnique\n authz:\n - fn: authz_wf\n args: <% inputs %>\n args:\n where:\n id: <% inputs.params.id %>\n```\n
\n" }, "docs/microservices-framework/authorization/overview.md": { "content": "---\ntitle: About Authorization\ndescription: Overview of authorization, including RBAC, ABAC, key agents, and workflow DSL.\nkeywords: [authorization, RBAC, ABAC, user, resource, action, context, workflow DSL]\n---\n\n# About Authorization\nAuthorization is a crucial component of access control, determining who can access what resources and perform specific actions. \n\n\"event\n\n## Two types of Authorization\n### Role-Based Access Control (RBAC)\nRBAC is a widely-used authorization model where access is granted based on predefined roles. Users are assigned roles, and these roles dictate the permissions associated with accessing resources and performing actions.\n\n### Attribute-Based Access Control (ABAC)\nABAC is a dynamic authorization model that considers various attributes associated with **users, resources, actions, and context**. Policies are defined based on these attributes, allowing for more granular control over access. **ABAC is a superset of RBAC**\n\n### Key Agents in Authorization\nAuthorization involves four key agents:\n\na. **User**\nUsers are entities seeking access to resources or the ability to perform actions within a system.\n\nb. **Resource**\nResources are entities or data within a system that users may want to access or manipulate.\n\nc. **Action**\nActions define the specific operations or activities that users may want to perform on resources.\n\nd. **Context**\nContext refers to the circumstances or conditions under which a user's request for access is evaluated. This includes factors such as time, location, or any other relevant contextual information.\n\n" }, "docs/microservices-framework/config-and-mappings/config.md": { "content": "---\ntitle: Config\ndescription: Learn about configuration variables, environment variables, and static variables in Godspeed.\nkeywords: [configuration, environment variables, static variables, YAML, config directory]\n---\n# Config\n\nThe configuration variables, along with their respective values, are specified within YAML files located in the `config/` directory. These variables are easily customizable to align with specific business use cases. \n### Config structure\nThe default directory structure is outlined as follows\n\n```\n├── config\n│   ├── custom-environment-variables.yaml\n│   ├── default.yaml\n├\n├── .env\n```\n\n### File naming and Load order\nThe configuration files under `config/` directory can have specific naming conventions and load order. Please refer [File name and Load order](https://github.com/node-config/node-config/wiki/Configuration-Files#file-load-order) for more information.\n\n\n## Environment variables\n\nEnvironment variables in Godspeed are essential for securely managing configuration details like API keys, database URLs, and other sensitive information. Godspeed allows two primary ways to configure and access these variables.\n\n### Configuration Setup for Environment Variables\n\nEnvironment variables can be set up using: \n1. **Define variables** in `config/custom-environment-variables.yaml`.\n2. **Set values** in `.env` or use `export` in the terminal.\n3. **Access variables** using `<% config.variable_path %>` in yaml and `ctx.config` in typeScript workflows.\n\n### Step 1: Define Environment Variables in YAML Configuration\n\nIn the `config/custom-environment-variables.yaml` file, specify the environment variables you want to use across your project. This YAML file **maps environment variables** to meaningful keys, which you’ll use in your code.\n\n**Example `custom-environment-variables.yaml`:**\n\n```yaml\nmy_datasource:\n base_url: MY_DATASOURCE_BASE_URL\n api_key: MY_DATASOURCE_API_KEY\n api_token: MY_DATASOURCE_API_TOKEN\n\nkafka:\n brokers:\n __name: KAFKA_BROKERS\n __format: json\n client_id: KAFKA_CLIENT_ID\n\njwt:\n issuer: JWT_ISS\n audience: JWT_AUD\n secretOrKey: JWT_SECRET\n\nprisma_secret: PRISMA_SECRET\n```\n\n### Step 2: Set Environment Variable Values\n\nAfter defining environment variable keys in `custom-environment-variables.yaml`, you can provide actual values in two ways:\n\n1. **Define in `.env` file**:\n - Open `.env` file and assign values to each variable specified in `custom-environment-variables.yaml`.\n\n **Example `.env` file:**\n ```plaintext\n MY_DATASOURCE_BASE_URL=https://httpbin.org/\n MY_DATASOURCE_API_KEY=your_api_key_here\n MY_DATASOURCE_API_TOKEN=your_api_token_here\n MY_ACCESS_TOKEN=yor_access_token\n \n KAFKA_BROKERS=[\"localhost:9092\"]\n KAFKA_CLIENT_ID=my-kafka-client\n \n JWT_ISS=https://your-issuer.com\n JWT_AUD=https://your-audience.com\n JWT_SECRET=your_jwt_secret\n \n PRISMA_SECRET=your_prisma_secret\n ```\n\n2. **Set values directly in the environment**:\n You can export these variables temporarily to environment by following the below syntax, based on the environment you are using:\n - **For shell/git bash**, use `export` command as:\n ```bash\n $ export MY_DATASOURCE_BASE_URL=https://httpbin.org\n $ export JWT_SECRET=mysecret\n $ export JWT_ISS= mycompany\n ```\n - **For windows powershell**\n ```\n $env:JWT_SECRET=\"mysecret\"\n $env:JWT_ISS=\"mycompany\"\n ```\n\n:::info To reflect the updated values of the .env variables, you need to export them again after making changes. This ensures that the updated values are accessible and used in your application.\n:::\n\n\n### Step 3: Access Environment Variables in Your Project\n\nOnce environment variables are set up, they can be accessed in Godspeed Project as:\n\n- **In Yaml Files**: Use the templating syntax `<% config.variable_path %>` to access the variable.\n \n **Example**:\n ```yaml\n datasource:\n base_url: <% config.my_datasource.base_url %>\n ```\n- For cases where the variable needs to be embedded as part of a string, you can use JavaScript template string syntax within the Godspeed configuration, as shown below:\n \n ```yaml\n headers: \n Content-Type: application/json\n Authorization: <% `Bearer ${config.buildPiper.accessToken}` %> # to use Bearer + token \n ```\n \n- **In TypeScript/JavaScript files**:\n Use `ctx.config` to access values within your TypeScript workflows.\n\n **Example TypeScript Workflow**:\n ```typescript\n import { GSContext, GSStatus } from \"@godspeedsystems/core\";\n\n export default async function myWorkflow(ctx: GSContext, args: any) {\n const apiKey = ctx.config.my_datasource.api_key;\n const apiUrl = ctx.config.my_datasource.base_url;\n\n if (!apiKey || !apiUrl) {\n throw new GSStatus(false, 500, 'Failed', \"Missing configuration variables\");\n }\n\n // Use apiKey and apiUrl in your logic\n console.log(\"API Key:\", apiKey);\n console.log(\"API URL:\", apiUrl);\n\n return new GSStatus(true, 200, 'OK', { message: \"Environment variables accessed successfully\" });\n }\n ```\n This setup provides a secure, centralized way to manage sensitive configuration data, making it easy to change values without modifying the source code.\n\n\n## Static variables\nThe static variables as well as their values are defined in yaml files under `config/` directory. These variables can be replaced as per the business use cases. The default directory structure is given as below:\n\n```\n├── config\n│   ├── default.yaml\n```\n\n:::note\nAny configuration which includes secrets or passwords is recommended to be defined using environment variables only. Avoid using static variables for secrets and passwords.\n:::\n\n### default.yaml\nThis file contains some predefined variables. Below is a sample file which defines the static variables used in Godspeed.\n```yaml\nlog:\n redact: # ['a.b.c', 'a.b.*', 'req.headers', 'mobileNumber'] #pino redact rules. Default null.\n level: debug #by default info\n sync: true #By default sync is false. For debugging, keep it true. For performance keep it false.\n timestamp: stdTimeFunctions.isoTime #Pino date formats\n bindings: # should pid and hostname be enabled in pino log bindings.\n pid: false\n hostname: true \ndefaults:\n lang: coffee #default language of inline scripting e.g. js/coffee\n on_error:\n continue: false # default behavior of on_error.continue property\nserver_url: https://api.example.com:8443/v1/api\nhttpbin: # sample api datasource url\n base_url: https://httpbin.org\n```\n\nHere, `defaults` key has the variables which are used by the framework for default values in workflow DSL. \n**1. lang** - By default, the framework sets it to `js`. Check [default language](../inline-scripting/overview.md/#default-language-at-global-level) for more information. \n**2. on_error.continue** - By default, the framework sets it to `false`.\n" }, "docs/microservices-framework/config-and-mappings/env.md": { "content": "---\ntitle: Environment Variables\ndescription: Learn how to set up and use environment variables in a Godspeed project using the .env file.\nkeywords: [environment variables, .env file, security, configuration, API keys, secrets]\n---\n# Environment Variables\n\nIn a Godspeed project, the .env file is a standard way to define environment-specific variables. \nUsing this file offers several advantages:\n\n- **Security:** Sensitive information (like API keys and secrets) is stored outside of the codebase, helping prevent accidental exposure.\n- **Environment-Based Configuration:** You can specify different values for variables based on your environment, allowing seamless transitions between development, testing, and production.\n- **Easy Access Across the Project:** The variables defined in .env can be accessed easily in Godspeed configurations, workflows, and scripts.\n\n.env file is placed in the root directory of your project.\n```\n|\n├── .env\n```\n\n> **Note**: For sensitive information such as secrets or passwords, it is recommended to use environment variables in this way to avoid hardcoding values directly in your configuration files.\n\n\n## How to Set Up and Use the .env File\n\n**Define Variables in the .env File:**\n Add key-value pairs for each environment variable. Here’s an example of what you might define in a Godspeed project:\n\n`.env`\n```\n # API configurations\n\n API_URL=https://api.example.com\n API_KEY=your_api_key_here\n\n # Database configurations\n DATABASE_URL=postgres://user:password@localhost:5432/mydatabase\n\n # JWT Authentication configurations\n JWT_SECRET=my_jwt_secret\n JWT_ISSUER=https://my-issuer.com\n JWT_AUDIENCE=https://my-audience.com\n\n # Third-party service credentials\n GOOGLE_CLIENT_ID=your_google_client_id\n GOOGLE_CLIENT_SECRET=your_google_client_secret\n GOOGLE_CLIENT_SECRET=your_callback_url\n```\n\n\n ### To access environment variables directly:\n\n You can access env variables directly using **process.env** in your code if: \n\n - Your application is small or you have **simple configuration needs**.\n - You want to keep things lightweight without adding external dependencies.\n - You’re deploying on a platform that easily handles environment variables, such as **cloud services** (AWS, Heroku) or **Docker containers**.\n - Your configuration is **mostly static** and doesn’t require complex overrides or defaults.\n \n\n **Syntax to refer to these variables in YAML** \n ```\n <% process.env.VARIABLE_NAME %> //with scripting\n ```\n **Syntax to refer to these variables in typescript**\n \n ```\n process.env.VARIABLE_NAME //without scripting\n\n ``` " }, "docs/microservices-framework/config-and-mappings/mappings.md": { "content": "---\ntitle: Mappings\ndescription: Learn about mappings, a global object available in your microservice for defining key/value pairs, arrays, etc.\nkeywords: [mappings, global object, microservice, key/value pair, YAML]\nsidebar_position: 3\n---\n\n# Mappings\n\nMappings is a global object which will be available in your microservice. You can define anything in the mappings i.e. key/value pair map, array, etc. You can access these mappings inside your workflows at any time.\n\n## Project structure\nMappings are present in `src/mappings` directory. The default format is yaml and you can store mappings in the nested directories also. The nested directories are also accessible in the same `mappings` object.\n```\n.\n├── config\n└── src\n └── mappings\n └── index.yaml\n └── generate.yaml\n```\n\n## Sample mappings\nThis is a sample mapping which is accessible in the workflows inside mappings object using `mappings.Gender` and `mappings.generate.genId` \n```yaml title=\"index.yaml\"\nGender:\n Male: M\n Female: F\n Others: O\n```\n\n```yaml title=\"generate.yaml\"\ngenId: 12345\n```\n:::tip Note\nIf the file name is index.yaml then its content is available directly at global level i.e. you don't need to write index explicitly while accessing the mappings object like `mappings.Gender`. \nHowever, for other file names you need to mention the file name while accessing the mappings object like `mappings.generate.genId`\n:::\n\n### Sample Datasource Configuration using Mappings\n ```yaml\n # /src/datasources/aws.yaml\n type: aws\n default_client_config:\n region: <% mappings.aws_region %>\n services:\n s3:\n type: s3\n config:\n bucket: <% mappings.aws_s3_bucket %>\n dynamodb:\n type: dynamodb\n config:\n tableName: <% mappings.aws_table_name %>\n ```\n **Explanation**:\n - `aws_region`, `aws_s3_bucket`, and `aws_table_name` are defined in `mappings`.\n - This configuration dynamically injects these values into the datasource.\n\n---\n\n### Sample Event Source Configuration using Mappings\n ```yaml\n # /src/eventsources/cron.yaml\n type: cron\n events:\n cron.0 0 * * *.UTC:\n description: Daily task at midnight UTC\n fn: daily_update\n args:\n reportPath: <% mappings.report_path %>\n ```\n **Explanation**:\n - The `report_path` mapping is used to dynamically define the report path for the workflow triggered by the event.\n\n---\n### Sample Yaml Workflow accessing mappings\n```\n - id: httpbinCof_step1\n description: Hit http bin with some dummy data. It will send back same as response\n fn: datasource.api.post./anything\n args:\n params:\n data:\n personal_email_id: 'ala.eforwich@email.com'\n gender: <% mappings.Gender[inputs.body.Gender] %>\n id: <% mappings.generate.genId %>\n```\n### Sample typescript Workflow accessing mappings\n\nIn this example, mappings is utilized to dynamically retrieve error messages based on an error code.\n\n```\nimport { GSContext, GSStatus } from '@godspeedsystems/core';\n\nexport default async function(ctx: GSContext, args: any) {\n const { mappings, childLogger, inputs } = ctx;\n let error: any = {};\n error.code = args?.code; \n error.message = args?.message;\n \n if (!error.code) {\n error.code = \"E001\";\n } \n if (!error.message) {\n error.message = mappings.error_codes.codes[error.code];\n }\n return new GSStatus(false, 400, 'Failed', error.message);\n }\n\n```\n### Step-by-Step Explanation of above ts workflow\n\n- The mappings object is destructured from the ctx (Godspeed context).\n- In this case, mappings.error_codes.codes contains a predefined set of error codes and their corresponding messages.\n- The function checks if the args object contains an error code and a message, if the code is missing, it defaults to \"E001\".\n- If the message is not provided in the args, the function retrieves it from `mappings.error_codes.codes` using the error code as the key. \n\n### Mappings file containing error codes\n`src/mappings/error_codes.yaml`\n```yaml\n codes:\n E001: Internal Server Error\n E002: Missing mandatory field in the request body.\n E003: Extra properties found in the request body.\n E004: Wrong field type/format/pattern/length in the request body.\n E005: Record already exists in the database.\n E006: Record to update is not present in the database.\n E007: Record to delete is not present in the database.\n```\nThis ensures that error messages are consistent and centralized. Instead of hardcoding messages in multiple places, they are stored in the mappings file.\n\n\n## Using mapping constants in other mapping files\nYou can use mapping constants in other mapping files using coffee/js scripting.\n\nFor example, you have mapping files `index.yaml`, `relations.json` and `reference.yaml`. Use the mappings from first two files as reference in the third file as follows: \n \n```yaml title=\"index.yaml\"\nGender:\n Male: M\n Female: F\n Others: O\n```\n\n```json title=\"relations.json\"\n{\n \"id\": 1,\n \"title\": \"Hello World\",\n \"completed\": false,\n \"gender\": \"<%mappings.Gender.Male%>\"\n}\n```\n\n```yaml title=\"reference.yaml\"\nNewGender: <% mappings.Gender.Others %>\ntitle: <% mappings.relations.title %>\n```" }, "docs/microservices-framework/databases/CokroachDB.md": { "content": "---\ntitle: CokroachDB\ndescription: Learn how to use CoackrochDB with Prisma plugin.\nkeywords: [CoackrochDB, Prisma, database, connection URL, schema]\n---\nCoackroch, a distributed database that is designed for scalability and high availability, is largely compatible with PostgreSQL, and can be used with Prisma plugin in the same way.\n\n### Pre-requisites\nIn order to use Coackroch database you need:\n 1.\tan existing godspeed project with “prisma-as-datasource plugin” installed\n 2.\ta CoackrochDB database server running\n 3.\tand database connection URL\n\n### CONNECTION_URL\nCockroachDB uses the PostgreSQL format for its connection URL.\n```\ncockroachDB://USER:PASSWORD@HOST:PORT/DATABASE\n```\nYou have to define the database connection url as an environment variable in .env as :\n```\nDATABASE_URL=\"cockroachDB://USER:PASSWORD@HOST:PORT/DATABASE\"\n```\nAnd then this environment variable is provided in the url field of the datasource block in your prisma schema.\nsrc/datasources/cockroachDB.prisma\n```\ndatasource db {\n provider = \"cockroachDB\"\n url = env(\"DATABASE_URL\") \n}\n\n```\n### Sample prisma schema for cockroachDB \n
\n Sample prisma schema for cockroachDB \n\n```\ndatasource db {\n provider = \"cockroachDB\"\n url = env(\"DATABASE_URL\")\n}\ngenerator client {\n provider = \"prisma-client-js\"\n output = \"./prisma-clients/cockroachDB\"\n previewFeatures = [\"metrics\"]\n}\n\nmodel User {\n id Int @id @default(autoincrement())\n email String @unique\n name String?\n posts Post[]\n}\n\nmodel Post {\n id Int @id @default(autoincrement())\n title String\n content String?\n published Boolean @default(false)\n author User @relation(fields: [authorId], references: [id])\n authorId Int\n}\n```\n
\n\n### Generate prisma client\n```bash\ngodspeed prisma prepare\n```\nThis command will generate the prisma client and will sync the database with prisma schema\n\n### Generate CRUD APIs\nThen You can generate the CRUD API'S by entering the below command:\n```bash\ngodspeed gen-crud-api\n```\n* This command will generate the crud apis based on the sample prisma schema provided at ./src/datasources/cockroachDB.prisma\n\n* Now you can view the event and workflows according to the defined prisma schema" }, "docs/microservices-framework/databases/MariaDB.md": { "content": "---\ntitle: MariaDB\ndescription: Learn how to use MariaDB with Prisma plugin.\nkeywords: [MariaDB, Prisma, database, connection URL, schema]\n---\nMariaDB is a fork of MySQL, created to maintain MySQL’s features while ensuring open-source development. It can be connected using the same way as MySql with Prisma plugin.\n\n### Pre-requisites\nIn order to use MariaDB database you need:\n 1.\tan existing godspeed project with “prisma-as-datastore plugin” installed\n 2.\ta MariaDB database server running\n 3.\tand database connection URL\n\n### CONNECTION_URL\n```\nmariadb://USER:PASSWORD@HOST:PORT/DATABASE\n```\n### Setting Environment Variable\nYou have to define the database connection url as an environment variable in .env as :\n```\nDATABASE_URL=\"mariadb://USER:PASSWORD@HOST:PORT/DATABASE\"\n```\nAnd then this environment variable is provided in the url field of the datasource block in your prisma schema.\nsrc/datasources/mariaDB.prisma\n```\ndatasource db {\n provider = \"mariaDB\"\n url = env(\"DATABASE_URL\") \n}\n```\n
\n Sample prisma schema for mariaDB database \n\n```\ndatasource db {\n provider = \"mariaDB\"\n url = env(\"DATABASE_URL\")\n}\ngenerator client {\n provider = \"prisma-client-js\"\n output = \"./prisma-clients/mariaDB\"\n previewFeatures = [\"metrics\"]\n}\n\nmodel User {\n id Int @id @default(autoincrement())\n email String @unique\n name String?\n posts Post[]\n}\n\nmodel Post {\n id Int @id @default(autoincrement())\n title String\n content String?\n published Boolean @default(false)\n author User @relation(fields: [authorId], references: [id])\n authorId Int\n}\n```\n
\n\n### Generate prisma client\n```bash\ngodspeed prisma prepare\n```\nThis command will generate the prisma client and will sync the database with prisma schema\n\n### Generate CRUD APIs\nThen You can generate the CRUD API'S by entering the below command:\n```bash\ngodspeed gen-crud-api\n```\n* This command will generate the crud apis based on the sample prisma schema provided at ./src/datasources/mariaDB.prisma\n\n* Now you can view the event and workflows according to the defined prisma schema\n" }, "docs/microservices-framework/databases/MongoDB.md": { "content": "---\ntitle: MongoDB Integration with Godspeed Framework\ndescription: A comprehensive guide to integrating MongoDB in Godspeed applications using both Mongoose and Prisma plugins. Features include data modeling, schema management, CRUD operations, and seamless database interactions through multiple integration approaches.\nkeywords: [mongodb, nosql database, mongoose, prisma, database integration, data modeling, crud operations with mongodb, godspeed mongodb plugin, database connectivity, orm, odm]\n---\n\n# MongoDB Integration\n\nMongoDB is a popular NoSQL database. In Godspeed, you can connect to MongoDB in two ways:\n#### 1. [Using Mongoose-as-datasource Plugin](#mongoose-as-datasource-plugin) [(Plugin Repository)](https://github.com/godspeedsystems/gs-plugins/tree/main/plugins/mongoose-as-datastore)\n#### 2. [Using Prisma-as-datastore Plugin](#prisma-as-datastore-plugin) [(Plugin Repository)](https://github.com/godspeedsystems/gs-plugins/tree/main/plugins/prisma-as-datasource)\n\n## Mongoose as datasource Plugin\n\nThis plugin provides seamless integration with MongoDB through the Mongoose library. With this plugin, you can harness the power of Mongoose to model your data, perform queries, and interact with MongoDB in a structured and efficient manner.\n\n### How to Add \nCreate a godspeed project from the CLI, open the created project in vscode and then add the plugin:\n\n```\ngodspeed plugin add @godspeedsystems/plugins-mongoose-as-datasource \n```\n\n**-** You will find two files in your project related to the mongoose plugin at `src/datasources/types/mongoose.ts` and `src/datasources/mongoose.yaml`\n\n```typescript title=src/datasources/types/mongoose.ts\nimport { DataSource } from '@godspeedsystems/plugins-mongoose-as-datastore';\nexport default DataSource;\n```\n\n```yaml title=src/datasources/mongoose.yaml\ntype: mongoose\nsuccessResponseCodes: #default response codes for success responses\n create: 201\n find: 200\n findOne: 200\n aggregate: 200\n findOneAndUpdate: 201\n findOneAndDelete: 202\n```\n- You can keep the file by any name. This file is used to initialize a mongoose datasource instance. Whatever is the name of the file, you will need to invoke the mongoose datasource commands by the same name. Also your models will be needed to be kept in a folder with the same name as your yaml file (i.e. your datasource instance name). For example mongoose1.yaml would mean\ncalling `fn:datasources.mongoose1..` from yaml workflows and\n`ctx.datasources.mongoose1..` from TS/JS files. Also you will need to create a folder `datasources/mongoose1/models` and keep your models there as detailed below.\n\n- You can override the default response codes for success cases for different methods by putting them in the datasource instance's yaml file\n\n\n### Provide Connection URL\n\nSet an environment variable named `MONGO_URL` as your connection string to running mongodb instance.\nYou can save url in .env file as\n \n ```\n MONGO_URL='mongodb+srv://:@cluster0.xyzabc.mongodb.net/?retryWrites=true&w=majority'\n ```\n\n### Setting up Mongoose models\nThis datasource loads the [Mongoose models](https://mongoosejs.com/docs/models.html) from `datasources//models` folder.\n\n![Alt text](../../../static/img/mongoose_folder_structure.png)\n\n \nThese files are stored in `datasources//models` folder.\n\n### Writing Mongoose Models in Godspeed\n\nGodspeed framework uses **CommonJS** module syntax. When defining Mongoose models, ensure you follow the below format:\n\n**Import Mongoose:** Use the require syntax to import Mongoose:\n```ts\nconst { model, Schema, Document } = require('mongoose');\n```\n\n**Export the Model:**\n Export the model using the module.exports syntax. The export should include a type property (used for accessing the model) and the Mongoose model instance itself.\n```ts\nmodule.exports = {\n type: 'SomeModel', // The name by which you will access methods of this collection/model\n model: SomeModel // The Mongoose Model\n};\n```\n\n### An example Mongoose model file\n\n```typescript title=datasources//models/Participant.ts\n\nconst { model, Schema } =require('mongoose');\n\nconst participantSchema = new Schema({\n participant_id: { type: Number, required: true, unique: true },\n name: { type: String, required: true },\n email: { type: String, required: true },\n region: { type: String, required: true },\n city: { type: String, required: true }\n});\n\nconst Participant = model('Participant', participantSchema);\nmodule.exports = {\n type: 'Participant', // The name by which you will access methods of this collection/model\n model: Participant // The Mongoose Model\n};\n\n```\n### Sample Event and workflow\n\n\n### Event\n```yaml title=events/createParticipant.yaml\nhttp.post./participant:\n fn: createUser1\n body:\n content:\n application/json:\n schema:\n type: object\n properties:\n participant_id:\n type: number\n name:\n type: string\n description: Name\n email:\n type: string\n description: Email id\n region:\n type: string\n responses:\n 201:\n description: Participant created successfully\n content:\n application/json:\n schema:\n type: object\n properties:\n someModel:\n type: object\n description: The created Participant object\n```\n### Workflow\nWhen calling any api function it will be called as `ctx.datasources.mongoose1..` from TS/JS files.\n```ts\nimport { GSContext, GSDataSource, GSStatus } from \"@godspeedsystems/core\";\n\nexport default async function (ctx: GSContext) {\n\n const mongoClient: GSDataSource = ctx.datasources.mongoose;\n const body =ctx.inputs.data.body ;\n const data = { \n meta: {\n entityType: 'Participant', \n method: 'create'\n },\n ...body\n };\n \n const response = await mongoClient.execute(ctx, data);\n return response;\n}\n```\n\n### Error response\nWhen a call has an error the datasource returns following `GSStatus`.\n```yaml\n code: 500\n success: false\n data: \n message: Internal Server Error\n```\n\n### Run the service\n- Set an environment variable `MONGO_URL` as your connection string to running mongodb instance.\n For example, setting via a unix shell.\n \n ```shell\n export MONGO_URL='mongodb+srv://:@cluster0.xyzabc.mongodb.net/?retryWrites=true&w=majority'\n ```\n- From your command line run your service in the auto-watch mode\n ```bash\n godspeed serve\n ```\n\n## Prisma as Datastore Plugin\n\nPrisma has experimental support for MongoDB, although this support may not be as mature as for relational databases. \nMake sure you have access to a MongoDB 4.2+ server with a replica set deployment. As, MongoDB database connector uses transactions to support nested writes, which require a replica set deployment. The easiest way to deploy a replica set is with Atlas.\n\n### How to Use\n**-** Open the godspeed project in vscode and then add the plugin from the CLI of vscode, select the 'prisma-as-datastore' to integrate the plugin.\n\n```\n> godspeed plugin add \n ,_, ╔════════════════════════════════════╗\n (o,o) ║ Welcome to Godspeed ║\n ({___}) ║ World's First Meta Framework ║\n \" \" ╚════════════════════════════════════╝\n? Please select godspeed plugin to install: (Press to select, to move rows)\n┌──────┬────────────────────────────────────┬────────────────────────────────────────────────────────────────────┐\n│ │ Name │ Description │\n├──────┼────────────────────────────────────┼────────────────────────────────────────────────────────────────────┤\n│ ❯◯ │ prisma-as-datastore │ Prisma as a datastore plugin for Godspeed Framework. │\n├──────┼────────────────────────────────────┼────────────────────────────────────────────────────────────────────┤\n│ ◯ │ aws-as-datasource │ aws as datasource plugin for Godspeed Framework │\n├──────┼────────────────────────────────────┼────────────────────────────────────────────────────────────────────┤\n│ ◯ │ excel-as-datasource │ excel as datasource plugin for Godspeed Framework │\n├──────┼────────────────────────────────────┼────────────────────────────────────────────────────────────────────┤\n│ ◯ │ mailer-as-datasource │ mailer as datasource plugin for Godspeed Framework │\n├──────┼────────────────────────────────────┼────────────────────────────────────────────────────────────────────┤\n│ ◯ │ kafka-as-datasource-as-eventsource │ kafka as datasource-as-eventsource plugin for Godspeed Framework │\n└──────┴────────────────────────────────────┴────────────────────────────────────────────────────────────────────┘\n```\n\nor directly install the Prisma plugin as:\n\n```bash\ngodspeed plugin add @godspeedsystems/plugins-prisma-as-datastore\n```\n\n### Related files\nYou will find an auto genrated file in your project `src/datasources/types/prisma.ts`.\n\n```typescript title=prisma.ts\nimport { DataSource } from '@godspeedsystems/plugins-prisma-as-datastore';\nexport default DataSource;\n```\n\n### Set your connection url in .env file, See example\n\n```\nMONGO_TEST_URL=mongodb+srv://atlas_username:pswd@cluster0.w3bbqrp.mongodb.net/prisma_test?retryWrites=true&w=majority&appName=Cluster0\n```\n### Create your prisma schema\n\nNow, you can create your [prisma schema](https://www.prisma.io/docs/orm/prisma-schema) in `src/datasources` directory. \n\n### Sample prisma schema for mongo db\n\n```prisma title=src/datasources/mongo.prisma\ndatasource db {\n provider = \"mongodb\"\n url = env(\"MONGO_TEST_URL\") //Connection string added in the .env file\n}\n\ngenerator client {\n provider = \"prisma-client-js\"\n output = \"./prisma-clients/mongo\" // mongo is the name of prisma schema file\n}\n\nmodel User {\n id String @id @default(auto()) @map(\"_id\") @db.ObjectId\n createdAt DateTime @default(now())\n email String @unique\n name String?\n role Role @default(USER)\n posts Post[]\n}\n\nmodel Post {\n id String @id @default(auto()) @map(\"_id\") @db.ObjectId\n createdAt DateTime @default(now())\n updatedAt DateTime @updatedAt\n published Boolean @default(false)\n title String\n author User? @relation(fields: [authorId], references: [id])\n authorId String @db.ObjectId\n}\nenum Role {\n USER\n ADMIN\n}\n```\n\n\n### Generate prisma client\nThis command will generate the prisma client and will sync the database with prisma schema\n```bash\ngodspeed prisma prepare\n```\nOnce you [generate prisma client](#generate-prisma-client), the multiple clients get generated in `src/datasources/prisma-clients` directory. Godspeed automatically loads all the clients present in this directory.\n\n### Generate CRUD APIs\nYou can generate the CRUD API'S enter the below command:\n```bash\ngodspeed gen-crud-api\n```\n* This command will generate the crud apis based on the sample prisma schema provided at ./src/datasources/mongo.prisma\n\n* Now you can view the event and workflows according defined prisma schema\n\n### Sample API\nHere is a sample event and workflow for mongodb, which is fetching data from the database.\n\n```yaml title=src/events/getPost.yaml\n\nhttp.get./mongo/post/{id}:\n summary: Fetch Post\n description: Fetch Post from database\n fn: com.biz.mongo.post.one\n params:\n - name: id\n in: path\n required: true\n schema:\n type: string\n responses:\n content:\n application/json:\n schema:\n type: object\n```\n```ts title = src/functions/com/biz/mongo/post/one.ts\n\nimport { GSContext, GSStatus } from \"@godspeedsystems/core\";\n\nmodule.exports = async (ctx: GSContext) => {\n const { inputs: { data: { params } }, logger, datasources } = ctx;\n\n const response = await datasources.mongo.Post.findUnique({\n where: { id: params.id }\n });\n return new GSStatus(true, 200, \"Post fetched\", response );\n}\n```\n\n \n## Reference links\n**-** [Prisma Plugin Repository](https://github.com/godspeedsystems/gs-plugins/tree/main/plugins/prisma-as-datastore) \n**-** [Mongoose Plugin Repository](https://github.com/godspeedsystems/gs-plugins/tree/main/plugins/mongoose-as-datasource) \n**-** [npm package for Prisma](https://www.npmjs.com/package/@godspeedsystems/plugins-prisma-as-datastore)\n" }, "docs/microservices-framework/databases/MySQL.md": { "content": "---\ntitle: MySQL\ndescription: Learn how to use MySQL with Prisma plugin.\nkeywords: [MySQL, Prisma, database, connection URL, schema]\n---\nMySQL, a widely used open-source relational database management system can be connected easily using the Prisma-as-datasource plugin.\n\n### Pre-requisites\n\nIn order to use MySql database you need:\n 1.\tan existing godspeed project with “prisma-as-datasource plugin” installed\n 2.\ta MySQL database server running\n 3.\tand database connection URL\n\n### CONNECTION URL\nThe format of the connection URL for MySQL looks as follows \n```\nmysql://USER:PASSWORD@HOST:PORT/DATABASE\n```\nFor example,\nIf your MySQL database is hosted on AWS RDS, the connection URL might look like this:\n```\n\"mysql://johndoe:password@mysql–instance1.123456789012.us-east-1.rds.amazonaws.com:3306/mydb\"\n```\n\nIf you are running MySQL locally, your connection URL will be like this:\n```\n\"mysql://root:password@localhost:3306/yourdb\"\n```\n\n### Setting Environment Variable\nYou can define the database connection url as an environment variable in .env as :\n```\nDATABASE_URL=\"mysql://root:password@localhost:3306/yourdb\"\n```\nAnd then this environment variable is provided in the url field of the datasource block in your prisma schema.\nsrc/datasources/mysql.prisma\n\n```\ndatasource db {\n provider = \"MySQL\" // database provider name which you are using\n url = env(\"DATABASE_URL\") // DATABASE_URL is the name of env variable\n}\ngenerator client {\n provider = \"prisma-client-js\"\n output = \"./prisma-clients/mysql\" // dbName should be same as prisma schema file name\n previewFeatures = [\"metrics\"]\n}\n```\n\n### Sample prisma schema for MySQL\n\n```\ndatasource db {\n provider = \"mysql\"\n url = env(\"DATABASE_URL\")\n}\ngenerator client {\n provider = \"prisma-client-js\"\n output = \"./prisma-clients/mysql\"\n previewFeatures = [\"metrics\"]\n}\n\nmodel User {\n id Int @id @default(autoincrement())\n email String @unique\n name String?\n posts Post[]\n}\n\nmodel Post {\n id Int @id @default(autoincrement())\n title String\n content String?\n published Boolean @default(false)\n author User @relation(fields: [authorId], references: [id])\n authorId Int\n}\n```\n\n### Generate prisma client\n```bash\ngodspeed prisma prepare\n```\nThis command will generate the prisma client and will sync the database with prisma schema\n\n### Generate CRUD APIs\nThen You can generate the CRUD API'S by entering the below command:\n```bash\ngodspeed gen-crud-api\n```\n* This command will generate the crud apis based on the sample prisma schema provided at ./src/datasources/mysql.prisma\n\n* Now you can view the event and workflows according to the defined prisma schema" }, "docs/microservices-framework/databases/Overview.md": { "content": "---\ntitle: Database Support Overview\ndescription: A comprehensive guide to database integration in Godspeed using schema-driven development. Features support for multiple SQL and NoSQL databases through Prisma, automatic CRUD generation, and built-in authorization policies.\nkeywords: [database integration, prisma orm, schema driven development, postgresql, mysql, mongodb, sqlite, sql server, cockroachdb, mariadb, planetscale, crud operations, database schema, database authorization]\n---\n\n# Database Support Overview\n\nThe framework takes the approach of schema driven development. It supports multiple kinds of SQL and NoSQL databases. The developer only needs to specify or generate the schema for a database, with authorization policies. The CRUD events and workflows are automatically generated from the schema itself.\n\nThe framework provides functionality to access almost all popular databases through Prisma-as-datastore plugin [Plugin Repository](https://github.com/godspeedsystems/gs-plugins/tree/main/plugins/prisma-as-datastore) which is a powerful tool for efficient and secure database interactions in web applications. \n \n### List of currently supported Databases \n\n**1. [PostgreSQL](PostgreSQL)**: Prisma has strong support for PostgreSQL, one of the most popular open-source relational database systems.\n\n**2. [MySQL](MySQL)**: Prisma can be used with MySQL, another widely used open-source relational database management system.\n\n**3. [SQLite](https://www.prisma.io/docs/orm/overview/databases/sqlite)**: SQLite is a serverless, self-contained, and zero-configuration database engine, and Prisma supports it as well.\n\n**4. [SQL Server](SQLServer)**: Prisma offers support for Microsoft SQL Server, a popular commercial relational database management system.\n\n**5. [MongoDB (Experimental)](MongoDB)**: Prisma also has experimental support for MongoDB, a NoSQL database, although this support may not be as mature as for relational databases.\n\n**6. [CockroachDB](CockroachDB)**: A distributed, resilient SQL database for large-scale, cloud-native applications.\n\n**7. [MariaDB](MariaDB)**: An open-source, high-performance relational database system and MySQL-compatible alternative.\n\n**8. [PlanetScale](PlanetScale)**: PlanetScale is a database-as-a-service platform designed for distributed SQL databases. It provides a managed, scalable, and highly available database solution for modern, cloud-native applications.\n\n## How it works\nYou can integrate any of the above databases into your project by following a few simple steps:\n\n**Step 1: Install the Prisma-as-datastore-plugin**, See [How to install](../datasources/datasource-plugins/Prisma%20Datasource.md#how-to-add-plugin)\n\n**Step 2: Set your Database_Connection_URL** \nTo connect with the database, give your db_connection_url in .env file. You can check the format and example of Connection_url for all the supported databases in the [next section](MySQL#connection-url)\n\n**Step 3: Schema specification**\nThe framework extends Prisma specification to define the schema for any database. You can create your prisma schema in the src/datasources folder as a file `{database_name}`.prisma.\n\n**Check [Sample Prisma Schema](../datasources/datasource-plugins/Prisma%20Datasource.md#sample-prisma-schema)**\n\n**Step 4: Prisma Database and Models Setup**\nThe framework has inbuilt feature of setting up database automatically whenever a new schema `{database_name}`.prisma file is created in the src/datasources folder. Just type this command in CLI\n```\n 'godspeed prisma prepare'\n```\n### Generate CRUD API\nNow you can generate CRUD API using following command\n```\n 'godspeed gen crud api'\n```\n\n#### To see advanced functionalities related to databases, please visit these sections\n\n#### [Database Encryption](../datasources/datasource-plugins/Prisma%20Datasource.md#database-encryption)\n\n\n\n\n## Reference links\n**-** [Plugin Repository](https://github.com/godspeedsystems/gs-plugins/tree/main/plugins/prisma-as-datastore) \n " }, "docs/microservices-framework/databases/PlanetScale.md": { "content": "---\ntitle: PlanetScale\ndescription: Learn how to use PlanetScale with Prisma plugin.\nkeywords: [PlanetScale, Prisma, database, connection URL, schema]\n---\n\nPlanetScale is also a relational database management system which can be connected using the Prisma-as-datasource plugin.\n\n### Pre-requisites\n\nIn order to use PlanetScale database you need:\n 1.\tan existing godspeed project with “prisma-as-datasource plugin” installed\n 2.\ta PlanetScale database server running\n 3.\tand database connection URL\n\n### CONNECTION_URL\nThe format of the connection URL for Planet Scale looks as follows \n```\nplanetscale://USER:PASSWORD@HOST:PORT/DATABASE\n```\n\n### Setting Environment Variable\nYou can define the database connection url as an environment variable in .env as :\n```\nDATABASE_URL=\"planetscale://root:password@localhost:3306/yourdb\"\n```\nAnd then this environment variable is provided in the url field of the datasource block in your prisma schema.\nsrc/datasources/planetscale.prisma\n```\ndatasource db {\n provider = \"planetscale\"\n url = env(\"DATABASE_URL\") \n}\n```\n\n### Sample prisma schema for PlanetScale \n\n```\ndatasource db {\n provider = \"planetscale\"\n url = env(\"DATABASE_URL\")\n}\ngenerator client {\n provider = \"prisma-client-js\"\n output = \"./prisma-clients/planetscale\"\n previewFeatures = [\"metrics\"]\n}\n\nmodel User {\n id Int @id @default(autoincrement())\n email String @unique\n name String?\n posts Post[]\n}\n\nmodel Post {\n id Int @id @default(autoincrement())\n title String\n content String?\n published Boolean @default(false)\n author User @relation(fields: [authorId], references: [id])\n authorId Int\n}\n```\n\n### Generate prisma client\n```bash\ngodspeed prisma prepare\n```\nThis command will generate the prisma client and will sync the database with prisma schema\n\n### Generate CRUD APIs\nThen You can generate the CRUD API'S by entering the below command:\n```bash\ngodspeed gen-crud-api\n```\n* This command will generate the crud apis based on the sample prisma schema provided at ./src/datasources/planetscale.prisma\n\n* Now you can view the event and workflows according to the defined prisma schema\n" }, "docs/microservices-framework/databases/PostgreSQL.md": { "content": "---\ntitle: PostgreSQL\ndescription: Learn how to use PostgreSQL with Prisma plugin.\nkeywords: [PostgreSQL, Prisma, database, connection URL, schema]\n---\nPostgreSQL can also be connected using the Prisma-as-datastore plugin.\n\n### Pre-requisites\nIn order to use PostgreSQL database you need:\n 1.\tan existing godspeed project with “prisma-as-datastore plugin” installed\n 2.\ta PostgreSQL database server running\n 3.\tand database connection URL\n\n### CONNECTION_URL\nThe format of the connection URL for PostgreSQL looks as follows \n```\npostgresql://USER:PASSWORD@HOST:PORT/DATABASE?schema=SCHEMA\n```\nFor example,\nIf your PostgreSQL database is hosted on Heroku, the connection URL might look like this:\n```\nDATABASE_URL=\"postgresql://opnmyfngbknppm:XXX@ec2-46-137-91-216.eu-west-1.compute.amazonaws.com:5432/d50rgmkqi2ipus?schema=hello-prisma\"\n```\n\nIf you are running PostgreSQL locally, your connection URL will be like this:\n```\nDATABASE_URL=\"postgresql://johndoe:password@localhost:5432/mydb?schema=public\"\n```\n### Setting Environment Variable\nYou can define the database connection url as an environment variable in .env as :\n```\nDATABASE_URL=\"postgresql://johndoe:password@localhost:5432/mydb?schema=public\"\n```\nIn the above examples, DATABASE_URL is just a variable name given to connection url, this same environment variable name will be then provided in the url field of the datasource block in your prisma schema src/datasources/fileName.prisma as:\n\n```\ndatasource db {\n provider = \"PostgreSQL\"\n url = env(\"DATABASE_URL\")\n}\ngenerator client {\n provider = \"prisma-client-js\"\n output = \"./prisma-clients/Postgres\"\n previewFeatures = [\"metrics\"]\n}\n\n```\n### Sample prisma schema for PostgreSQl\n\n```src/datasources/Postgres.prisma\n\ndatasource db {\n provider = \"PostgreSQL\"\n url = env(\"DATABASE_URL\")\n}\ngenerator client {\n provider = \"prisma-client-js\"\n output = \"./prisma-clients/Postgres\"\n previewFeatures = [\"metrics\"]\n}\n\nmodel User {\n id Int @id @default(autoincrement())\n email String @unique\n name String?\n posts Post[]\n}\n\nmodel Post {\n id Int @id @default(autoincrement())\n title String\n content String?\n published Boolean @default(false)\n author User @relation(fields: [authorId], references: [id])\n authorId Int\n}\n```\n\n### Generate prisma client\n```bash\ngodspeed prisma prepare\n```\nThis command will generate the prisma client and will sync the database with prisma schema\n\n### Generate CRUD APIs\nThen You can generate the CRUD API'S by entering the below command:\n```bash\ngodspeed gen-crud-api\n```\n* This command will generate the crud apis based on the sample prisma schema provided at ./src/datasources/Postgres.prisma\n\n* Now you can view the event and workflows according to the defined prisma schema" }, "docs/microservices-framework/databases/SQLServer.md": { "content": "---\ntitle: SQLServer\ndescription: Learn how to use SQLServer with Prisma plugin.\nkeywords: [SQLServer, Prisma, database, connection URL, schema]\n---\nMicrosoft SQL Server database can be connected using Prisma-as-datasource plugin.\n\n### Pre-requisites\nIn order to use Microsoft SQL Server database, you need:\n 1.\tan existing godspeed project with “prisma-as-datasource plugin” installed\n 2.\ta Microsoft SQL Server database\n - Microsoft SQL Server on Linux for Docker\n - Microsoft SQL Server on Windows (local)\n 3.\tand database connection URL\n\n### CONNECTION_URL\nThe format of the connection URL for MySQL looks as follows \n```\nsqlserver://HOST[:PORT];database=DATABASE;user=USER;password=PASSWORD;encrypt=true\n```\nTo know more about the connection_url format, you can check [details here.](https://www.prisma.io/docs/orm/overview/databases/sql-server) \n\n### Setting Environment Variable\n```\nThen define the database connection url as an environment variable in .env as :\n```\nDATABASE_URL=\"sqlserver://HOST[:PORT];database=DATABASE;user=USER;password=PASSWORD;encrypt=true\"\n```\nAnd then this environment variable is provided in the url field of the datasource block in your prisma schema.\n\n```\nsrc/datasources/sqlserver.prisma\n```\ndatasource db {\n provider = \"sqlserver\"\n url = env(\"DATABASE_URL\") \n}\n```\n### Sample prisma schema for Sql Server\n\n```\ndatasource db {\n provider = \"sqlserver\"\n url = env(\"DATABASE_URL\")\n}\ngenerator client {\n provider = \"prisma-client-js\"\n output = \"./prisma-clients/sqlserver\"\n previewFeatures = [\"metrics\"]\n}\n\nmodel User {\n id Int @id @default(autoincrement())\n email String @unique\n name String?\n posts Post[]\n}\n\nmodel Post {\n id Int @id @default(autoincrement())\n title String\n content String?\n published Boolean @default(false)\n author User @relation(fields: [authorId], references: [id])\n authorId Int\n}\n```\n\n\n### Generate prisma client\n\n```bash\ngodspeed prisma prepare\n```\nThis command will generate the prisma client and will sync the database with prisma schema\n\n### Generate CRUD APIs\nThen You can generate the CRUD API'S by entering the below command:\n```bash\ngodspeed gen-crud-api\n```\n* This command will generate the crud apis based on the sample prisma schema provided at ./src/datasources/mysql.prisma\n\n* Now you can view the event and workflows according to the defined prisma schema" }, "docs/microservices-framework/databases/SQLite.md": { "content": "---\ntitle: SQLite\ndescription: Learn how to use SQLite with Prisma plugin.\nkeywords: [SQLite, Prisma, database, connection URL, schema]\n---\nSQLite is a lightweight, serverless database engine that operates in-memory by default. This means it doesn't require a separate database server process or a network connection to access data.\nWhen you use SQLite with Prisma ORM, the database is created and managed entirely within your application's memory space.\n\n### Pre-requisites\n\nA godspeed project with “prisma-as-datastore plugin” installed\n\nSee [How to set up SQLite on your computer](https://www.prisma.io/dataguide/sqlite/setting-up-a-local-sqlite-database#setting-up-sqlite-on-windows) \n\n### Connecting to SQLite in Godspeed with Prisma\n\nThe SQLite data source connector connects Prisma ORM to a SQLite database file (which always has a .db extension).\n\n#### Connection URL\nThe connection URL for SqLite will point to the location of the database file.\n\nFormat:\n```\nfile: followed by the path to the file`\n```\nExample:\n```\nfile:./testing.db\n```\n\nIf the database file doesn't exist at the specified path, it will automatically be created in datasource directory of your godspeed project.\n\n### Configure the datasource block in your Prisma schema\n\nsqlite.prisma\n```\ndatasource db {\n provider= \"sqlite\"\n url = \"file:./testing.db\" # target files from the project root\n or\n url = \"file:/Users/john/testing.db\" # target files from any other place in your file system\n}\n```\n\n### Setting Environment Variables\n\nSQLite operates in-memory and Prisma handles the connection automatically. For enhanced configuration management, you can store the database file path in an environment variable and reference it in your Prisma schema:\n\n#### In your .env file\n```\nDATABASE_URL= file:./testing.db\n```\n#### In your Prisma schema\n```\ndatasource db {\n provider = \"sqlite\"\n url = env(\"DATABASE_URL\")\n}\ngenerator client {\n provider = \"prisma-client-js\"\n output = \"./prisma-clients/sqlite\"\n previewFeatures = [\"metrics\"]\n}\n```\nThis approach allows you to easily switch between different database files at runtime if necessary.\n\n### Sample prisma schema for SqLite\n\n```\ndatasource db {\n provider = \"sqlite\"\n url = env(\"DATABASE_URL\")\n}\ngenerator client {\n provider = \"prisma-client-js\"\n output = \"./prisma-clients/sqlite\"\n previewFeatures = [\"metrics\"]\n}\n\nmodel User {\n id Int @id @default(autoincrement())\n email String @unique\n name String?\n posts Post[]\n}\n\nmodel Post {\n id Int @id @default(autoincrement())\n title String\n content String?\n published Boolean @default(false)\n author User @relation(fields: [authorId], references: [id])\n authorId Int\n}\n```\n\n### Generate prisma client\nAfter having saved your schema under datasources directory, you can run the command:\n```bash\ngodspeed prisma prepare\n```\nThis command will generate the prisma client and will sync the database with prisma schema.\n\n### Generate CRUD APIs\nThen You can generate the CRUD API's by entering the below command:\n```bash\ngodspeed gen-crud-api\n```\n* This command will generate the crud apis based on the sample prisma schema provided at ./src/datasources/sqlite.prisma\n\n* Go to the src/Events and src/Functions to vview the events and workflowss generated as per your defined prisma schema." }, "docs/microservices-framework/datasources/caching.md": { "content": "---\ntitle: Caching\ndescription: Learn about caching tasks using Redis/Mem-cache or any other custom cache.\nkeywords: [caching, Redis, Mem-cache, GSCacheAsDatasource, plugin]\n---\n# Caching\nGodspeed provides caching of the tasks using redis/mem-cache or any other custom cache. You can cache the result of any task in the workflows. \nGodspeed provides two pre-defined types of cache i.e. [redis](./datasource-plugins/Redis%20Datasource.md) and in-memory. It allows using multiple caches in a single service with a default cache. Cache datasource should implement abstract class `GSCacheAsDatasource` [code is below](#how-to-write-your-own-cache-plugin). You can make multiple caches using this abstract class.\n\n> You can read and return from cache if data is present there, before executing any task where caching instruction is set. \n> The data is stored in form of [GSStatus](../workflows/native-language-functions.md/#gsstatus) of the executed task. Both success and failure statuses can be stored and loaded.\n\n## Configuration\n### Add pre-defined cache in Godspeed\nYou can use Godspeed CLI to browse and install redis/mem-cache, maintained by Godspeed.\n```bash\ngodspeed plugin add\n```\n```bash\n\n\n ,_, ╔════════════════════════════════════╗\n (o,o) ║ Welcome to Godspeed ║\n ({___}) ║ World's First Meta Framework ║\n \" \" ╚════════════════════════════════════╝\n\n\n? Please select godspeed plugin to install: (Press to select, to move rows)\n┌──────┬────────────────────────────────────────┬────────────────────────────────────────────────────────────────────────────────┐\n│ │ Name │ Description │\n├──────┼────────────────────────────────────────┼────────────────────────────────────────────────────────────────────────────────┤\n│ ◯ │ prisma-as-datastore │ Prisma as a datasource plugin for Godspeed Framework. │\n├──────┼────────────────────────────────────────┼────────────────────────────────────────────────────────────────────────────────┤\n│ ◯ │ aws-as-datasource │ aws as datasource plugin for Godspeed Framework │\n├──────┼────────────────────────────────────────┼────────────────────────────────────────────────────────────────────────────────┤\n│ ❯◯ │ redis-as-datasource │ redis as datasource plugin for Godspeed Framework │\n├──────┼────────────────────────────────────────┼────────────────────────────────────────────────────────────────────────────────┤\n│ ◯ │ graphql-as-eventsource │ graphql as eventsource plugin for Godspeed Framework │\n├──────┼────────────────────────────────────────┼────────────────────────────────────────────────────────────────────────────────┤\n│ ◯ │ mem-cache-as-datasource │ mem-cache as datasource plugin for Godspeed Fraework Framework │\n└──────┴────────────────────────────────────────┴────────────────────────────────────────────────────────────────────────────────┘\n```\nAlternatively, you can specify `redis-as-datasource` or `mem-cache-as-datasource` to add cache in your project directly.\n\n```sh\ngodspeed plugin add \n```\n\nOnce you add the desired cache, you can find its code in `src/types/redis.ts` where you can reuse our existing plugins or have code of your own plugin. \nYou can find its configuration in the `src/datasources` e.g. `redis.yaml`. All the redis database related configuration need to be set in this file. The `redis.yaml` is an instance of `type: redis`. You can have multiple instances of `type: redis` called `redis1.yaml`, `redis2.yaml` etc. \n\n```bash\n├── api.yaml\n├── redis.yaml\n└── types\n ├── axios.ts\n └── redis.ts\n```\n\n### Default cache\nDefine default cache datasource in [static configuration](/docs/microservices-framework/config-and-mappings/config.md) in `caching` key.\n\n```yaml\nlog_level: debug\ndefaults:\n lang: coffee\nserver_url: https://api.example.com:8443/v1/api\ncaching: \n```\n\n### How to use caching in your tasks\n\n#### Using caching in Typescript/Javascript tasks\n:::note\nCurrently caching support is not provided when you call a datasource or function/workflow from typescript code. Check this [github issue](https://github.com/godspeedsystems/gs-node-service/issues/1008) for more information.\n:::\nFor now if you want to use cache datasource within typescript code, you need to call it like any other datasource from within a typescript code. You need to manage caching and invalidations in your code. This requires some boilerplate till the above issue is implemented.\n\n```javascript\nexport default async function (ctx: GSContext, args: any) {\n const redis_client = ctx.datasources['redis'].client;\n let res: GSStatus;\n const key = 'helloworld2';\n\n try {\n const cached_value = await redis_client.get(key);\n if (!cached_value) {\n res = await ctx.functions['com.gs.helloworld2'](ctx, {nice: \"name\", ...args});\n await redis_client.set(key, JSON.stringify(res));\n } else {\n return JSON.parse(cached_value);\n } \n } catch(ex) {\n return new GSStatus(false, 500, 'Failed', {message: \"Internal Server Error\", info: ex.message});\n }\n\n return res;\n}\n```\n\n\n\n### How to write your own cache plugin\nYou need to implement abstract class `GSCacheAsDatasource` interface to write your own cache plugin.\n```javascript\nexport abstract class GSCachingDataSource extends GSDataSource {\n //Redis options are available [here](https://redis.io/commands/set/) Client may or may not support all actions. RedisOptions is a superset based on what Redis supports\n public abstract set(key:string, val: any, options: RedisOptions): any; \n public abstract get(key: string): any; //Return the value stored against the key\n public abstract del(key: string): any; //Delete the key\n}\n\nexport type RedisOptions = {\n EX? : number,\n PX? : number,\n EXAT?: number,\n NX?: boolean,\n XX?: boolean,\n KEEPTTL?: boolean,\n GET?: boolean\n}\n```\n\n#### Sample caching datasource implementation\nThis code is sourced from [mem-cache plugin](https://github.com/godspeedsystems/gs-plugins/blob/main/plugins/mem-cache-as-datasource/README.md). You can use this as a reference to make or customise your own caching implementations.\n\n
\n mem-cache datasource plugin\n\n\n#### Project structure\n```bash\n.\n├── src\n│ ├── datasources\n│ │ ├── api.yaml\n│ │ ├── mem-cache.yaml\n│ │ └── types\n│ │ ├── axios.ts\n│ │ ├── mem-cache.ts\n```\n\n#### mem-cache config ( src/datasources/mem-cache.yaml )\n```yaml\ntype: mem-cache\n```\n\n#### initializing client and execution ( src/datasources/types/mem-cache.ts ):\n```ts\nimport { GSContext, GSCachingDataSource, PlainObject, logger } from \"@godspeedsystems/core\";\n\nexport default class DataSource extends GSCachingDataSource {\n protected async initClient(): Promise {\n this.client = {};\n return this.client;\n } \n\n set(key: string, val: any, options: { EX?: number | undefined; PX?: number | undefined; EXAT?: number | undefined; NX?: boolean | undefined; XX?: boolean | undefined; KEEPTTL?: boolean | undefined; GET?: boolean | undefined; }) {\n logger.debug('set key %s %o', key, this.client);\n this.client[key] = val;\n }\n\n get(key: string) {\n return this.client[key];\n }\n\n del(key: string) {\n delete this.client[key];\n }\n\n execute(ctx: GSContext): Promise {\n throw new Error(\"Method not implemented.\");\n }\n}\n```\n
\n\n\n" }, "docs/microservices-framework/datasources/create-custom-datasource.md": { "content": "---\ntitle: Create a Custom Datasource\ndescription: Learn how to create and interact with custom data sources in Godspeed.\nkeywords: [custom datasource, API, SQL, NoSQL, RDBMS, mongodb, postgresql, key value stores, document stores]\n---\n# Create a Custom Datasource\n\n**A datasource** in Godspeed is any entity that provides a read/write mechanism for data. For example, an API, a SQL or NoSQL datastore which includes RDBMS or mongodb,postgresql, key value stores, document stores etc. The congfiguration for each datasource lies in src/datasources directory.\n\n## Steps to Create a Custom Datasource in Godspeed\n\nYou can create and interact with custom data source in few steps by:\n\n**1. Defining configuration file**\n\n**2. Implementing datasource logic and**\n\n**3. Creating event and workflow to manage interactions**\n\nLets understand how to implement the above steps :\n\n### Step 1: Define Configuration file \n\n- Inside the `datasources` directory, create `YAML` files to setup your datasource integrations. For example, chatgpt.yaml or kafka.yaml. \n\n- In this YAML file, ensure you specify a `type` field, and there must be a corresponding `TypeScript` file in the `types` directory that shares the same name as the `type` you defined. \n\n:::tip\nYou can create multiple instances of same datasource type by creating multiple yaml files. For example, chatgpt1.yaml and chatgpt2.yaml. \n\nFYI you can do the same for eventsources as well, For example http1.yaml and http2.yaml where both are of `type: express` and run on different ports.\n:::\n\nLet's see an example of creating chatgpt as custom datasource.\n\n```\n .\n ├── src\n ├── datasources\n │ ├── types\n │ | └── chatgpt.ts\n | |\n │ └── chatgpt.yaml\n │\n ├── events\n |\n ├── eventsources\n │ \n └── functions\n```\n\n### Example\n\nChatgpt Config file- `src/datasources/chatgpt.yaml`\n\n```yaml\ntype: chatgpt # should be same as the name of your datasource typescript file in `src/datasources/types/`\n\n# custom configurations as per your datasource logic\nmodel: gpt-4o \ntemperature: 1\nmax_tokens: 200\n\n```\nThis file defines the model, temperature, and token settings for the ChatGPT API.\n\n## Step 2: Implement datasource logic\n\n1. In `src/datasources/types`, create a TypeScript file for your datasource logic (e.g., `chatgpt.ts`).\n2. Import `GSDataSource` and other required modules from `@godspeedsystems/core`.\n3. Extend the `GSDataSource` abstract class to implement custom methods to interact with your custom datasource.\n4. Initialize and return your client by implementing the abstract `initClient()` method of `GSDatasource`.\n5. Once your client is initialized, then implement the abstract `execute()` method of `GSDatasource`.\n\n### Template for your custom datasource logic :\nYou can use the following template to start writing your custom datasource logic and then modify it as per your requirement.\n\n```typescript\n import { GSContext, GSDataSource, GSStatus, PlainObject,} from \"@godspeedsystems/core\";\n\n export default class DataSource extends GSDataSource {\n protected async initClient(): Promise {\n try {\n \n // initialize your client\n\n } catch (error) {\n throw error;\n }\n }\n async execute(ctx: GSContext): Promise {\n try {\n\n // execute methods\n \n } catch (error) {\n throw error;\n }\n }\n }\n const SourceType = 'DS';\n const Type = \"y\"; // this is the loader file of the plugin, So the final loader file will be `types/${Type.js}`\n const CONFIG_FILE_NAME = \"y\"; // in case of event source, this also works as event identifier, and in case of datasource works as datasource name\n const DEFAULT_CONFIG = {};\n\n export {\n DataSource,\n SourceType,\n Type,\n CONFIG_FILE_NAME,\n DEFAULT_CONFIG\n }\n ```\n\n### Example Datasource Logic for GPT plugin \n\n`src/datasources/types/chatgpt.ts`\n\n```typescript\n\nimport { GSContext, GSDataSource, GSStatus, logger, PlainObject,} from \"@godspeedsystems/core\";\nimport OpenAI from 'openai';\n\nexport default class DataSource extends GSDataSource {\nprotected async initClient(): Promise {\n // initialize your client\n const client = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });\n return client;\n}\n\nasync execute(ctx: GSContext): Promise {\n const client = this.client as OpenAI;\n const { prompt, meta: { fnNameInWorkflow } } = args;\n // Parse method from fnNameInWorkflow\n let method = fnNameInWorkflow?.split(\".\")[2];\n\n // Validate that client and method are available\n if (!client) {\n return new GSStatus(false, 500, \"ChatGPT client is not initialized\");\n }\n if (!method) {\n return new GSStatus(false, 400, \"Method name is missing in fnNameInWorkflow\");\n }\n\n // Use destructuring with defaults to get config values\n const { model= 'gpt-4o',temperature = 1, max_tokens = 500 } = this.config;\n\n try {\n // execute methods\n if (method === \"chat\") {\n // Execute ChatGPT completion\n const response = await client.chat.completions.create({\n model,\n messages: [{ role: \"user\", content: prompt }],\n temperature,\n max_tokens,\n });\n const responseContent = response.choices[0]?.message?.content ?? \"No response generated\";\n return new GSStatus(true, 200, \"Success\", responseContent);\n } else {\n return new GSStatus(false, 400, `Invalid method: ${method}`);\n }\n } catch (error) {\n throw error;\n }\n}\n}\nconst SourceType = 'DS';\nconst Type = \"chatgpt\"; // this is the loader file of the plugin, So the final loader file will be `types/${Type.js}`\nconst CONFIG_FILE_NAME = \"chatgpt\"; // in case of event source, this also works as event identifier, and in case of datasource works as datasource name\nconst DEFAULT_CONFIG = {};\n\nexport {\n DataSource,\n SourceType,\n Type,\n CONFIG_FILE_NAME,\n DEFAULT_CONFIG\n}\n```\n\n\n## Step 3: Define an Event to trigger the Datasource\n\nIn the `src/events` directory, create an event file, e.g. `gpt.yaml`\n\n**Example (src/events/gpt.yaml)**\n\n```yaml\nhttp.post./chatgpt:\n summary: \"Generate response from ChatGPT\"\n description: \"Endpoint to send a user prompt to ChatGPT and retrieve the AI-generated response.\"\n fn: prompt\n authn: false\n body:\n content:\n application/json:\n schema:\n type: object\n properties:\n prompt:\n type: string\n description: \"The user's prompt or question for ChatGPT to respond to.\"\n required:\n - prompt\n responses:\n 200:\n description: \"Successful response from ChatGPT\"\n content:\n application/json:\n schema:\n type: string\n```\n\n## Step 4: Define the Workflow\n\nIn the `src/functions` directory, create a typescript workflow file (e.g., `prompt.ts`)\n\n```typescript\nimport { GSContext, GSDataSource, GSStatus } from \"@godspeedsystems/core\";\nexport default async function (ctx: GSContext, args: any) {\n const { inputs: {data: { body } }, }= ctx;\n const prompt = body.prompt; \n const ds: GSDataSource = ctx.datasources.chatgpt;\n \n const response = await ds.execute(ctx, {\n prompt,\n meta: {fnNameInWorkflow: 'datasource.chatgpt.chat'}\n });\n return response;\n}\n```\n\n## Step 5: Access and Test the Custom Datasource\n\n1. **Start Your Godspeed Server**:\n ```bash\n godspeed serve\n ```\n\n2. **Test the API Endpoint:**\n\n Use Swagger UI at http://localhost:3000/api-docs or a tool like Postman to test the endpoint.\n Send a POST request to /chatgpt with the prompt in the JSON body.\n\n Example request:\n\n ```json\n {\n \"prompt\": \"How to create an API in godspeed ?\"\n }\n ```\n\n3. **Verify the Response**:\nThe API should respond with the ChatGPT-generated message in the format specified.\n\n### Error response\nWhen a call has an error the datasource returns following `GSStatus`.\n```yaml\n code: 500\n success: false\n data: \n message: Internal Server Error\n```\n\n\n" }, "docs/microservices-framework/datasources/create-datasource-plugin.md": { "content": "---\ntitle: Create npm plugin for your custom datasource\ndescription: Learn how to create an npm plugin for your custom datasource.\nkeywords: [npm plugin, custom datasource, CLI, plugin repo]\n---\n# Create npm plugin for your custom datasource\n\nGodspeed has a [plugin based](https://github.com/godspeedsystems/gs-plugins.git) ecosystem and you can install the plugins using [CLI](/docs/microservices-framework/CLI.md).Now let us understand how can you contribute a plugin.\n\nIn this section, let us understand how a developer can contribute a plugin to godspeed [plugin repo](https://github.com/godspeedsystems/gs-plugins.git) or for internal use.\n\n### Steps to create new plug-in in godspeed framework:\n\n1. Begin by installing the `godspeed-plugin-generator` globally using the following commands:\n\n ```bash\n npm install -g generator-godspeed-plugin\n npm install -g yo\n ```\n\n2. To initiate the creation of your plugin, execute the following command in your terminal:\n\n ```bash\n yo godspeed-plugin\n ```\n\n3. After running the above command, you'll be prompted to enter your desired plugin name. Proceed by typing it in:\n\n ```bash\n ? Enter your plugin name: (your-plugin-name)\n ```\n\n4. Select the type of plugin that aligns with your project's requirements. You can choose from the following options:\n\n ```bash\n ? Select the type of plugin: (Use arrow keys)\n ❯ DataSource \n EventSource \n DataSource-As-EventSource \n ```\n\n ⚠️ **Note:** In certain network environments, particularly on WiFi connections, you may encounter a `Request failed with status code 403` error due to network security protocols or firewall restrictions that \n block specific API requests. To bypass this limitation, consider switching to a **mobile data hotspot**, which often has fewer restrictions. This workaround can help ensure successful command execution and \n avoid network-related request failures.\n\n5. Depending on your selection, the plugin generator will generate a template with your chosen plugin name, such as \"your-plugin-name-as-datasource.\" The structure of the generated files will be as follows:\n\n ```\n .\n ├── src\n | └── index.ts\n |\n ├── package.json\n |\n ├── README.md\n |\n ├── tsconfig.json\n |\n ├── .gitignore\n |\n └── .npmignore\n ```\n\n6. To customize your plugin, navigate to the `index.ts` file located in the `src` directory. You can modify the content within this file to meet your specific plugin requirements. There's no need to make changes to any other files outside of `index.ts`.\n\n\n**If you opt for `DataSource`, the `index.ts` file appears as follows:**\n\n ```\n\n import { GSContext, GSDataSource, GSStatus, PlainObject,} from \"@godspeedsystems/core\";\n\n export default class DataSource extends GSDataSource {\n protected async initClient(): Promise {\n try {\n // initialize your client\n } catch (error) {\n throw error;\n }\n }\n\n async execute(ctx: GSContext): Promise {\n \n try {\n // execute methods\n \n } catch (error) {\n throw error;\n }\n }\n }\n const SourceType = 'DS';\n const Type = \"y\"; // this is the loader file of the plugin, So the final loader file will be `types/${Type.js}`\n const CONFIG_FILE_NAME = \"y\"; // in case of event source, this also works as event identifier, and in case of datasource works as datasource name\n const DEFAULT_CONFIG = {};\n\n export {\n DataSource,\n SourceType,\n Type,\n CONFIG_FILE_NAME,\n DEFAULT_CONFIG\n }\n ```\n**If you opt for `DataSource-As-EventSource `, the `index.ts` file appears as follows:**\n\n ```\n import { GSContext, GSDataSource, PlainObject, GSDataSourceAsEventSource, GSCloudEvent, GSStatus, GSActor} from \"@godspeedsystems/core\";\n\n class DataSource extends GSDataSource {\n protected async initClient(): Promise {\n try {\n \n // initialize your client\n } catch (error) {\n throw error;\n }\n\n }\n\n async execute(ctx: GSContext): Promise {\n try {\n // execute methods\n \n } catch (error) {\n throw error;\n }\n }\n }\n\n class EventSource extends GSDataSourceAsEventSource {\n async subscribeToEvent(\n eventKey: string,\n eventConfig: PlainObject,\n processEvent: (\n event: GSCloudEvent,\n eventConfig: PlainObject\n ) => Promise\n ): Promise {\n\n // subscribeToEvent\n }\n }\n const SourceType = 'BOTH';\n const Type = \"shirisha\"; // this is the loader file of the plugin, So the final loader file will be `types/${Type.js}`\n const CONFIG_FILE_NAME = \"shirisha\"; // in case of event source, this also works as event identifier, and in case of datasource works as datasource name\n const DEFAULT_CONFIG = {};\n\n export {\n DataSource,\n EventSource,\n SourceType,\n Type,\n CONFIG_FILE_NAME,\n DEFAULT_CONFIG\n }\n ``` \n### Watch the videos below for better understanding!\n\n### 1. How to create plugins using godspeed?\n\nHere is a video which helps you create a plugin using the godspeed. Do watch for better understanding\n\n
\n\n
\n\n\n### 2. How to get started with a godspeed datasource plugin - installed from npm\n\n
\n \n
\n" }, "docs/microservices-framework/datasources/datasource-plugins/AWS Datasource.md": { "content": "---\ntitle: AWS Datasource Plugin\ndescription: A comprehensive AWS integration plugin that enables seamless access to Amazon Web Services in Godspeed applications. Features include S3 storage, DynamoDB, Lambda functions, SQS messaging, and SSM parameter management with configurable service clients.\nkeywords: [aws plugin, amazon web services, cloud integration, s3, dynamodb, lambda, sqs, ssm, godspeed plugin, cloud storage, serverless, cloud services]\n---\n\n**[Link to Plugin Source Code](https://github.com/godspeedsystems/gs-plugins/tree/main/plugins/aws-as-datasource)**\n\nTurbocharge your app by tapping into Amazon Web Services. Unleash the power of cloud-based data, storage, and more to supercharge your application.\n\n## Steps to use aws plug-in in godspeed framework:\n\n### How to install \nCreate a godspeed project from the CLI, open the created project in vscode and then add the plugin:\n\n```\ngodspeed plugin add @godspeedsystems/plugins-aws-as-datasource \n```\n\n### Related files\nAfter installation, you will find two auto-generated files in your project related to the plugin at `src/datasources/types/aws.ts` and `src/datasources/aws.yaml`.\n\n### Configuration\n\nConfigure this `aws.yaml` file as per your requirement :\n\n```yaml\ntype: aws\ndefault_client_config: #any aws specific configurations\n credentials:\n accessKeyId: <%config.accessKeyId%>\n secretAccessKey: <%config.secretAccessKey%>\n# service type is the name of the npm module for ex. @aws-sqk/client-dynamodb or @aws-sqk/client-s3 etc\n# The `types` key can have service type to sdk's client names mappings when coding\ntypes: #mappings\n dynamodb: DynamoDB\n s3: S3\n lambda: Lambda\n ssm: SSM\n sqs: SQS\nservices:\n s3:\n type: s3\n config:\n region: <%config.anotherAccessKeyId%>\n credentials:\n accessKeyId: <%config.anotherAccessKeyId%>\n secretAccessKey: <%config.anotherSecretAccessKey%>\n s3_1: #uses default config\n type: s3\n dynamodb:\n type: dynamodb\n sqs:\n type: sqs\n ssm:\n type: ssm\n lamdba:\n type: lambda\n```\n\n**1. ** type: aws (type of the datasource) \n**2. ** default_client_config (optional) for initializing your clients, as per the aws config specs \n**3. ** Client type to client name mappings via the `types` key \n**4. ** `services` contains settings for the services you want to invoke via this datasource. Each service has a type like s3, lamdba etc. They can have their own config overriding the default under the `config` key. \nThere can be multiple services configured for the same type. For example, `s3` and `s3_1` in the above configuration.\n\n### Example usage\n\nIn an event, we establish HTTP endpoint that accepts json objects in request body. When this endpoint is invoked, it triggers the `aws_list` function with the args coming from request body.\n\n```yaml title=event\n\"http.post./aws\":\n fn: aws_list\n body:\n type: object\n responses:\n 200:\n content:\n application/json:\n```\n\nIn workflow we need to mention `datasource.aws.${serviceName}.${method}` as function `fn` to perform operations in this case `datasource.aws.s3.listObjects`.\n\n```ts title='TS workflow'\nimport { GSContext, GSDataSource, GSStatus } from \"@godspeedsystems/core\";\n\nexport default async function (ctx: GSContext, args: any) {\n const ds: GSDataSource = ctx.datasources.aws;\n const response = await ds.execute(ctx, {\n //Pass exactly same args as this aws service's method takes\n ...args,\n //Along with args, pass meta object\n // meta can contain {entityName, method}\n meta: {entityName: 's3', method: 'listBuckets'},\n //Or meta can contain {fnNameInWorkflow} which is same as \n //the 'fn' that we write when invoking datasource from yaml workflow\n //For example, this will also work\n //meta: {fnNameInWorkflow: 'datasource.aws.s3.listBuckets'}\n });\n return response;\n}\n```\n### Example to upload file on aws s3\n```yaml title=event\n# event for upload s3 file\n\nhttp.post./aws:\n fn: aws_upload\n body:\n content:\n multipart/form-data:\n schema:\n $ref: '#/definitions/event/upload_file'\n responses:\n 200:\n content:\n application/json:\n\n```\nthe above event calling `aws_upload.ts` workflow from `src/functions/` \n \n```js \nimport { GSContext, GSStatus } from \"@godspeedsystems/core\";\nimport fs from 'fs'\n\nmodule.exports = async (ctx: GSContext) => {\n const { files: { myfile } } = ctx.inputs.data;\n const { datasources, logger } = ctx;\n try {\n return new Promise((resolve, reject) => {\n fs.readFile(myfile.tempFilePath, async function (err, data) {\n if (err) {\n resolve(new GSStatus(false, 500, 'S3 document upload failed', { error: { message: err.message } }));\n\n } // Something went wrong!\n const contentType = ctx.inputs.data.headers['content-type']\n var params = {\n Key: myfile.name, //file name\n Body: data,\n Bucket: 'userdocs123',\n ContentType: contentType,\n };\n\n const res = await datasources.aws.client.s3.putObject(params);\n\n resolve(new GSStatus(true, 200, 'successfully uploaded document', res))\n\n })\n });\n\n } catch (e: any) {\n logger.error('S3 document upload failed %o', e)\n return new GSStatus(false, 500, 'S3 document upload failed', { error: { message: e.message } })\n }\n\n};\n```\n\n## Reference links\n**-** [Plugin Repository](https://github.com/godspeedsystems/gs-plugins/tree/main/plugins/aws-as-datasource) \n**-** [Issue Tracker](https://github.com/godspeedsystems/gs-plugins/issues) \n**-** [npm package](https://www.npmjs.com/package/@godspeedsystems/plugins-aws-as-datasource)\n" }, "docs/microservices-framework/datasources/datasource-plugins/Axios Datasource.md": { "content": "---\ntitle: Axios Datasource Plugin\ndescription: A powerful HTTP client plugin that enables seamless API integration, efficient data fetching, and robust error handling in Godspeed applications. Features include authentication, retry mechanisms, and unified datasource API.\nkeywords: [axios, http client, api integration, godspeed plugin, data fetching, rest api, http requests, authentication, retry mechanism, api client]\n---\n\n**[Link to Plugin Source Code](https://github.com/godspeedsystems/gs-plugins/tree/main/plugins/axios-as-datasource)**\n\nLevel up your data-fetching game with Axios. Seamlessly integrate this powerful HTTP client into your app for smooth and efficient data transactions. The Godspeed Axios Plugin provides seamless integration with the Axios library for making HTTP requests within the Godspeed framework. It simplifies the process of defining and executing HTTP requests, making it easy to interact with external APIs.\n\n## How to Install Axios Plugin\n\nWhen you create a godspeed project from the CLI, by default the axios plugin is integrated into your project if not, add the plugin from the CLI\n\n```\ngodspeed plugin add @godspeedsystems/plugins-axios-as-datasource\n```\n\n### Related files\n\nAfter installation, you will find two auto-generated files in your project related to the plugin at `src/datasources/types/axios.ts` and `src/datasources/api.yaml`.\n\n```typescript title=src/datasources/types/axios.ts\nimport { DataSource } from '@godspeedsystems/plugins-axios-as-datasource';\nexport default DataSource;\n```\n\n### Config file for Axios\n\nThe sample config can be modified as per the usecase of your application.\n\n```yaml title=src/datasources/api.yaml\n type: axios\n base_url: https://httpbin.org\n\n # print all api calls in curl format\n curlifiedLogs: true \n\n # Authentication of API calls with token refresh logic\n authn: \n fn: my_bank.authn\n refreshOn:\n statusCode: [401]\n\n # Common headers to be set in all API calls\n headers:\n Content-Type: application/json\n Cookie: <% mappings.my_bank.auth_workflow_cookie %>\n Authorization: <% `Bearer ${config.my_bank.apiToken}` %> # to use Bearer token \n\n # Retry logic for failed API calls for ex on Internal server errors or request timeouts\n retry:\n when: #the condition\n status: [500, 503] # an array or single value of codes (optional). Default 500\n message: my custom expected message for retry #And (optionally) when response has this message\n max_attempts: 5\n type: constant # or random, exponential\n interval: PT15s\n # type: exponential\n # min_interval: PT5s\n # max_internal: PT15s\n # type: random\n # min_interval: PT5s\n # max_internal: PT15s\n```\nRetry interval values will be based on [ISO Temporal Duration standard](https://tc39.es/proposal-temporal/docs/duration.html)\n\n\n\n## How to Use:\n\n\n### Set Up an Event\n\n1. **Create an Event to Trigger the Workflow**: Define an HTTP event to trigger the workflow and call the third-party API.\n Example `fetchData.yaml`:\n\n```yaml\nhttp.get./fetch-data:\n fn: fetchDataWorkflow\n authn: false\n responses:\n 200:\n content:\n application/json:\n schema:\n type: object\n```\n ### Explanation:\n - **http.get./fetch-data:**: The first line of any http event defines the (protocol.Method./end-point)\n Here, /fetch-data is the end point which will send a get request to third party API by calling a workflow.\n - **fn**: fn defines the workflow function to be called. here it is, fetchDataWorkflow.\n\n\n### Set Up Workflow to Use the Axios Datasource\n\n Go to `src/functions/` and create a file (e.g., `fetchDataWorkflow.ts`).Use the Axios configuration to make the API call. You can specify the endpoint, HTTP method, and any parameters needed for the request.\n\n```ts\nimport { GSContext, GSDataSource, logger, PlainObject } from \"@godspeedsystems/core\";\n\nexport default async function (ctx: GSContext) {\n const client: GSDataSource = ctx.datasources.api;\n\n const res = await client.execute(ctx, {\n meta: {\n method: 'get',\n url: '/api/items',\n },\n });\n return res;\n};\n```\n\nTo get more clarity checkout about [Axios configuration]( https://axios-http.com/docs/req_config)\n\n\n## How It Helps\n\nThe Godspeed Axios Plugin offers the following advantages:\n\n**1. Axios Integration:** The plugin abstracts away the complexities of setting up Axios instances, making it effortless to configure and execute HTTP requests.\n\n**2. Unified DataSource:** Developers can use a uniform API to define data sources that make HTTP requests using Axios. This enhances consistency and ease of use across different parts of the application.\n\n**3. Error Handling:** The plugin includes robust error handling, allowing developers to gracefully handle various scenarios, such as server timeouts, request setup failures, and server-side errors.\n\n**4. Integration with Godspeed Core:** The plugin seamlessly integrates with the Godspeed Core library, aligning with the principles of the Godspeed framework and enabling streamlined event-driven workflows.\n\n\n## Plugin Components\n:::info\nYou can deep dive into the plugin code [here](https://github.com/godspeedsystems/gs-plugins/tree/main/plugins/axios-as-datasource)\n:::\nThe plugin consists of the following key components:\n\n### 1. `DataSource` Class\n\n- This class extends `GSDataSource`, a base class provided by the Godspeed framework for creating data sources.\n\n- It initializes an Axios instance to make HTTP requests based on the provided configuration options.\n\n- The `execute` method is used to define how the plugin should execute HTTP requests. It maps incoming parameters to Axios request properties, processes the request, and handles various response scenarios.\n\n### 2. Constants\n\n- `SourceType`: A constant representing the source type of the plugin, which is 'DS' (data source).\n\n- `Type`: A constant representing the loader file of the plugin. The final loader file will be located in the 'types' directory and named `${Type.js}`, where `Type` is 'axios' in this case.\n\n- `CONFIG_FILE_NAME`: In the context of a data source, this constant also serves as the data source name. In this plugin, it is set to 'api'.\n\n- `DEFAULT_CONFIG`: A default configuration object with Axios options like base URL and other settings.\n\n\n### 3. Axios retry\n\nYou can set default retry at datasource level within datasource config yaml file.(src/datasources/api.yaml)\n\n```yaml\ntype: axios\nbase_url: http://localhost:4000\nretry:\n when: #the condition\n status: [500, 501, 502] # an array or single value of codes\n message: my custom expected message for retry\n max_attempts: 5\n type: constant ##[constant,exponential,random]\n interval: PT15s\n```\nthe above config works on two conditions if status from the api is 500,501 or 502 and message value is as mentioned in the config. When condition is optional and if retry is without when condition, the retry will be made on failures of the API.\n\n\n\n### 4. Authentication of API calls with token refresh logic\n\nHTTP requests sometimes need authentication, means they are validated against a token which is passed in the headers\nand also the token needs to be refreshed before the API call. \n\nPlease checkout the above sample config of the datasource.\n\nAPI calls with token refresh logic and authentication is configured in your datasource config file, by setting `authn` and the `fn` is called before calling the API endpoint and token will be refreshed on statusCode mentioned in the array of [`statusCode`](/docs/microservices-framework/datasources/list-of-plugins#sample-config-apiyaml).\n\nExample `fn` of `authn`:\n\n```ts\nimport { logger } from \"@godspeedsystems/core\";\n\nconst axios = require('axios');\nconst client = axios.create({\n headers: {\n 'Content-Type': 'application/json'\n }\n});\n/**\n * Generate and return all the headers which are required to be sent\n * in the API calls which require authentication tokens\n */\nmodule.exports = async function (ctx: any) {\n try {\n const res = await client({\n method: 'get',\n url: `https://httpbin.org/anything`,\n data: {\n \"Authorization\": 'access_token'\n }\n })\n // Retrieve the authn tokens\n const headers = {\n \"Authorization\": res.data.access_token || 'access_token'\n };\n \n logger.info('Auth token successfully refreshed and following headers set: %o', Object.keys(headers));\n return headers;\n } catch (error) {\n logger.error('Error in refreshing token %o', error);\n throw error;\n }\n}\n```\n\n### 4.1 Skip auth\n\nIn an axios datasource call, if `skipAuth` is set in `args` then auth flow will be ignored. This is useful when generating token from the same api.\n\n\n## Conclusion\n\nThe Godspeed Axios Plugin is a valuable addition to the Godspeed framework, providing a standardized way to make HTTP requests using the Axios library. With this plugin, you can easily integrate with external APIs, handle responses, and streamline data retrieval within your applications.\n\n## Reference links\n**-** [Plugin Repository](https://github.com/godspeedsystems/gs-plugins/tree/main/plugins/axios-as-datasource) \n**-** [Issue Tracker](https://github.com/godspeedsystems/gs-plugins/issues) \n**-** [npm package](https://www.npmjs.com/package/@godspeedsystems/plugins-axios-as-datasource)\n\n" }, "docs/microservices-framework/datasources/datasource-plugins/ChatGPT datasource.md": { "content": "---\ntitle: ChatGPT Datasource Plugin\n---\n**[Link to Plugin Source Code](https://github.com/godspeedsystems/gs-plugins/tree/main/plugins/chatgpt-as-datasource)**\n\nThe ChatGPT plugin for Godspeed allows seamless integration with OpenAI’s language models, enabling dynamic AI-driven responses in your workflows and APIs.\n\n### 1. Install the Plugin\n\nFirst, add the ChatGPT plugin using\n\n```bash\ngodspeed plugin add @godspeedsystems/plugins-chatgpt-as-datasource\n```\n\n### 2. Set Up Environment Variables\n\nIn your `.env` file, add your OpenAI API key to securely access the OpenAI models:\n\n```env\nOPENAI_API_KEY=\"your_openai_api_key\"\n```\nThis key is required by the plugin to authenticate with OpenAI.\n\n\n### 3. Check the Datasource Configuration\n\nCheck the file `src/datasources/chatgpt.yaml`:\n\n```yaml\ntype: chatgpt\nmodel: gpt-4o\ntemperature: 1\nmax_tokens: 200\n```\n\n* `type`: Must be `chatgpt` to register correctly with Godspeed.\n* `model`: The model name to use (e.g., `gpt-4o`, `gpt-4`, `gpt-3.5-turbo`).\n* `temperature`: Controls the randomness of the output.\n* `max_tokens`: The maximum number of tokens to generate in the completion.\n\n\n### 4. Define the Event (API Endpoint)\n\nCreate the file `src/events/chatgpt_event.yaml` to expose an HTTP POST endpoint for ChatGPT interaction:\n\n```yaml\nhttp.post./chatgpt:\n summary: \"Generate response from ChatGPT\"\n description: \"Endpoint to send a user prompt to ChatGPT and retrieve the AI-generated response.\"\n fn: prompt\n authn: false\n body:\n content:\n application/json:\n schema:\n type: object\n properties:\n prompt:\n type: string\n description: \"The user's prompt or question for ChatGPT to respond to.\"\n required:\n - prompt\n responses:\n 200:\n description: \"Successful response from ChatGPT\"\n content:\n application/json:\n schema:\n type: string\n```\nThis creates a REST API at `POST /chatgpt`.\n\n\n### 5. Add the Function Logic\n\nCreate the file `src/functions/prompt.ts` for the actual function that interacts with the ChatGPT datasource:\n\n```ts\nimport { GSContext, GSDataSource, GSStatus } from \"@godspeedsystems/core\";\n\nexport default async function (ctx: GSContext, args: any) {\n const {\n inputs: {\n data: { body }\n }\n } = ctx;\n const prompt = body.prompt;\n const ds: GSDataSource = ctx.datasources.chatgpt;\n\n const response = await ds.execute(ctx, {\n prompt,\n meta: { method: 'chat' }\n });\n\n return response;\n}\n```\n\n* `ctx.datasources.chatgpt`: Loads the plugin's configured datasource.\n* `ds.execute(...)`: Invokes the chat method on the ChatGPT API with your prompt.\n\n\n### 6. Test the Endpoint\n\nOnce your Godspeed server is running (`godspeed serve`), you can test the API:\n\n```bash\ncurl -X POST http://localhost:4000/chatgpt \\\n -H \"Content-Type: application/json\" \\\n -d '{\"prompt\": \"Explain quantum computing in simple terms.\"}'\n```\n\nYou should receive a response directly from the ChatGPT model as JSON.\n\n### 7. Notes\n\n* ✅ **No manual axios or fetch setup** needed — the plugin handles it.\n* ✅ You can adjust temperature, model, and tokens in `chatgpt.yaml`.\n* ✅ Extend the plugin with retry logic, caching, or prompt templates using standard Godspeed techniques.\n\n---\n### Summary\n\n| File | Purpose |\n| ------------------------------- | ---------------------------------- |\n| `.env` | Stores your OpenAI API key |\n| `src/datasources/chatgpt.yaml` | Datasource config for the plugin |\n| `src/events/chatgpt_event.yaml` | HTTP API event to invoke ChatGPT |\n| `src/functions/prompt.ts` | Function that calls OpenAI’s model |\n\n---" }, "docs/microservices-framework/datasources/datasource-plugins/Kafka Datasource.md": { "content": "---\ntitle: Kafka Plugin - Datasource and Eventsource\ndescription: A powerful messaging system plugin that enables real-time event streaming, secure data transfer, and distributed messaging in Godspeed applications. Features include producer/consumer functionality, configurable topics, and seamless event handling.\nkeywords: [kafka, event streaming, messaging system, producer consumer, godspeed plugin, real-time events, distributed messaging, event sourcing, message queue, data streaming]\n---\n\n**[Link to Plugin Source Code](https://github.com/godspeedsystems/gs-plugins/tree/main/plugins/kafka-as-datasource-as-eventsource)**\n\nKafka is a versatile messaging system designed to securely transfer data between various systems. Its functionality can be tailored through configuration, allowing it to serve as a reliable conduit for real-time event tracking or even function as a replicated distributed database. While it's often colloquially labeled as a queue, it's more precisely described as a hybrid system that combines characteristics and trade-offs from both queue and database systems.\n\n## Steps to use kafka plugin in godspeed\n\n### Install Kafka Plugin\n```\ngodspeed plugin add @godspeedsystems/plugins-kafka-as-datasource-as-eventsource\n```\n### Related files\n\nAfter installation, you will find auto-generated files in your project related to the plugin at `src/datasources/types/kafka.ts` and `src/datasources/kafka.yaml`and `src/eventsources/types/kafka.ts` and `src/datasources/kafka.yaml`.\n\n### Use as Datasource (Producer)\n\n**1.** Update configuration file based on your requirements in `src/datasources/kafka.yaml` file.\n\n```yaml title=src/datasources/kafka.yaml\ntype: kafka \nclientId: \"kafka_proj\"\nbrokers: [\"kafka:9092\"]\n```\n\n**2.** In the event, we establish an HTTP endpoint that accepts parameters such as the topic name and message content. When this endpoint is invoked, it triggers the `datasource.kafka.producer` function. This function, in turn, takes the provided topic name and message as input arguments and performs the task of publishing the message to the specified Kafka topic.\n```yaml title=src/events/kafka_pub.yaml\n# event for Publish\n'http.post./kafka-pub':\n fn: kafka-publish\n body:\n content:\n application/json:\n schema:\n type: object\n properties:\n message:\n type: string\n required: ['message']\n responses:\n 200:\n content:\n application/json:\n schema:\n type: object\n properties:\n name:\n type: string\n```\n\n**3. ** In workflow we need to mension `datasource.kafka.producer` as function `fn` to produce data.\n```yaml title=src/functions/kafka-publish.yaml\nid: kafka-publish\nsummary: kafka publish message\ntasks:\n - id: publish\n fn: datasource.kafka.producer\n args:\n topic: \"publish-producer1\"\n message: <% inputs.body.message%>\n```\n\n### Use as EventSource (Consumer)\n**1. ** Update configuration file based on your requirements in `src/eventsources/kafka.yaml`.\n```yaml title=src/eventsources/kafka.yaml\ntype: kafka\ngroupId: \"kafka_proj\"\n\n```\n\n**2. ** To use Consumer we need to follow the below event key format.\n```yaml title=src/events/kafka_pub.yaml\nkafka.{Topic}.{GroupId}: \n```\nThe consumer event is triggered whenever a new message arrives on the specified topic. Upon triggering, it retrieves the incoming message and forwards it to the `kafka_consume` function. Inside this function, the incoming message is processed, and the result is then returned.\n\n``` yaml title=src/events/kafka_pub.yaml\n# event for consume data from Topic\nkafka.publish-producer1.kafka_proj: // event key\n id: kafka__consumer\n fn: kafka_consume\n body:\n description: The body of the query\n content:\n application/json: \n schema:\n type: string\n```\n**3. ** kafka workflow for consumer\n``` title=src/functions/kafka_consume.yaml\n# function for consume data\nid: kafka-consumer\nsummary: consumer\ntasks:\n - id: set_consumer\n fn: com.gs.return\n args: <% inputs %>\n```\n\n## Project Structure:\n\nAfter implementing Kafka plugin, your project structure will look like this:\n\n```\n.\n├── src\n│ ├── datasources\n│ │ ├── types\n│ │ │ └── kafka.ts # Custom datasource logic\n│ │ └── kafka.yaml # Datasource configuration\n│ ├── events\n│ │ ├── kafka_publish_event.yaml # Event to trigger the Kafka publish\n│ │ └── kafka_consumer_event.yaml # Event to trigger the Kafka consume\n│ ├── eventsources\n│ │ └── kafka.yaml # Eventsource configuration\n│ └── functions\n│ ├── kafka-publish.yaml # Workflow to publish message\n│ └── kafka-consume.yaml # Workflow to consume message\n```\n\n## Reference links\n**-** [Plugin Repository](https://github.com/godspeedsystems/gs-plugins/tree/main/plugins/kafka-as-datasource-as-eventsource) \n**-** [Issue Tracker](https://github.com/godspeedsystems/gs-plugins/issues) \n**-** [npm package](https://www.npmjs.com/package/@godspeedsystems/plugins-kafka-as-datasource-as-eventsource)\n" }, "docs/microservices-framework/datasources/datasource-plugins/Memcached Datasource.md": { "content": "---\ntitle: Memcached Plugin for Godspeed\ndescription: A high-performance in-memory caching plugin that enables seamless data caching and retrieval in Godspeed applications. Features include key-value storage, cache invalidation, and configurable caching strategies for optimized application performance.\nkeywords: [memcached, in-memory cache, caching, key-value store, godspeed plugin, performance optimization, cache invalidation, data caching, memory cache, cache management]\n---\n\n**[Link to Plugin Source Code](https://github.com/godspeedsystems/gs-plugins/tree/main/plugins/mem-cache-as-datasource)**\n\nThe Godspeed mem-cache plugin provides caching interfaces, allowing developers to seamlessly use in-memory cache within the Godspeed framework.\n\n## How to add mem-cache plugin in your project\n\n- **Create a godspeed project from the CLI and add the mem-cache plugin**\n\n```\ngodspeed plugin add @godspeedsystems/plugins-mem-cache-as-datasource\n```\n\nYou will find two files in your project related to the plugin at `src/datasources/types/mem-cache.ts` and `src/datasources/mem-cache.yaml`.\n\n```yaml title=src/datasources/mem-cache.yaml\ntype: mem-cache\n```\n\n### Sample Usage\nCreate two events and two event handler workflows for each event, by the name helloworld2 and helloworld3 respectively.\n\n### Events\n```yaml\nhttp.get./helloworld2:\n fn: helloworld2\n\nhttp.get./helloworld3:\n fn: helloworld3\n```\n### Functions (Helloworld2 workflow)\n\n```\nid: helloworld2_workflow\ntasks:\n - id: helloworld2_workflow_first_task\n fn: com.gs.transform\n args:\n name: helloworld2\n caching:\n key: xyz\n # datasource: memcache #This field should be definitely set if config/default.caching is not set. Else is optional\n # noRead: true #if this is set get(key) method will not be called for this task\n # noWrite: true #the result of this task will not be written, even if cache_on_failure is set to true. i.e. set() method will not be called\n``` \n### Functions (Helloworld3 workflow)\n```\nid: helloworld3_workflow\ntasks:\n - id: helloworld3_workflow_first_task\n caching:\n key: abc\n invalidate: xyz #helloworld2 key\n # noRead: true #if this is set get(key) method will not be called for this task\n # noWrite: true #the result of this task will not be written, even if cache_on_failure is set to true. i.e. set() method will not be called\n datasource: mem-cache #This field should be definitely set if config/default.caching is not set. Else is optional\n fn: com.gs.transform\n args:\n name: helloworld3\n```\n\n## Plugin Components\n\nThe plugin consists of the following key components:\n\n### 1. `DataSource` Class\n\n- This class extends `GSCachingDatasource`, a base class provided by the Godspeed framework for creating data sources.\n\n- It initializes a client which provides an object to use as in-memory cache.\n\n- The `set` method is used to set the key and value of the cache.\n\n- The `get` method is used to get value from the cache.\n\n- The `del` method is used to delete value from the cache.\n\n### 2. Constants\n\n- `SourceType`: A constant representing the source type of the plugin, which is 'DS' (data source).\n\n- `Type`: A constant representing the loader file of the plugin. The final loader file will be located in the 'types' directory and named `${Type.js}`, where `Type` is 'mem-cache' in this case.\n\n- `CONFIG_FILE_NAME`: In the context of a data source, this constant also serves as the data source name. In this plugin, it is set to 'mem-cache'.\n\n### Conclusion\n\nThe Godspeed mem-cache Plugin is a valuable addition to the Godspeed framework, providing a standardized way to use in-memory cache.\n\nWe value your feedback and contributions. If you have any questions, suggestions, or encounter issues while using the plugin, please reach out to us. Your insights and ideas help us enhance and improve this plugin for the entire Godspeed community.\n\nWe're excited to see how you leverage the Godspeed mem-cache Plugin in your projects and look forward to collaborating with you to make your applications even more powerful. Happy coding!\n\n## Reference links\n**-** [Plugin Repository](https://github.com/godspeedsystems/gs-plugins/tree/main/plugins/mem-cache-as-datasource) \n**-** [Issue Tracker](https://github.com/godspeedsystems/gs-plugins/issues) \n**-** [npm package](https://www.npmjs.com/package/@godspeedsystems/plugins-mem-cache-as-datasource)\n" }, "docs/microservices-framework/datasources/datasource-plugins/Mongoose Datasource.md": { "content": "---\ntitle: Mongoose Datasource Plugin\ndescription: A powerful MongoDB integration plugin that enables seamless database operations through Mongoose ODM in Godspeed applications. Features include data modeling, CRUD operations and efficient query handling with MongoDB.\nkeywords: [mongoose plugin, prisma with mongodb, nosql database, godspeed datasource plugin, mongoose integration]\n---\n\n**[Link to Plugin Source Code](https://github.com/godspeedsystems/gs-plugins/tree/main/plugins/mongoose-as-datasource)** \n\nIt provides seamless integration with MongoDB through the Mongoose library. MongoDB is a popular NoSQL database, and with this plugin, you can harness the power of Mongoose to model your data, perform queries, and interact with MongoDB in a structured and efficient manner.\n\n### How to Add \nCreate a godspeed project from the CLI, open the created project in vscode and then add the plugin:\n\n```\ngodspeed plugin add @godspeedsystems/plugins-mongoose-as-datasource \n```\n\n**-** You will find two files in your project related to the mongoose plugin at `src/datasources/types/mongoose.ts` and `src/datasources/mongoose.yaml`\n\n```typescript title=src/datasources/types/mongoose.ts\nimport { DataSource } from '@godspeedsystems/plugins-mongoose-as-datastore';\nexport default DataSource;\n```\n\n```yaml title=src/datasources/mongoose.yaml\ntype: mongoose\nsuccessResponseCodes: #default response codes for success responses\n create: 201\n find: 200\n findOne: 200\n aggregate: 200\n findOneAndUpdate: 201\n findOneAndDelete: 202\n```\n ![Alt text](../../../../static/img/mongoose_folder_structure.png)\n- You can keep the file by any name. This file is used to initialize a mongoose datasource instance. Whatever is the name of the file, you will need to invoke\nthe mongoose datasource commands by the same name. Also your models will be needed to be kept in a folder with the same name as your yaml file (i.e. your datasource instance name). For example mongoose1.yaml would mean\ncalling `fn:datasources.mongoose1..` from yaml workflows and \n`ctx.datasources.mongoose1..` from TS/JS files. Also you will need to create a folder `datasources/mongoose1/models` and keep your models there as detailed below.\n\n- You can override the default response codes for success cases for different methods by putting them in the datasource instance's yaml file\n\n### Provide Connection URL\n\nSet an environment variable named `MONGO_URL` as your connection string to running mongodb instance.\nYou can save url in .env file as\n \n ```\n MONGO_URL='mongodb+srv://:@cluster0.xyzabc.mongodb.net/?retryWrites=true&w=majority'\n ```\n\n### Setting up Mongoose models\nThis datasource loads the [Mongoose models](https://mongoosejs.com/docs/models.html) from `datasources//models` folder.\n\n![Alt text](../../../../static/img/mongoose_folder_structure.png)\n\n \nThese files are stored in `datasources//models` folder.\n\n### Writing Mongoose Models in Godspeed\n\nGodspeed framework uses **CommonJS** module syntax. When defining Mongoose models, ensure you follow the below format:\n\n**Import Mongoose:** Use the require syntax to import Mongoose:\n```ts\nconst { model, Schema, Document } = require('mongoose');\n```\n\n**Export the Model:**\n Export the model using the module.exports syntax. The export should include a type property (used for accessing the model) and the Mongoose model instance itself.\n```ts\nmodule.exports = {\n type: 'SomeModel', // The name by which you will access methods of this collection/model\n model: SomeModel // The Mongoose Model\n};\n```\n\n### An example Mongoose model file\n\n```typescript title=datasources//models/Participant.ts\n\nconst { model, Schema } =require('mongoose');\n\nconst participantSchema = new Schema({\n participant_id: { type: Number, required: true, unique: true },\n name: { type: String, required: true },\n email: { type: String, required: true },\n region: { type: String, required: true },\n city: { type: String, required: true }\n});\n\nconst Participant = model('Participant', participantSchema);\nmodule.exports = {\n type: 'Participant', // The name by which you will access methods of this collection/model\n model: Participant // The Mongoose Model\n};\n\n```\n\n### Sample Event and workflow\n\n### Event\n```yaml title=events/createParticipant.yaml\nhttp.post./participant:\n fn: createParticipant\n body:\n content:\n application/json:\n schema:\n type: object\n properties:\n participant_id:\n type: number\n name:\n type: string\n description: Name\n email:\n type: string\n description: Email id\n region:\n type: string\n responses:\n 201:\n description: Participant created successfully\n content:\n application/json:\n schema:\n type: object\n properties:\n someModel:\n type: object\n description: The created Participant object\n```\n### Workflow\nWhen calling any api function it will be called as `ctx.datasources.mongoose1..` from TS/JS files.\n\n### Create\n```ts title=createParticipant.ts\nimport { GSContext, GSDataSource, GSStatus } from \"@godspeedsystems/core\";\n\nexport default async function (ctx: GSContext) {\n\n const mongoClient: GSDataSource = ctx.datasources.mongoose;\n const { body }= ctx.inputs.data ;\n const data = { \n meta: {\n entityType: 'Participant', \n method: 'create'\n },\n ...body\n };\n \n const response = await mongoClient.execute(ctx, data);\n return response;\n}\n```\n\n### Find\n```ts title=getParticipant.ts\nimport { GSContext, GSDataSource, GSStatus } from \"@godspeedsystems/core\";\n\nexport default async function (ctx: GSContext){\n const { query } = ctx.inputs.data; // for GET request, query params are here\n const { email } = query;\n\n const mongoClient: GSDataSource = ctx.datasources.mongoose;\n\n try {\n const result = await mongoClient.execute(ctx, {\n meta: {\n entityType: 'Participant',\n method: 'findOne'\n },\n email: email \n });\n\n return result;\n } catch (error: any) {\n ctx.childLogger.error(\"Failed to fetch user by email: %o\", error);\n return new GSStatus(false, 500, \"Internal Server Error\", error);\n }\n}\n```\n\n### Error response\nWhen a call has an error the datasource returns following `GSStatus`.\n```yaml\n code: 500\n success: false\n data: \n message: Internal Server Error\n```\n\n### Run the service\n- From your command line run your service in the auto-watch mode\n ```bash\n godspeed serve\n ```\n\n## Reference links\n**-** [Plugin Repository](https://github.com/godspeedsystems/gs-plugins/tree/main/plugins/mongoose-as-datasource) \n**-** [Issue Tracker](https://github.com/godspeedsystems/gs-plugins/issues) \n**-** [npm package](https://www.npmjs.com/package/@godspeedsystems/plugins-mongoose-as-datasource)\n\n" }, "docs/microservices-framework/datasources/datasource-plugins/Nodemailer Datasource.md": { "content": "---\ntitle: Nodemailer Datasource Plugin\ndescription: A powerful email integration plugin that enables seamless email delivery in Godspeed applications using Nodemailer. Features include dynamic templating, error resilience, secure delivery, and support for various email services like Gmail and SMTP.\nkeywords: [nodemailer, email delivery, smtp, email templates, transactional email, godspeed plugin, email service, messaging, email notifications, email integration]\n---\n\n**[Link to Plugin Source Code](https://github.com/godspeedsystems/gs-plugins/tree/main/plugins/mailer-as-datasource)**\n\nAmp up your communication game by using a mailer as a powerful data source. Connect seamlessly to send information through emails. Transform your app into a messaging maestro, unlocking a world of possibilities.\n\nSending emails in your Node.js application has never been smoother. The Godspeed Nodemailer Plugin provides seamless integration between the robust Godspeed framework and Nodemailer, the go-to solution for email delivery in Node.js.\n\n## Features\n\n**- Effortless Setup:** Get up and running in minutes with our easy-to-follow setup guide. \n**- Dynamic Templating:** Craft personalized emails with dynamic content using popular templating engines. \n**- Error Resilience:** Robust error handling ensures reliable email delivery, even in challenging scenarios. \n**- Scalable and Secure:** Designed for scalability and security, so your email system can grow with your application.\n\nWhether you're sending transactional emails, newsletters, or notifications, this plugin empowers you to deliver messages with Godspeed. Let's elevate your email game together!\n\n### How to Add \nCreate a godspeed project from the CLI, open the created project in vscode and then add the plugin:\n\n```\ngodspeed plugin add @godspeedsystems/plugins-mailer-as-datasource\n```\n\n### Example usage\n\n### Config\n```yaml title=src/datasources/mailer.yaml\ntype: mailer\nservice: gmail \nuser: godspeed@gmail.com\npass: rmeb bjak xcam xkub # app specific password\n```\n\n### Event for Producer\n```yaml title=src/events/mail_send_event.yaml\nhttp.post./email:\n summary: Send email to user\n description: Sends an email to a user using the mailer datasource\n fn: send_email\n authn: false\n body:\n content:\n application/json:\n schema:\n type: object\n properties:\n recipient:\n type: string\n subject:\n type: string\n body:\n type: string\n required:\n - recipient\n - subject\n - body\n responses:\n '200':\n description: Email sent successfully\n content:\n application/json:\n schema:\n type: object\n properties:\n success:\n type: boolean\n example: true\n data:\n type: object # Or a more specific schema if the mailer returns structured data\n '400':\n description: Failed to send email\n content:\n application/json:\n schema:\n type: object\n properties:\n success:\n type: boolean\n error:\n type: string\n```\n### Typescript Workflow to send mail\n```typescript title=src/functions/send_email.ts\n\nimport { GSContext, GSStatus, GSDataSource } from \"@godspeedsystems/core\";\nexport default async function (ctx: GSContext) {\n const { body} = ctx.inputs.data;\n const mailerClient: GSDataSource = ctx.datasources.mailer;\n\n try {\n const response = await mailerClient.execute(ctx, {\n to: body.recipient,\n subject: body.subject,\n text: body.body\n });\n return new GSStatus(true, 200, \"Email Sent successfully\", response);\n } catch (error: any) {\n const errorData = error.stack || error;\n return new GSStatus(false, 400, \"Failed to send email\", errorData);\n }\n}\n```\n\n\n\n## Reference links\n**-** [Plugin Repository](https://github.com/godspeedsystems/gs-plugins/tree/main/plugins/mailer-as-datasource) \n**-** [npm package](https://www.npmjs.com/package/@godspeedsystems/plugins-mailer-as-datasource)\n" }, "docs/microservices-framework/datasources/datasource-plugins/Overview.md": { "content": "---\ntitle: Datasource Plugins Overview for Godspeed Framework\ndescription: A comprehensive guide to Godspeed's pluggable datasource system. Learn about available plugins, installation process, and how to integrate various data sources into your Godspeed applications with ease.\nkeywords: [godspeed plugins, datasource plugins, plugin management, data integration, plugin installation, framework plugins, data sources, plugin system, plugin architecture]\n---\n\n# Datasource Plugins Overview\n\nGodspeed framework adopts a pluggable approach that empowers you to define data sources effortlessly. Our framework graciously provides an interface that caters to diverse data source needs. Here's a glimpse into the exceptional datasource plugins crafted by our core framework team.\n\nTo seamlessly integrate these plugins into your project, simply run the command:\n\n```bash\n$ godspeed plugin add\n```\n```bash\n\n\n ,_, ╔════════════════════════════════════╗\n (o,o) ║ Welcome to Godspeed ║\n ({___}) ║ World's First Meta Framework ║\n \" \" ╚════════════════════════════════════╝\n\n\n? Please select godspeed plugin to install: (Press to select, to move rows)\n┌──────┬────────────────────────────────────────┬────────────────────────────────────────────────────────────────────────────────┐\n│ │ Name │ Description │\n├──────┼────────────────────────────────────────┼────────────────────────────────────────────────────────────────────────────────┤\n│ ❯◯ │ cron-as-eventsource │ Cron as eventsource plugin for Godspeed Framework │\n├──────┼────────────────────────────────────────┼────────────────────────────────────────────────────────────────────────────────┤\n│ ◯ │ aws-as-datasource │ aws as datasource plugin for Godspeed Framework │\n├──────┼────────────────────────────────────────┼────────────────────────────────────────────────────────────────────────────────┤\n│ ◯ │ excel-as-datasource │ excel as datasource plugin for Godspeed Framework │\n├──────┼────────────────────────────────────────┼────────────────────────────────────────────────────────────────────────────────┤\n│ ◯ │ mangoose-as-datasource │ mongoose-as-datasource as datasource plugin for Godspeed Framework │\n├──────┼────────────────────────────────────────┼────────────────────────────────────────────────────────────────────────────────┤\n│ ◯ │ mailer-as-datasource │ mailer as datasource plugin for Godspeed Framework │\n├──────┼────────────────────────────────────────┼────────────────────────────────────────────────────────────────────────────────┤\n│ ◯ │ kafka-as-datasource-as-eventsource │ kafka as datasource-as-eventsource plugin for Godspeed Framework │\n└──────┴────────────────────────────────────────┴────────────────────────────────────────────────────────────────────────────────┘\n\n```\n\nyou can specify plugin name to add directly to your project\n\n```sh\ngodspeed plugin add @godspeedsystems/plugins-\n```\n\n### Examples\n```bash\n$ godspeed plugin add @godspeedsystems/plugins-prisma-as-datastore\n```\n\n```bash\n$ godspeed plugin add @godspeedsystems/plugins-kafka-as-datasource-as-eventsource \n```" }, "docs/microservices-framework/datasources/datasource-plugins/Prisma Datasource.md": { "content": "---\ntitle: Prisma Datasource Plugin\ndescription: A powerful database ORM plugin that enables seamless integration with multiple databases in Godspeed applications. Features include type-safe queries, schema migrations, CRUD API generation, data encryption, and row/column level authorization.\nkeywords: [prisma, orm, database integration, datasource, generate crud apis, postgresql, mysql, mongodb, mongoose, sqllite, database migration, crud operations, godspeed prisma plugin, type safety, mongodb with prisma]\ntoc_min_heading_level: 2\ntoc_max_heading_level: 4\n---\n\n\n**[Link to Plugin Source Code](https://github.com/godspeedsystems/gs-plugins/tree/main/plugins/prisma-as-datastore)**\n\n## Overview\nPrisma-as-datasource plugin provides functionality to access most popular databases like, PostgreSQL, MySQL, SQL Server, SQLite, MongoDB, CockroachDB, Planetscale and MariaDB through [Prisma ORM](https://www.prisma.io/docs).\n\n**\"Prisma: Bridging Databases for Seamless Development. One Toolkit, Any Database.\"**\n\nPrisma is a modern and open-source database toolkit that simplifies database access for developers. It offers a strongly typed query builder, schema migrations, support for various databases, real-time data synchronization, and enhanced security, making it a powerful tool for efficient and secure database interactions in web applications.\n\n### How to Add plugin\nCreate a godspeed project from the CLI, open the created project in vscode and then add the plugin:\n\n```\ngodspeed plugin add @godspeedsystems/plugins-prisma-as-datastore\n```\n\n### Related files\nYou will find a file in your project related to the Prisma plugin at `src/datasources/types/prisma.ts`.\n\n```typescript title=prisma.ts\n\nimport { DataSource } from '@godspeedsystems/plugins-prisma-as-datastore';\nexport default DataSource;\n```\n\n## How to use\n\n### 1. Write a prisma schema\n\n 1.1 You can start using this plugin by writing a [prisma schema](https://www.prisma.io/docs/orm/prisma-schema). For this you need to create a file with .prisma extension inside `src/datasources/`.\n\n 1.2 Set the url field of the datasource block in your schema to your database connection URL as shown below:\n\n src/datasources/schema.prisma\n ```\n datasource db { \n provider = \"mysql\" // name of database provider\n url = env(\"DB_URL\") // DB_URL string will be added in .env file\n }\n generator client {\n provider = \"prisma-client-js\"\n output = \"./prisma-clients/schema\" //schema is the name of your prisma schema file\n previewFeatures = [\"metrics\"]\n }\n ```\n Set provider to the type of database you are using. In this case it’s mysql. The url property will take the value of the connection url which is defined in the .env file.\n :::tip\n Ensure you add the `output field` in your Prisma schema's generator block. This field should point to this location `src/datasources/prisma-clients/` where the generated prisma client files will be stored.\n :::\n\n### 2. Set your Database Connection URL\n\nSet your Database Connection URL as environment variable in .env file as per the format. The format of the connection URL for your database depends on the [database](../../databases/Overview.md) you use. For PostgreSQL, it looks as below\n\n DB_URL = postgresql://USER:PASSWORD@HOST:PORT/DATABASE\n \n The parts spelled all-uppercased are placeholders for your specific connection details\n \n Example Connection String\n ```\n DB_URL = \"postgresql://johndoe:password@localhost:5432/mydb?schema=public\"\n ```\n### 3. Generate data models\n\nThe next step is to generate or define the data models. Prisma uses the connection url you provided to connect to the database. \n\n#### 3.1 If you have an existing database\n\nTo connect with your existing database, first install prisma and run prisma db pull command by giving path of your schema.prisma file.\n```\nnpm install prisma --save-dev\n\nprisma db pull --schema = src/datasources/schema.prisma\n```\n\nIf the command has run successfully Prisma will generate models from your database server and save in schema.prisma file. If the Prisma schema is new to you, have a look at their [documentation](https://www.prisma.io/docs/getting-started).\n\n#### 3.2 If you don't have an existing database,\n\nThen add the data models to your Prisma schema in datasources/schema.prisma as:\n\n### Sample prisma schema\n\n```\ndatasource db {\n provider = \"PostgreSQL\" // database provider name which you are using\n url = env(\"DB_URL\") // DB_URL is the name of env variable\n}\ngenerator client {\n provider = \"prisma-client-js\"\n output = \"./prisma-clients/schema\" // dbName should be same as prisma schema file name\n previewFeatures = [\"metrics\"]\n}\n\nmodel User {\n id Int @id @default(autoincrement())\n email String @unique\n name String?\n posts Post[]\n}\n\nmodel Post {\n id Int @id @default(autoincrement())\n title String\n content String?\n published Boolean @default(false)\n author User @relation(fields: [authorId], references: [id])\n authorId Int\n}\n\n```\n:::tip Support for multiple prisma schema\n:::\nBy default, only single prisma schema can be created in a project that can use only one database. To support multiple prisma schemas for different databases, you need to add `output` key in `generator client` block as given in the above sample prisma schema. \n\n### 4. Generate prisma client\n\nRun `godspeed prisma prepare`. It will generate your prisma client for given schema and will place the generated client in the `src/datasources/prisma-clients/` folder. This is achieved internally by `prisma generate` command. \nIt will also setup the provided schema on database. This is achieved internally by the command `prisma db push`\n\n ```bash\n $ godspeed prisma prepare\n ```\n \nOnce you have generated prisma client, multiple clients get generated in `src/datasources/prisma-clients` directory. Godspeed automatically loads all the clients present in this directory.\n\n### Generate CRUD APIs\nYou can generate the CRUD API'S by entering the below command:\n ```bash\n godspeed gen-crud-api\n ```\n* This command will generate the crud apis based on the sample schema provided at ./src/datasources/schema.prisma\n\n* You can now view events and workflows generated under events and functions folder. They follow a structure similar to the APIs below.\n\n### Sample API\nIf your schema name is mysql.prisma and model name is 'post', then your event and workflow to fetch data from the database, will look like :\n\n```yaml title = src/events/post.yaml\n\nhttp.get./mysql/post/{id}:\n summary: Fetch Post\n description: Fetch Post from database\n fn: com.biz.mysql.post.one\n params:\n - name: id\n in: path\n required: true\n schema:\n type: string\n responses:\n content:\n application/json:\n schema:\n type: object\n```\n\n```ts title = src/functions/com/biz/mysql/post/one.ts\n\nimport { GSContext, GSStatus, PlainObject } from \"@godspeedsystems/core\";\nimport { PrismaClient } from \"@prisma/client\";\n\nmodule.exports = async (ctx: GSContext, args: PlainObject) => {\n const { inputs: { data: { params } }, logger, datasources } = ctx;\n\n const client: PrismaClient = datasources.mysql.client;\n\n const response = await client.Post.findUnique({\n where: { id: params.id }\n });\n return new GSStatus(true, 200, \"Post fetched\", response );\n}\n```\n### More Examples\n\n[Check more typescript function examples to interact with prisma datasource](/docs/microservices-framework/how-to/sample-ts-functions.md)\n\n\n\n### Database Encryption\nGodspeed provides AES-256 GCM both way deterministic hashing encryption in Prisma plugin. You can apply encryption only on `String` type fields.\n\n#### Annotate prisma schema\nIn your prisma schema, add `/// @encrypted` annotation to the fields you want to encrypt.\n\n
\nsample schema to encrypt email field\n\n```prisma title=src/datasources/mongo.prisma\ndatasource db {\n provider = \"mongodb\"\n url = env(\"MONGO_TEST_URL\") //Connection string can be found in the .env file, you can add your own database connection string\n}\n\ngenerator client {\n provider = \"prisma-client-js\"\n output = \"./prisma-clients/mongo\"\n}\n\nmodel User {\n id String @id @default(auto()) @map(\"_id\") @db.ObjectId\n createdAt DateTime @default(now())\n email String @unique /// @encrypted\n name String?\n role Role @default(USER)\n posts Post[]\n}\n\nmodel Post {\n id String @id @default(auto()) @map(\"_id\") @db.ObjectId\n createdAt DateTime @default(now())\n updatedAt DateTime @updatedAt\n published Boolean @default(false)\n title String\n author User? @relation(fields: [authorId], references: [id])\n authorId String @db.ObjectId\n}\n\nenum Role {\n USER\n ADMIN\n}\n```\n\n
\n\n#### Add secret\nYou can specify secret in `prisma_secret` variable in [config environment variables](../../config-and-mappings/config.md/#custom-environment-variablesyaml).\n\n\n\n \n" }, "docs/microservices-framework/datasources/datasource-plugins/Redis Datasource.md": { "content": "---\ntitle: Redis Datasource Plugin\ndescription: A high-performance Redis integration plugin that enables fast caching, real-time data operations, and key-value storage in Godspeed applications. Features include seamless Redis operations, error handling, and support for common Redis commands.\nkeywords: [redis, key-value store, caching, real-time data, in-memory database, godspeed plugin, data storage, performance optimization, redis operations, data retrieval]\n---\n\n**[Link to Plugin Source Code](https://github.com/godspeedsystems/gs-plugins/tree/main/plugins/redis-as-datasource)** \n\nElevate your data game with the speed and efficiency of Redis. Use it as a powerhouse for caching, real-time analytics, and lightning-fast data retrieval. Embrace the key-value magic to supercharge your application's performance.\n\nThe Godspeed Redis Plugin provides integration with Redis, allowing developers to seamlessly interact with Redis databases within the Godspeed framework. This plugin simplifies the process of working with Redis data, providing a standardized way to perform common Redis operations.\n\nYou can use redis datasource in caching also. Check [caching](../caching.md) for more information.\n\n### How to Add plugin\nCreate a godspeed project from the CLI, open the created project in vscode and then add the plugin:\n\n```\ngodspeed plugin add @godspeedsystems/plugins-redis-as-datasource\n```\n\nYou will find the files in your project related to the Redis plugin at `src/datasources/types/redis.ts` and `src/datasources/redis.yaml`.\n\n\n### 1. redis config\n```yaml title=src/datasources/redis.yaml\ntype: redis\nurl: redis://alice:foobared@awesome.redis.server:6380\n\n```\nConfigure your Redis data source with connection string. \n\n### 2. Get a Value\nWorkflow to get a value from redis.\n```yaml\nid: get_redis_value\ntasks:\n - id: get_task\n fn: datasource.redis.get\n args:\n key: 'example_key'\n```\n\n### 3. Set a Value\nWorkflow to set a value in redis.\n```yaml\nid: set_redis_value\ntasks:\n - id: set_task\n fn: datasource.redis.set\n args:\n key: 'example_key'\n value: 'example_value'\n```\n\n## How it helps\n\nThe Godspeed Redis Plugin offers the following benefits:\n\n** 1. Redis Integration:** The plugin abstracts away the complexities of setting up a Redis client, making it effortless to connect to Redis databases and perform operations.\n\n**2. Unified Data Source:** Developers can use a uniform API to define data sources that interact with Redis. This enhances consistency and ease of use across different parts of the application.\n\n**3. Error Handling:** The plugin includes robust error handling, allowing developers to gracefully handle scenarios such as connection issues, key not found, and other Redis-related errors.\n\n**4. Integration with Godspeed Core:** The plugin seamlessly integrates with the Godspeed Core library, aligning with the principles of the Godspeed framework and enabling streamlined event-driven workflows.\n\n## Plugin Components\n\n:::info\nYou can deep dive into the plugin code [here](https://github.com/godspeedsystems/gs-plugins/tree/main/plugins/redis-as-datasource)\n:::\n\nThe plugin consists of the following key components:\n### 1. `DataSource` Class\n- This class extends `GSDataSource` a base class provided by the Godspeed framework for creating data sources.\n- It initializes a Redis client to interact with Redis databases based on the provided configuration options.\n\n- The `execute` method is used to define how the plugin should perform Redis operations. It maps incoming parameters to Redis commands, processes the operation, and handles various response scenarios.\n\n### 2. Constants\n- `SourceType`: A constant representing the source type of the plugin, which is 'DS' (data source).\n\n- `Type`: A constant representing the loader file of the plugin. The final loader file will be located in the 'types' directory and named `${Type.js}`, where `Type` is 'redis' in this case.\n\n- `CONFIG_FILE_NAME`: In the context of a data source, this constant also serves as the data source name. In this plugin, it is set to 'redis'.\n\n- `DEFAULT_CONFIG`: A default configuration object with Redis options like host, port, and other settings.\n\n\n## Conclusion\n\nThe Godspeed Redis Plugin is a valuable addition to the Godspeed framework, providing a standardized way to interact with Redis databases. With this plugin, you can effortlessly perform Redis operations, handle responses, and streamline data storage within your applications.\n\nWe value your feedback and contributions. If you have any questions, suggestions, or encounter issues while using the plugin, please reach out to us. Your insights and ideas help us enhance and improve this plugin for the entire Godspeed community.\n\nWe're excited to see how you leverage the Godspeed Redis Plugin in your projects and look forward to collaborating with you to make your applications even more powerful. Happy coding!\n\n## Reference links\n**-** [Plugin Repository](https://github.com/godspeedsystems/gs-plugins/tree/main/plugins/redis-as-datasource) \n**-** [Issue Tracker](https://github.com/godspeedsystems/gs-plugins/issues) \n**-** [npm package](https://www.npmjs.com/package/@godspeedsystems/plugins-redis-as-datasource)" }, "docs/microservices-framework/datasources/datasource-plugins/Sendgrid Datasource.md": { "content": "---\ntitle: Sendgrid Datasource Pluginc\n---\n\n**[Link to Plugin Source Code](https://github.com/godspeedsystems/gs-plugins/tree/main/plugins/sendgrid-as-datasource)**\n\nThis plugin helps you automate email sending and track email metrics using SendGrid, with status logging to Google Sheets.\n\n## Features\n\n- Automated email sending via SendGrid API\n- Real-time email tracking (opens, clicks, bounces, etc.)\n- Automatic logging of email status to Google Sheets\n- Webhook integration for event tracking\n\n## Prerequisites\n\n- Node.js installed on your system\n- A SendGrid account\n- A Google Cloud account\n\n### Install SendGrid Plugin:\n\nCreate a godspeed project from the CLI, open the created project in vscode and then add the plugin:\n\n```\ngodspeed plugin add @godspeedsystems/plugins-sendgrid-as-datasource\n```\n\n### SendGrid Configuration:\n\n#### A. API Setup\n\n1. Log in to [SendGrid Dashboard](https://sendgrid.com/)\n2. Navigate to: Settings → API Keys → Create API Key\n3. Set permissions to \"Full Access\"\n4. Copy the generated API key\n5. Add to your `.env`:\n ```\n SENDGRID_API_KEY=your_api_key_here\n SENDGRID_DEFAULT_SENDER=your_default_sender_email\n ```\n\n#### B. Sender Authentication\n\n1. Go to Settings → Sender Authentication\n2. Choose authentication method:\n - **Recommended**: Domain Authentication\n - Follow the DNS configuration steps\n - Add provided DNS records to your domain provider\n - **Alternative**: Single Sender Verification\n - If you don't have a domain, then you opt for Single Sender Verification but this is not encouraged by SendGrid.\n - Use for testing only\n - Verify your sender email address, even the default sender that you set in env\n\n### Project Setup:\n\n### Related files\nAfter installing the plugin, you will get two auto generated files at `src/datasources/types/sendgrid.ts` and `src/datasources/sendgrid.yaml`\n\n### Configuration file\n\n```yaml title=sendgrid.yaml\ntype: sendgrid\napiKey: <%process.env.SENDGRID_API_KEY%>\ndefaultSender: <%process.env.SENDGRID_DEFAULT_SENDER%>\n```\n\n### Write API or event schema\n\n1. Create a file `sendgrid.yaml` inside events folder.\n2. Copy this inside that file:\n\n```\nhttp.post./send-mail:\n summary: 'Send mail'\n description: 'Endpoint to send a mail to a mail address'\n fn: sendMail\n authn: false\n body:\n content:\n application/json:\n schema:\n type: object\n properties:\n to:\n type: string\n format: email\n description: 'Recipient email address'\n subject:\n type: string\n description: 'Email subject'\n text:\n type: string\n description: 'Plain text content of the email'\n html:\n type: string\n description: 'HTML content of the email'\n from:\n type: string\n format: email\n description: 'Sender email address'\n required: ['to', 'subject', 'text', 'html']\n responses:\n 200:\n description: 'Email sent successfully'\n content:\n application/json:\n schema:\n type: object\n properties:\n message:\n type: string\n example: 'Email sent successfully.'\n 400:\n description: 'Bad request - Missing or invalid fields'\n content:\n application/json:\n schema:\n type: object\n properties:\n error:\n type: string\n example: 'Invalid request body: missing or invalid fields.'\n 500:\n description: 'Internal server error'\n content:\n application/json:\n schema:\n type: object\n properties:\n error:\n type: string\n example: 'Failed to send email.'\n```\n\n### Write Workflow/Function\n\n1. Create a file `sendMail.ts` inside the src/functions folder.\n2. Copy this inside that file:\n\n```ts\nimport { GSContext, GSDataSource, GSStatus } from '@godspeedsystems/core';\n\nexport default async function (ctx: GSContext, args: any) {\n const {\n inputs: {\n data: { body },\n },\n } = ctx;\n\n const { to, subject, text, html, from } = body;\n\n // Access the SendGrid datasource from the context\n const ds: GSDataSource = ctx.datasources.sendgrid;\n\n try {\n // Validate required fields before attempting to send the email\n if (!to || !subject || !text || !html) {\n const missingFields = [\n !to && 'to',\n !subject && 'subject',\n !text && 'text',\n !html && 'html',\n ]\n .filter(Boolean)\n .join(', ');\n\n ctx.childLogger.error(`Missing required fields: ${missingFields}`);\n return new GSStatus(\n false,\n 400,\n `Missing required fields: ${missingFields}`,\n );\n }\n // Execute the send function in SendGrid\n const response = await ds.execute(ctx, {\n to,\n subject,\n text,\n html,\n from: from || undefined,\n meta: { fnNameInWorkflow: 'datasource.sendgrid.sendMail' },\n });\n\n return response;\n } catch (error: unknown) {\n // Log and handle known and unknown errors\n if (error instanceof Error) {\n ctx.childLogger.error(`Failed to send email: ${error.message}`);\n return new GSStatus(false, 500, 'Failed to send email', error.message);\n }\n // Handle non-standard errors\n ctx.childLogger.error(`An unknown error occurred: ${error}`);\n return new GSStatus(false, 500, 'Failed to send email', `${error}`);\n }\n}\n```\n\n**That's it, now you can start the project using `godspeed serve`. To test it out, go to `/api-docs` endpoint to access the swagger UI. Then try to send a mail using `/send-mail` endpoint. Remember, only verified email addresses with SendGrid can be used in the `from` field.**\n\n---\n\n## Steps to store the email status logs in a google spreadsheet:\n\n#### For this, we will manually create another datasource to track the events and log them on the spreadsheet.\n\n### SendGrid Event Webhook Setup:\n\n1. Go to Settings → Mail Settings → Event Webhook\n2. Click on **Create Webhook URL** and configure webhook:\n - URL: `https://your-domain.com/event-tracking`\n - For local development:\n 1. Install [ngrok](https://ngrok.com/)\n 2. Run: `ngrok http 3000`\n 3. Use the generated HTTPS URL with /event-tracking\n - Select events: processed, delivered, open, click, bounce\n3. Save configuration\n\n### Google Sheets Integration:\n\n#### A. Google Cloud Setup\n\n1. Visit [Google Cloud Console](https://console.cloud.google.com/)\n2. Create/select a project\n3. Enable Google Sheets API:\n - Search for \"Google Sheets API\"\n - Click Enable\n\n#### B. Service Account Configuration\n\n1. In the left sidebar, navigate to IAM & Admin → Service Accounts\n2. Create service account:\n - Name: `godspeed-sendgrid`\n - Role: Project → Owner\n - Click on **create**\n3. Create key:\n - In the **Service Accounts** section, find the account you just created.\n - Click the three dots on the right and select **Manage Keys**.\n - Click **Add Key > Create New Key**.\n - Choose **JSON** as the key type and click **Create**.\n - A .json file containing the service account's credentials will be downloaded. Save this file in the root of your project with name **google-key.json**.\n\n#### C. Spreadsheet Setup\n\n1. Create a copy of our [template spreadsheet](https://docs.google.com/spreadsheets/d/13w9a9fwpGJDZ68husCHv2VU6rWtdGk7gtAxrIYTDxFc/edit?usp=sharing)\n2. Share with service account:\n - Open the Google Sheet that you just copied, we will use this to store email logs.\n - Click the **Share** button in the top-right corner.\n - In the **Share with people and groups** field, enter the **Service Account email** (you can find it in the downloaded .json file under the client_email field).\n - Set the permission to **editor**.\n - Click **Send** to share the sheet with the service account.\n3. Set env variables for google spreedsheet in the root of your project:\n\n- Your service account's credentials file\n\n```env\nGOOGLE_CREDENTIAL_FILE_PATH=your_google_key_filename\n```\n\n- Worksheet name\n\n```env\nGOOGLE_WORKSHEET_NAME=\"sheet1\"\n```\n\n- Google spreasheet ID. To extract the Google Sheet ID, find it in the URL between /d/ and /edit (e.g. 1A2B3C4D5E6F7G8H9I0J1K2L3M4N5O6P7Q8R9S0T1U2V).\n\n```env\nGOOGLE_SHEET_ID=sheet_id\n```\n\n### Project Setup:\n\n- Install googleapis npm package: `npm i googleapis`\n\n#### 1. We will make a custom datasource.\nCreate a `track.ts` file inside the `datasources/types` folder and copy this code there:\n\n```ts\nimport {\n GSContext,\n GSDataSource,\n GSStatus,\n PlainObject,\n} from '@godspeedsystems/core';\nimport { google } from 'googleapis';\n\nexport default class DataSource extends GSDataSource {\n private sheets?: any;\n private auth?: any;\n\n protected async initClient(): Promise {\n try {\n this.sheets = google.sheets('v4');\n this.auth = new google.auth.GoogleAuth({\n keyFile: this.config.keyFile,\n scopes: ['https://www.googleapis.com/auth/spreadsheets'],\n });\n return { sheets: this.sheets, auth: this.auth };\n } catch (error) {\n throw new Error(`Failed to initialize google client: ${error}`);\n }\n }\n\n async getSheetData(): Promise {\n try {\n const auth = await this.auth.getClient();\n const request = {\n spreadsheetId: this.config.sheetId,\n range: `${this.config.worksheetName}`,\n auth,\n };\n\n const response = await this.sheets.spreadsheets.values.get(request);\n return response.data.values || [];\n } catch (error) {\n throw new Error(`Failed to get data from Google Sheets: ${error}`);\n }\n }\n\n // Helper function to find the row number by sg_message_id\n async findRowById(sg_message_id: string): Promise {\n try {\n const sheetData = await this.getSheetData();\n for (let i = 1; i < sheetData.length; i++) {\n if (sheetData[i][0] === sg_message_id) {\n return i + 1; // Convert index to row number\n }\n }\n return null;\n } catch (error) {\n throw new Error(`Failed to find row by ID: ${error}`);\n }\n }\n\n // Method to find the next empty row in the sheet\n async getNextEmptyRow(): Promise {\n const sheetData = await this.getSheetData();\n return sheetData.length + 1; // Next empty row is one after the last filled row\n }\n\n async writeData(data: PlainObject) {\n const { range, values } = data;\n\n try {\n const auth = await this.auth.getClient();\n const request = {\n spreadsheetId: this.config.sheetId,\n range: `${this.config.worksheetName}!${range}`,\n valueInputOption: 'RAW',\n resource: { values },\n auth,\n };\n\n const response = await this.sheets.spreadsheets.values.update(request);\n return response;\n } catch (error) {\n throw new Error(`Failed to write data to google sheets: ${error}`);\n }\n }\n\n async execute(ctx: GSContext, args: any): Promise {\n // Ensure client is available\n if (!this.sheets || !this.auth) {\n ctx.childLogger.error('Google client is not initialized');\n return new GSStatus(false, 500, 'Google client is not initialized');\n }\n\n try {\n for (const data of args) {\n const { sg_message_id, email, event } = data;\n let range: string;\n let values: any[][];\n\n // Check if a row already exists for the `sg_message_id`\n let rowNumber = await this.findRowById(sg_message_id);\n\n // If no existing row, add a new entry with the sg_message_id and email\n if (!rowNumber) {\n rowNumber = await this.getNextEmptyRow();\n range = `A${rowNumber}`;\n values = [[sg_message_id, email]];\n await this.writeData({ range, values });\n }\n\n // Determine the column and value based on event type\n switch (event) {\n case 'processed':\n range = `C${rowNumber}`;\n values = [['processed']];\n break;\n case 'delivered':\n range = `D${rowNumber}`;\n values = [['delivered']];\n break;\n case 'open':\n range = `E${rowNumber}`;\n values = [['opened']];\n break;\n case 'click':\n range = `F${rowNumber}`;\n values = [['clicked']];\n break;\n case 'bounce':\n range = `G${rowNumber}`;\n values = [['bounced']];\n break;\n default:\n ctx.childLogger.warn(`Unhandled event type: ${event}`);\n continue; // Skip unhandled events\n }\n\n // Write the event-specific data to the designated cell\n await this.writeData({ range, values });\n }\n\n return new GSStatus(true, 200, 'Event tracked successfully');\n } catch (error: any) {\n ctx.childLogger.error(`Failed to execute data tracking: ${error}`);\n return new GSStatus(\n false,\n 500,\n 'Failed to execute data tracking',\n error.message,\n );\n }\n }\n}\nconst SourceType = 'DS';\nconst Type = 'track'; // this is the loader file of the plugin, So the final loader file will be `types/${Type.js}`\nconst CONFIG_FILE_NAME = 'track'; // in case of event source, this also works as event identifier, and in case of datasource works as datasource name\nconst DEFAULT_CONFIG = {};\n\nexport { DataSource, SourceType, Type, CONFIG_FILE_NAME, DEFAULT_CONFIG };\n```\n\n#### 2. Configuration file\n\n1. Create a file `track.yaml` inside datasources folder.\n2. Copy this inside that file:\n\n```\ntype: track\nsheetId: <%process.env.GOOGLE_SHEET_ID%>\nkeyFile: <%process.env.GOOGLE_CREDENTIAL_FILE_PATH%>\nworksheetName: <%process.env.GOOGLE_WORKSHEET_NAME%>\n```\n\n#### B. Event trigger\n\n1. Create a file `track.yaml` inside events folder.\n2. Copy this inside that file:\n\n```\nhttp.post./event-tracking:\n summary: 'Track SendGrid Events'\n description: 'Endpoint to track multiple email events from SendGrid in a single request'\n fn: track\n authn: false\n body:\n content:\n application/json:\n schema:\n type: array\n items:\n type: object\n required:\n - sg_message_id\n - email\n - event\n - timestamp\n properties:\n sg_message_id:\n type: string\n description: 'Unique identifier for the SendGrid message'\n email:\n type: string\n format: email\n description: 'Recipient email address'\n event:\n type: string\n description: 'Event type (processed, delivered, open, click, bounce)'\n enum: ['processed', 'delivered', 'open', 'click', 'bounce']\n timestamp:\n type: integer\n description: 'Unix timestamp of the event'\n responses:\n 200:\n description: 'Events tracked successfully'\n content:\n application/json:\n schema:\n type: object\n properties:\n message:\n type: string\n example: 'Events tracked successfully.'\n 400:\n description: 'Bad request - Missing or invalid fields'\n content:\n application/json:\n schema:\n type: object\n properties:\n error:\n type: string\n example: 'Invalid request body: missing required fields.'\n 500:\n description: 'Internal server error'\n content:\n application/json:\n schema:\n type: object\n properties:\n error:\n type: string\n example: 'Failed to process events.'\n```\n\n#### C. Workflow Function\n\n1. Create a file `track.ts` inside the functions folder.\n2. Copy this inside that file:\n\n```\nimport { GSContext, GSDataSource, GSStatus } from '@godspeedsystems/core';\n\nexport default async function (ctx: GSContext, args: any) {\n try {\n const {\n inputs: {\n data: { body },\n },\n } = ctx;\n\n ctx.childLogger.info('Received event data for tracking:', body);\n\n // Access the 'track' datasource from the context\n const ds: GSDataSource = ctx.datasources.track;\n\n const response = await ds.execute(ctx, body);\n\n ctx.childLogger.info('Event tracked successfully:', response);\n return new GSStatus(true, 200, 'Event tracked successfully');\n } catch (error: unknown) {\n // Log and handle known errors\n if (error instanceof Error) {\n ctx.childLogger.error(`Failed to track event: ${error.message}`);\n return new GSStatus(false, 500, 'Failed to track event', error.message);\n }\n\n // Log and handle unexpected error types\n ctx.childLogger.error(`Internal server error: ${error}`);\n return new GSStatus(false, 500, 'Internal server error', `${error}`);\n }\n}\n```\n\n## Running the Project\n\n1. Start the server:\n\n ```bash\n godspeed serve\n ```\n\n2. Access Swagger UI:\n\n ```\n http://localhost:3000/api-docs\n ```\n\n3. Test email sending:\n - Send a mail using the data format provided in swagger UI\n - Check the spread sheet, you should be able to see the logs being updated on it.\n - Open the email on email app to see the **open** log getting update\n - Send any link in the html section of mail body using anchor tag, and click on that link in the email app to see the **click** log getting updated.\n\n## Troubleshooting\n\nCommon issues and solutions:\n\n1. **Webhook not receiving events**\n\n - Verify ngrok is running and URL is updated in SendGrid\n - Check webhook logs in SendGrid dashboard\n\n2. **Google Sheets access denied**\n\n - Verify service account email is correctly shared\n - Check `google-key.json` permissions\n\n3. **Not able to send Emails**\n\n- Check spam folder\n- Verify sender email is authenticated\n- Check SendGrid API key permissions\n\n\n" }, "docs/microservices-framework/datasources/datasource-plugins/Text to Sql Datasource.md": { "content": "# Text-to-SQL Datasource Plugin\n\n**[Link to Plugin Source Code](https://github.com/godspeedsystems/gs-plugins/tree/main/plugins/text-to-sql-as-datasource)** \n\n## Overview\n\nA Godspeed plugin that enables natural language to SQL conversion and multi-database query execution with built-in caching support. This plugin leverages Gemini models to convert natural language queries into SQL statements and executes them across various database types.\n\n## Features\n\n- Natural language to SQL conversion using Google Gemini\n- Multi-database support\n - PostgreSQL\n - MySQL\n - MongoDB\n - Oracle\n- Automatic query caching with Redis\n- Query validation\n- Schema management\n- Error handling and logging\n- Connection pooling\n- Resource cleanup\n\n### 1. Install the Plugin\n\nFirst, add the Text-to-SQL plugin using\n\n```bash\ngodspeed plugin add @godspeedsystems/plugins-text-to-sql-as-datasource \n```\n\n\n\n### Setup Environment\nCreate or update your `.env` file with the following:\n\n```env\n# Gemini API Configuration\nGEMINI_API_KEY=your_gemini_api_key\n\n# PostgreSQL Configuration\nPG_USER=your_postgres_user\nPG_HOST=localhost\nPG_DB=your_database\nPG_PASSWORD=your_password\nPG_PORT=5432\n\n# MySQL Configuration\nMYSQL_HOST=localhost\nMYSQL_USER=your_mysql_user\nMYSQL_PASSWORD=your_password\nMYSQL_DB=your_database\n\n# MongoDB Configuration\nMONGODB_URL=mongodb://localhost:27017\nMONGODB_DB=your_database\n\n# Oracle Configuration\nORACLE_USER=your_oracle_user\nORACLE_PASSWORD=your_password\nORACLE_CONNECT_STRING=localhost:1521/XEPDB1\n\n# Redis Configuration\nREDIS_URL=redis://localhost:6379\n```\n\n### DataSource Configuration\nCheck your auto generated YAML configuration file `src/datasources/text-to-sql.yaml`:\n\n```yaml\ntype: text-to-sql\nconfig:\n gemini:\n apiKey: ${GEMINI_API_KEY}\n databases:\n postgres:\n enabled: true\n config:\n user: ${PG_USER}\n host: ${PG_HOST}\n database: ${PG_DB}\n password: ${PG_PASSWORD}\n port: ${PG_PORT}\n cache:\n enabled: true\n url: ${REDIS_URL}\n```\n\n### Define the Event (API Endpoint)\n\nCreate `src/events/sql-query.yaml`:\n```yaml\nhttp.post./sql-query:\n summary: \"Convert natural language to SQL and execute\"\n description: \"Endpoint to convert natural language to SQL query and execute it\"\n fn: execute-query\n authn: false\n body:\n content:\n application/json:\n schema:\n type: object\n properties:\n query:\n type: string\n description: \"Natural language query to be converted to SQL\"\n dbType:\n type: string\n description: \"Target database type (postgres, mysql, etc.)\"\n default: \"postgres\"\n required:\n - query\n responses:\n 200:\n description: \"Successful query execution\"\n content:\n application/json:\n schema:\n type: object\n```\n\n### 3. Write Event Handler Function\n\nCreate `src/functions/execute-query.ts`:\n\n```typescript\nimport { GSContext, PlainObject } from \"@godspeedsystems/core\";\n\nasync function handler(ctx: GSContext): Promise {\n try {\n const { body } = ctx.inputs.data;\n \n if (!body.query) {\n throw new Error(\"Query is required\");\n }\n\n // Execute the query using the text-to-sql datasource\n const result = await ctx.datasources['text-to-sql'].execute(ctx, {\n query: body.query,\n dbType: body.dbType || 'postgres',\n validateOnly: false,\n cache: true\n });\n\n return {\n success: true,\n data: result\n };\n\n } catch (error: any) {\n ctx.logger.error('Query execution failed:', error);\n return {\n success: false,\n code: 500,\n message: error.message,\n data: {\n message: \"Query execution failed\"\n }\n };\n }\n}\nexport default handler;\n```\n\n## Database Setup\n\n### PostgreSQL Container Setup\n```bash\n# Run PostgreSQL container\nsudo docker run --name test-postgres \\\n -e POSTGRES_USER=testuser \\\n -e POSTGRES_PASSWORD=testpass \\\n -e POSTGRES_DB=testdb \\\n -p 5434:5432 \\\n -d postgres:latest\n\n# Create test data\npsql -h localhost -p 5434 -U testuser -d testdb\n\nCREATE TABLE users (\n id SERIAL PRIMARY KEY,\n name VARCHAR(100),\n email VARCHAR(100),\n status VARCHAR(20)\n);\n\nINSERT INTO users (name, email, status) VALUES \n ('John Doe', 'john@example.com', 'active'),\n ('Jane Smith', 'jane@example.com', 'inactive');\n```\n\n### Redis Cache Setup\n```bash\n# Run Redis container\nsudo docker run --name test-redis \\\n -p 6380:6379 \\\n -d redis:latest\n```\n\n### Test API \nPOST `/sql-query`\n\n### Request Body\n```json\n{\n \"query\": \"find all active users\",\n \"dbType\": \"postgres\"\n}\n```\n\n### Response\n```json\n{\n \"success\": true,\n \"data\": {\n \"rows\": [\n {\n \"id\": 1,\n \"name\": \"John Doe\",\n \"email\": \"john@example.com\",\n \"status\": \"active\"\n }\n ],\n \"metadata\": {\n \"rowCount\": 1,\n \"executionTime\": \"0.123s\"\n }\n }\n}\n```\n\n## Key Benefits\n\n1. **Developer Productivity**\n - No need to write SQL queries manually\n - Rapid prototyping and development\n - Focus on business logic instead of query syntax\n\n2. **End User Experience**\n - Natural language interface to databases\n - Faster query results with caching\n - Reduced learning curve\n\n3. **Enterprise Ready**\n - Production-grade security\n - Scalable architecture\n - Comprehensive logging\n - Performance optimization\n\n## Advanced Examples\n\n### Complex Queries\n```json\n{\n \"query\": \"Show me total sales by category for active products in last quarter\",\n \"dbType\": \"postgres\"\n}\n```\n\n### Analytical Queries\n```json\n{\n \"query\": \"Calculate monthly growth rate of user signups\",\n \"dbType\": \"postgres\"\n}\n```\n\n## Security Features\n\n- Query validation before execution\n- SQL injection prevention\n- Rate limiting support\n- Error handling and sanitization\n- Secure database connections\n\n## Performance\n\n- Redis caching reduces database load\n- Query optimization suggestions\n- Connection pooling\n- Execution time monitoring\n- Cache hit ratio tracking\n\n## Example Usage\n\n### Basic Query Event Configuration\nCreate a new file `src/events/query.yaml`:\n\n```yaml\n'http.post./query':\n fn: execute-query\n body:\n content:\n application/json:\n schema:\n type: object\n properties:\n query:\n type: string\n dbType:\n type: string\n default: postgres\n required: ['query']\n```\n\n### Query Execution Workflow\nDefine your function `src/functions/execute-query.yaml`:\n\n```yaml\nid: execute-query\nsummary: Execute natural language query\ntasks:\n - id: convert-and-execute\n fn: datasource.text-to-sql.execute\n args:\n query: <% inputs.body.query %>\n dbType: <% inputs.body.dbType || 'postgres' %>\n```\n\n### Example Requests\n\n1. **Simple Query Request:**\n```bash\ncurl -X POST http://localhost:3000/query \\\n -H \"Content-Type: application/json\" \\\n -d '{\"query\": \"Find all active users\"}'\n```\n\n2. **Complex Query Request:**\n```bash\ncurl -X POST http://localhost:3000/query \\\n -H \"Content-Type: application/json\" \\\n -d '{\n \"query\": \"Show total sales by category for the last quarter\",\n \"dbType\": \"mysql\"\n }'\n```\n\n### Example Responses\n\n```json\n{\n \"sql\": \"SELECT category, SUM(amount) AS total_sales FROM sales WHERE date >= DATE_SUB(CURDATE(), INTERVAL 3 MONTH)\",\n \"data\": [\n {\n \"category\": \"Electronics\",\n \"total_sales\": 105000\n }\n ],\n \"metadata\": {\n \"rowCount\": 1,\n \"executionTime\": 48\n }\n}\n```\n\n## Error Handling\n\n```json\n{\n \"error\": \"QUERY_ERROR\",\n \"message\": \"Failed to execute query\",\n \"details\": \"Syntax error at or near 'total'\"\n}\n```\n## Limitations\n\n1. Natural language processing depends on Google Gemini\n2. Redis required for caching functionality\n3. Database-specific features may vary\n\n## Troubleshooting\n\n### Common Issues\n\n1. Connection Failures\n\n - Verify database credentials\n - Check network connectivity\n - Ensure services are running\n\n2. Query Generation Issues\n\n - Validate Google gemini API key\n - Check query format\n - Review database schema\n\n3. Cache Problems\n - Verify Redis connection\n - Check cache configuration\n - Monitor memory usage\n \n\n\n\n\n\n\n\n" }, "docs/microservices-framework/datasources/datasource-plugins/elasticgraph/elasticgraph.md": { "content": "---\nsidebar_position: 3\ntitle: Elasticgraph Datasource Plugin\n---\n\n**[Link to Plugin Source Code](https://github.com/godspeedsystems/gs-plugins/tree/main/plugins/elasticgraph-as-datasource)**\n\nThe framework supports Elasticgraph as a datasource. It supports elasticsearch as datastore. In addition, you can use various features of Elasticgraph like deep graph search algorithms, de-normalization, joins, aggregations, multi-lingual support.\n\n### Folder Structure\nThe datasources for Elasticgraph are defined in `src/datasources`. Here, `elasticgraph1.yaml` and `elasticgraph2.yaml` are defined in datasources.\n```\n.\n├── config\n└── src\n ├── datasources\n │   └── httpbin.yaml\n │   ├── elasticgraph1.yaml\n │   ├── elasticgraph2.yaml\n ├── events\n ├── functions\n └── mappings\n```\n\n### Datasource DSL\n**elasticgraph1.yaml**\n```yaml\n type: elasticgraph\n schema_backend: ./eg_config/eg1/ # relative path to config files\n deep: false # deep feature of Elasticgraph to use graph algorithms\n collect: true # collect feature of elasticsearch\n```\n**elasticgraph2.yaml**\n```yaml\ntype: elasticgraph\nschema_backend: ./eg_config/eg1/ # relative path to config files\ndeep: false # deep feature of Elasticgraph to use graph algorithms\ncollect: true # collect feature of elasticsearch\n```\n\n\n### Configuration files of Elasticgraph\nAll the configuration files of Elasticgraph datasources should be defined in `src/datasources/eg_config/` directory.\n\nSample strucutre of config files.\n```\n.\n├── elasticgraph1.yaml\n├── elasticgraph2.yaml\n├── eg1\n│ ├── collect.toml\n│ ├── common.toml\n│ ├── config.toml\n│ ├── custom.toml\n│ ├── elasticsearch.toml\n│ ├── joins\n│ │ └── search.txt\n│ └── schema\n│ ├── aggregation.toml\n│ ├── dependencies.toml\n│ ├── entities\n│ │ ├── reconciled.toml\n│ │ └── auth_user.toml\n│ ├── entitiesInfo.toml\n│ ├── relationships.txt\n│ ├── suggestions.toml\n│ └── union.toml\n└── eg2\n ├── collect.toml\n ├── common.toml\n ├── config.toml\n ├── custom.toml\n ├── elasticsearch.toml\n ├── joins\n │ └── search.txt\n └── schema\n ├── aggregation.toml\n ├── dependencies.toml\n ├── entities\n │ ├── reconciled.toml\n │ └── auth_user.toml\n ├── entitiesInfo.toml\n ├── relationships.txt\n ├── suggestions.toml\n └── union.toml\n```\n\n### Elasticgraph Setup\nThe framework has inbuilt feature of setting up Elasticgraph model automatically whenever a new configuration is added in `src/datasources/eg_config/` directory. In case, you are getting any error in the setup, then you can refer execute below step for manual setup:\n\n> During the project setup, if you have not selected elasticsearch, then you will have to execute `godspeed update` in project root directory, outside the dev container. This will add elasticsearch in the dev container environment.\n\n\n#### Step 1: godspeed eg-push\n```\n$ godspeed eg-push\n _ _\n __ _ ___ __| | ___ _ __ ___ ___ __| |\n / _` | / _ \\ / _` | / __| | '_ \\ / _ \\ / _ \\ / _` |\n | (_| | | (_) | | (_| | \\__ \\ | |_) | | __/ | __/ | (_| |\n \\__, | \\___/ \\__,_| |___/ | .__/ \\___| \\___| \\__,_|\n |___/ |_|\n\n> eg_test@1.0.0 eg-push\n> for f in src/datasources/eg_config/*; do echo ${f}; node ../gs_service/elasticgraph/lib/mappingGenerator/reIndexer.js ${f} all; done\n\nsrc/datasources/eg_config/eg1\n```\n\n## Auto generating CRUD APIs for Elasticgraph\nDeveloper can generate CRUD APIs for all the entities in `src/datasources/eg_config/` directory. `Events` and `Workflows` will be auto generated for `Create`, `Read`, `Update` and `Delete` operations for each entity in respective datastore.\n\n Auto-generated events and workflows will be stored in `/events/{datasourceName}/{entityName}` and `/functions/com/gs/eg/{datasourceName}/{entityName}` folders respectively.\n\n```\n$ godspeed gen-crud-api\n _ _\n __ _ ___ __| | ___ _ __ ___ ___ __| |\n / _` | / _ \\ / _` | / __| | '_ \\ / _ \\ / _ \\ / _` |\n | (_| | | (_) | | (_| | \\__ \\ | |_) | | __/ | __/ | (_| |\n \\__, | \\___/ \\__,_| |___/ | .__/ \\___| \\___| \\__,_|\n |___/ |_|\n\n> eg_test@1.0.0 gen-crud-api\n> npx godspeed-crud-api-generator\n\nSelect datasource / schema to generate CRUD APIs\nEvents and Workflows are generated for elasticgraph.yaml\n```\n\n# Elasticgraph setup\n\n## Creating the mapping in Elasticsearch for first time\n\nTo create the mapping for the first time, run the following command:\n\n```bash\nDEBUG=*,-elasticsearch node ../gs_service/elasticgraph/lib/mappingGenerator/reIndexer.js ./datasources/eg_config/eg1 all| init\n```\n:::tip\n If there are existing data indexed in Elasticsearch and we want to make changes to the mapping, such as adding new fields, it is not recommended to use the command used for creating the mapping for the first time\n:::\n\n## Reindexing after mapping updates\n\nIf we have made any changes to the mapping, such as adding new fields, we will need to reindex our data to apply the changes to the existing documents. To reindex in Elasticsearch,run the following command:\n\n```bash\n$ cd \nDEBUG=*,-elasticsearch node ../gs_service/elasticgraph/lib/mappingGenerator/reIndexer.js ./datasources/eg_config/eg1 backend all|\n\n```\n\n\n## Configuration: Switching to OpenSearch in Elasticgraph\n\nElasticGraph supports both Elasticsearch and OpenSearch as underlying data stores. By default, Elasticsearch is used. To configure ElasticGraph to use OpenSearch instead of Elasticsearch, add the following either in an environment variable or in the elasticsearch.toml file in your project's configuration:\n\n\n### Way 1: Add the following line to the .env file:\n\n```bash\n ds=aws\n```\n\n### Way 2: Add the following line to the elasticsearch.toml file:\n```\nsample_project\n└── config\n ├── backend\n    └── elasticsearch.toml\n```\n\n```yaml title=elasticsearch.toml\n\nmaxConnections = 200\napiVersion = '7.4'\nrequestTimeout = 90000\nnode = 'http://localhost:9200'\nsniffOnStart = true\nds = 'aws'\n```\n\n## Custom Elasticsearch Mapping\n\nIf you want to override the auto-generated, default Elasticsearch mapping, You can override that in `custom-mapping.yaml`.\n\n```yaml title=custom-mapping.yaml\nreconciled: #The type of entity\n mappings:\n dynamic_templates:\n - full_name:\n path_match: charge_params.*\n mapping:\n type: float\n properties:\n charge_params:\n properties:\n fee (Fee):\n type: float\n fee (Phí dịch vụ):\n type: float\n```\n\n\n\n### API examples: Postman collection\n\nDownload the collection with documentation [here](https://api.postman.com/collections/17317391-ae9724b5-10ac-428a-b0d2-e7d8127659c0?access_key=PMAT-01HCM2RCXZCR9H840HKZRBXBP7)\nThere you will see core CRUD API (same in sync and async). Each CRUD api has its documentation in the collection itself." }, "docs/microservices-framework/datasources/datasource-plugins/elasticgraph/feature-set-of-elasticgraph.md": { "content": "---\nsidebar_position: 3\ntitle: Feature, Configurations & API\n---\n\n# About ElasticGraph\n## Benefits\n\n### Productivity\nEG saves LOT of effort in development of an Elasticsearch based app because it *abstracts some most common data operations into a simple configurable abstractions* saving hundreds to thousand lines of code, makes the code neat, elegantly abstracts business logic in configurable text files and saves many hours of development and testing.\n\n### Performance\nUsing ElasticGraph one can configure and run (out of the box) highly scalable microservices using *relational graph approach, optimized for storing and querying large informational graphs in greater depth and complexity.*\n\nThe node module internally uses **intelligent query batching and in memory caching**, to support thousands of concurrent users, and complex search and analytic graph queries.\n\n## Features\n* Supports relationships\n* Joins\n\t* You can do this for upto multiple depths of relationships in an easy fashion.\n* Graph search and aggregations (using denormalization + caching)\n\t* Graph search: Find friends whose friends are from India.\n\t* Graph aggrigations: Give countwise breakup of users across city.state.country.name property\n* Dependency management between data of related entities (At indexing time).\n\t* Sometimes the value of some field of an entity depends on the value of another field in (possibly) another entity related to it.\n\t* Example unions and copy values. More on it below.\n\t* Here you can configure such dependencies through configuration.\n* Multi lingual storage and read operations\n * You can store, retrieve and search text fields in any languages\n\t* Any new language support can be added though a simple configuration\n* Easy SQL\n\t* English like (or easy) query language is a custom DSL built to richly express queries in a very small and handy syntax, easily human readable.\n\t* Write complex database operation logic in \"very few lines\"\n\t* You (almost) don't need to be a developer to grasp it :-)\n* Performance features\n\t* Query batching\n\t\t* EG collects and executes, multiple queries from different places in your application logic, as one bulk query to your datastores.\n\t\t* Saves typical throughput time in high load scenario\n\t* In memory caching\n\t\t* Check and store queries and entities in an in-memory cache. This cache is internally populated and used during execution of EG’s deep API.\n\t\t* Saves N -1 round trips to the database for every N queries.\n\t\t* Developers can use this feature via EG’s npm module, to keep alive and share a cache object as long as they want for performance optimisation.\n\n\n## Entities and model\n\nIn ElasticGraph (EG) domain, there are entities (similar to rows in MySQL, or nodes in a Graph). Each entity has an id, type, fields like text, date etc. and relationships (akin to foreign keys). The configurations of a field are all declared in one place.\n\nEach entity is stored in a separate ElasticSearch index by the name entity._type + ‘s’\n\nFor example. for type video, the ES index will be videos\n\n\n**The simple fields of an entity and their settings are defined in**\n\nHere is a sample config for 'event' entity type in TOML format\n\n```yaml title=configFolder/schema/entities/{entityType}.toml\n[title]\ntype = 'String'\nmultiLingual = true\nautoSuggestion = true\nencrypted = true\nsort = true # Mark this field to be index as sortable or searchable\n[description]\ntype = 'String' # String | Boolean | Number(stored as Long) | Object | Date\nmultiLingual = true\n[startingDate]\ntype = 'date'\nmultiLingual = false\n```\n:::info\nIf you want to use `type = Float` then you need to specify in [custom-mappings.yaml](elasticgraph.md#custom-elasticsearch-mapping)\n\n:::\n\nCorresponding document of an Event, when returned from the API will look like shown below. It does not matter which underlying database the information is being fetched from.\n\n```json\n{\n \"_index\": \"events\",\n \"_type\": \"event\",\n \"_id\": \"294464\",\n \"_version\": 4,\n \"found\": true,\n \"_source\": {\n \"startingDate\": 489004200000,\n \"tibetan\": {\n \"description\": \"\\nལ་དཱགས་མི་མང་ནས་ཇི་ལྟར་གསོལ་བ་འདེབས་པ་བཞིན་༧སྤྱི་ནོར་༧གོང་ས་སྐྱབས་མགོན་\\n\\nཆེན་པོ་མཆོག་ནས་ནང་ཆོས་ངོ་སྤྲོད་སྩལ་།\",\n \"title\": \"Sample event\"\n },\n \"english\": {\n \"description\": \"His Holiness the Fourteenth Dalai Lama gives an introduction on basic Tibetan Buddhism in Ladakh.\",\n \"title\": \"པོ་མཆོག་ནས་ནང་ཆོས་ངོ་སྤྲོད་སྩལ\"\n }\n }\n}\n```\n\n\nThe data model you set is used to generate the appropriate mappings for ES. This way, EG can be used to automatically create schema in ES.\n\n### Field Encryption and Search in ElasticGraph\n\nProtecting sensitive data is crucial in any application. ElasticGraph offers a powerful feature that allows encryption of specific fields mentioned in the TOML file of the schema. This ensures the confidentiality and integrity of sensitive information stored in your database.\n\nFurthermore, ElasticGraph enables search functionality on encrypted fields in their plaintext form. To achieve this, ElasticGraph utilizes the robust SHA-256 algorithm for deterministic encryption.\n\nFor example, if you want to encrypt a mobile number field, you can easily achieve this by simply adding the line `encrypted = true` in the corresponding TOML file.\n\n```yaml\n[mobileNumber]\ntype =\"String\"\nsort = true\nencrypted = true\n```\n\n## Relationships\n\nYou must define the relationships of your data model in `configFolder/schema/relationships.txt`.\n\nThe format for specifying relationships in relationship file is\n\n```text\nrelationNameFromAToB <> relationNameFromBToA\nentityTypeA <> entityTypeB //One to one\nrelationNameFromAToB <> relationNameFromBToA\n[entityTypeA] <> entityTypeB //Many to one\nrelationNameFromAToBs <> relationNameFromBToA\nentityTypeA <> [entityTypeB] //One to many\nrelationNameFromAsToBs <> relationNameFromBsToAs\n[entityTypeA] <> [entityTypeB] //many to many\n```\n*As you can see, when an entity type is surrounded by square brackets [], it means cardinality of many*\n\n:::tip\nIt is compulsory to maintain relationship name both ways, from Entity A to B, and B to A.* This is so that one can express Graph traversal from both sides.\n:::\n\n\nSome examples\n```\nspeakers <> events\n[event] <> [speaker]\nsessions <> event\nevent <> [session]\n```\n\n### Linking entities via relation\n\n```js\n\tes.deep.link({\n\t\te1: {\n\t\t\t_type: ‘event’,\n\t\t\t_id: ‘674’\n\t\t},\n\t\te2: {\n\t\t\t_type: session,\n\t\t\t_id: 4\n\t\t},\n\t\te1ToE2Relation: ‘sessions’\n\t})\n\t.then(console.log)\n```\n\n### Un-linking entities via relation\n\n```js\n\tes.deep.unlink({\n\t\te1: {\n\t\t\t_type: ‘event’,\n\t\t\t_id: ‘674’\n\t\t},\n\t\te2: {\n\t\t\t_type: session,\n\t\t\t_id: 4\n\t\t},\n\t\te1ToE2Relation: ‘sessions’\n\t})\n\t.then(console.log)\n```\n\n## Graph Search and Graph analytics\n\n**We use denormalisation to make it fast**\n> Settings configFolder/joins/index.txt\n\nImagine you have a database composed of events, speakers and persons.\n\nAnd, you wish to do the following two queries.\n* Search events by speakers.person.name\n* Show countwise breakup of search results on events, based on speakers.person.name (like on ecommerce sites)\n\n> If your tables have only the foreign keys, you will have to do multiple hits to implement such cross table queries. And they will be slow. Depending on your data size, this may take a long long time before the final query result is returned. Also, your database will most probably get under heavy load.\n\n*With ElasticGraph you can denormalize based on simple rule setting and achieve the same result with a single hit to the database. *\n\nBy denormalizing (always ensuring latest copy of) the speaker.person.name information within the event object, *during index, update, link or unlink calls*.\n\n\n\nHow does this work?\n\nBy denormalizing (always ensuring latest copy of) the speaker.person.name information within the event object, *during index, update, link or unlink calls*.\n\nFor example, here is how ‘event’ may look like in denormalization settings (in the file joins/index.txt)\n\n```yaml\n[event]\nsessions{title, description}\nspeakers.person{name}\n```\n\nBased on your configuration ElasticGraph works to automatically maintain the denormalised storage of speaker and session data in the event entities. Y\n** You only need to link or unlink two entities by a relationship. Everything else is taken care by ElasticGraph. **\n\n#### Maintenance of the denormalised graph state\n\nHere are some scenarios in which the automatic denormalization will trigger in our example database.\n\n* Whenever you update the name of a person, the events where he or she spoke, will also get updated with person’s new name.\n\n* When you index (store) the event for first time in the database, and it contains speakers ids, the speaker’s name will also get copied inside the event entity as it gets stored/indexed.\n\n* When the event is linked to a speaker, the speaker’s name will get copied inside the event entity\n\n* When the event is unlinked from a speaker, the speaker’s id, name etc will get removed from the event entity\n\n\n\n#### The Butterfly effect\n\nAs you just saw, any update can potentially create a ripple update across entire Graph, for maintaining correct data state as per the denormalisation and also the data dependency rules like union and copy (more on the latter below).\n\nSince this is handled internally by ElasticGraph, it saves the developer from the overhead of maintaining a consistent, denormalised graph state across all updates. His code doesn’t need to save the updated field value at multiple places in the database- a big overhead, lots of confusing code, more bugs... Instead, he simply declares the behavior just once, in a human readable way. After that he leaves it to ElasticGraph to do all the internal bookkeeping to upkeep a correct denormalised graph state all the time.\n\nIn ElasticSearch and also in popular SQL stores, we can make use of the JSON style storage and do the joins within one document. In comparison to SQL way of rows, the document way of ES saves storage space and helps in faster analytics also. Have a look at how the denormalized speakers relationship is stored within an ElasticGraph event document.\n\n```json\n{\n \"_index\": \"events\",\n \"_type\": \"event\",\n \"_id\": \"294464\",\n \"_version\": 4,\n \"found\": true,\n \"_source\": {\n \"speakers\": [\n {\n \"_id\": \"c6c35e3b21815a4209054505ac5e1680a954efdf\",\n \"own\": true,\n \"fields\": {\n \"person\": {\n \"_id\": \"1\",\n \"_version\": 1,\n \"fields\": {\n \"english\": {\n \"name\": \"His Holiness the 14th Dalai Lama\"\n },\n \"tibetan\": {\n\t\t\t\t \"name\" : \"ང་ས་སྐུ་ཕྲེང་བཅུ་བཞི་\"\n\t\t\t }\n }\n }\n }\n }\n ]\n }\n}\n```\n\n## Data dependency implementation\n\n> Note: This strategy is perhaps best applied in write less and read more scenarios.\n\nIn many data models, data of an entity in your graph may depend on the data of other related entities. For ex. if a married woman has a new child, the husband also has a new child. And vice versa.\n\nElasticGraph gives you an easy way to manage complex data dependencies between related entities of your information graph. As any update is made to any Entity in your Graph, ElasticGraph checks if any part of the remaining Graph should be updated by this change as per your data model settings. If yes, it updates the entire affected Graph (Butterfly effect).\n\nFor now EG supports two kinds of dependencies - Union from and Copy.\n\n* **Union from**\n> Settings are in configFolder/schema/union.toml\n\nUnion from operation can be used to compute and store distinct values, whether relationships or data values, merged from field values of multiple related entities.\n\nThis is useful for one to many or many to many relationships. Please look at the following examples to understand.\n\n```yaml\n[conference]\nspeakers = '+talks.speaker' #As soon as a talk is linked to a conferece, or an already linked talk gets linked to a speaker, *the talk’s speaker is also linked to the conference as one of its speakers, if not already linked before*. Vice versa happens if the talk is unlinked to its speaker, or the talk is removed from the conference\ntopics = '+talks.topics' #As soon as a talk is linked to an conference, or a topic is set to an already linked talk, the talk’s topic is also added to the conference as one of its topics, if not already there. Vice versa happens if the talk is unliked to the conference, or the topic is removed from the talk.\n[‘person’]\ngrandChildren = +‘children.children’ #Whenever a person’s child gets a new child, the new child gets added to the person’s grandchildren\n[‘folder’]\nfileTypes = ‘+childFolders.fileTypes +childFiles.type’ #Calculate union of all file types existing in the entire folder tree (recursively). Anytime, any file gets added to any child folder in this tree, the type of that file gets unioned with the list of fileTypes of that child folder, and all its parent folders up in the hierarchy.\n```\n\n* **Copy**\n> Settings are in configFolder/schema/union.toml\n\nCurrently the copy functionality is achieved from within the union configuration.\n\nThis is effective for many to one or one to one relations. For ex.\n\n```yaml\n[person]\nchild = \"+wife.child +husband.child\" #This will ensure copy of child between husband and wife, whenever child is added to any one of the person entities\n[file]\npermissions = \"+folder.permissions\" #Whenever a folder’s permissions are updated the underlying files’ permissions are updated automatically. You can still manually override them, without affecting the folder. But whenever the folder’s permissions are updated again, the file’s permissions will get overwritten.\n```\n\n## **Read time joins**\n\nThis is helpful to create multiple views on the fly, during read time joins\n\nTwo ways to specify read time joins:\n\n* **Approach A:** Create a file in joins folder, and send the name of the file in the JSON query.\n\t> Settings folder: configFolder/joins\n\t* For read time joins, you specify name of a join configuration file stored in configFolder/joins. You can specify different joins for same entity in different contexts like read, search etc. The particular view can be referred by the `${filename}` in your code.\n\t* Ex. read.txt or search.txt. You can create multiple such files and refer them\n\t* Sample configuration in text file (Same as denormalization settings in joins/index.txt)\n\t```\n\t\t[event]\n\t\tsessions{title, description}\n\t\tspeakers.person{name}\n\t\tspeakers.primaryLanguages{name}\n\t```\n\n* **Approach B:** Send the view (join) info in the query as JSON object\n\n\t* This gives developer the flexibility to create any views on the fly.\n\n\t* Example joins for user who lives in a city belonging to a state\n\n\t```json\n\t\t{\n\t\t\t\"joins\": {\n\t\t\t\"name\": 1,\n\t\t\t\"city.name\": 1,\n\t\t\t\"city.state.name\": 1,\n\t\t\t\"city\": { // Same effect as above two lines\n\t\t\t\t\"name\": 1,\n\t\t\t\t\"state\": {\n\t\t\t\t\t\"name\": 1\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t\t}\n```\n\nSample API calls\n\n```js\n\tdeep.get({_id:1, _type: ‘event’ , joins: ‘read’});\n\tdeep.get({\n\t\t_id:1,\n\t\t_type: ‘event’ , joins: {\n\t\t\"name\": 1,\n\t\t\"city.name\": 1,\n\t\t\"city.state.name\": 1\n\t\t}}\n\t);\n\tdeep.search({\n\t\t_id:1,\n\t\t_type: ‘event’ ,\n\t\tquery: {\"match\": {“speakers.person.english.name”: “Dalai Lama”}},\n\t\tjoins: ‘search’\n\t})\n```\n\nThe joined response is returned in same structure as the denormalization join you saw just above. *You can apply joins across any relation depth.*\n\nFor read time joins, you specify name of a join configuration file stored in configFolder/joins. You can specify different joins for same entity in different contexts like read, search etc. The joined response is returned in same structure as the denormalization join you saw just above. *You can apply joins across any relation depth.*\n\n## **Multi Linguality**\n> Settings file: configFolder/common.toml.\n\nIn that set, `supportedLanguages = [‘english’ , ‘tibetan’, ‘thirdLanguage’]`\n\nIf your data is in a single language or is language agnostic, then supportedLanguages = []\n\nThe fields which are declared multilingual, are stored like this in the _source of the entities.\n\n~~~\n\"english\": {\n\t\"name\": \"His Holiness the 14th Dalai Lama\"\n},\n\"tibetan\": {\n\t\"name\": \"ྋགོང་ས་སྐུ་ཕྲེང་བཅུ་བཞི་པ།\"\n}\n~~~\n\n\n\n\nWhen creating, updating, searching or getting an entity, you have to specify the full path of every field, including its language. In search and get calls, you specify langs parameter, for the languages in which the data is to be fetched. By default data in all supported languages is fetched.\n\n\n\n\n\n## **Easy SQL**\n\n\n\n\n\nEnglish like SQL to get lot of data work done - fast and easy. Even non-programmers can easily learn to do complex work over big data using this.\n\n\n\n\n\nOne can use ESQL for working with EG entities or even pure ES indices.\n\n\n\n\n\nIts main features are\n\n\n\n* Get much done with very less lines of code.\n\n* Much elegant way compared to equivalent Javascript code.\n\n\n\n\n\nIt supports\n\n\n\n\n\n* Search, get, index, link, unlink, delete.\n\n\n\n\n\n* Creation of variables and assignment of values. Numbers string, boolean, objects supported\n\n\n\n\n\n* If/else operations\n\n\n\n* break, continue\n\n* Print log\n\n\n\n* Looping over array - Async each parallel\n\n\n\n* Useful for scanning over a search result or entire index and doing operations.\n\n* Loops can be nested within each other\n\n* Mixing pure JS functions as instructions of the script when the script can not handle the complexity of logic\n\n\n\n\n\nThe grammar of dsl engine is in the source code of ElasticGraph npm. lib/dslEngine/grammar.pegjs\n\n\n\nTo run the a single statement in EQL you call eg.dsl.execute(statement).\n\nTo execute a whole script you call eg.dsl.runScripts(script)\n\n```js\nconst fillSpeakersTranslatorsAndLinkWithEvent = [\n\t'iterate over old-contents where {$exist: event_id} as old-content. Get 25 at a time. Flush every 5 cycles. Wait for 100 millis',\n\t[\n\t\t'get event *old-content.event_id',\n\t\t'if *event is empty, display \"empty event\", *old-content.event_id',\n\t\t'if *event is empty, stop here',\n\t\t'search old-content-to-audio-channel where {content_id: *old-content._id} as cac',\n\t\t'async each *cac.hits.hits as old-content-to-audio-channel'\n\t\t[\n\t\t\t'get old-audio-channel *old-content-to-audio-channel.audiochannel_id as old-audio-channel', //No need to mention _source or fields. Both places, including top level object will be checked for existence of audiochannelId field\n\t\t\t'search first person where {_id: *old-audio-channel.speaker_id} as person.', //Creates person in index if not found there. Also sets person entity viz id+type as key in ctx.data, query as key with value being result in ctx.data\n\t\t\t//Handle event.speakers/translators. This guy is either a speaker or a translator. Set the relevent linking\n\t\t\t//Initializer\n\t\t\t'roleType is speaker if *old-audio-channel.translation_type is empty. Else is translator',\n\t\t\t'roleFeatures are {person._id: *old-audio-channel.speaker_id, primaryLanguages._id: *old-audio-channel.language_id}',\n\t\t\t//Can include pure JS functions within the script also\n\t\t\t(ctx) => {\n\t\t\t\t//TODO fix this 'roleFeatures.translationType is *old-audio-channel.translation_type if *roleType is translator.',\n\t\t\t\tif (ctx.get('roleType') === 'translator') {\n\t\t\t\tconst translationType = ctx.get('old-audio-channel')._source.translation_type\n\t\t\t\tctx.get('roleFeatures').translationType = translationType\n\t\t\t\t}\n\t\t\t\treturn ctx\n\t\t\t},\n\t\t\t'search first *roleType where *roleFeatures as speakerOrTranslator. Create if not exists.',\n\t\t\t'if *speakerOrTranslator is empty, display \"empty speaker\", *roleFeatures, *roleType',\n\t\t\t'if *speakerOrTranslator is empty, stop here',\n\t\t\t(ctx) => {\n\t\t\t\tconst speaker = ctx.get('speakerOrTranslator')\n\t\t\t\tconst speakerBody = speaker._source || speaker.fields\n\t\t\t\tconst pmName = _.get(_.first(speakerBody.primaryLanguages), 'fields.english.name')\n\t\t\t\tif (_.isObject(pmName)) {\n\t\t\t\t\tdebug('throw stopHere error to break the loop', JSON.stringify(speaker))\n\t\t\t\t\tthrow new Error('stopHere')\n\t\t\t\t}\n\t\t\t},\n\t\t\t//'display *roleType, *speakerOrTranslator._id, *roleFeatures',\n\t\t\t'link *speakerOrTranslator with *event as events',\n\t\t],\n\t],\n\t(ctx) => debug('Done ' + n++ + ' iterations')\n];\n//Now run the script\neg.dsl.execute(fillSpeakersTranslatorsAndLinkWithEvent);\n```\n\n## **Performance features**\n\n\n\n\n\nThere are two internal feature which stand behind the awesome performance of ElasticGraph - Collect and Cache.\n\n\n\n\n\n### Collect\n\n\n\n\n\nA typical program, during runtime, sends multiple queries to the database from different places. In case of using ES from NodeJS, each query entails an HTTP hit. Each such hit is an overhead on the system. Both to the Nodejs client and the ES cluster.\n\n\n\n\n\nThis feature allows you to save this overhead to achieve greater system speed and performance. Using this you can collect multiple queries and when a specified timeout or batch size threshold is reached, you send them to ES *as a single bulk request*. You can collect multiple queries from any parts of your middle ware.\n```js\n es.{methodName}.collect({methodParams})\n```\n> Sample settings in configFolder/collect.toml\n\n```yaml\n[batchSizes]\nmsearch = 200\nindex = 200\nmget = 200\nget = 200\nsearch = 200\nbulk = 200\n[timeouts] #in milliseconds\nindex = 30\nget = 30\nbulk = 30\nmget = 30\nmsearch = 30\nsearch = 30\n```\n\n\n*Each type of query is collected in a batch till any one of the batchSize threshold or the timeout threshold is reached.*\n\nThe supported es methods are get, mget, search, msearch, bulk and index.\nFor ex. \n```\nes.get.collect({_id:..,_type:..}).then()\n```\n\n\n*The deep functions and esql scripts of EG internally use this feature.* This feature is available as part of the npm module.\n\n\n### Cache\n\nIn the deep EG operations, a cache is used like a temporary EG index in memory. Hit to ES for each get/search query is done only once. After that each retrieved entity or document, and search result, is kept in the in memory store. Further, the graph update operations are also done in memory. Once the time to flush the updated graph to ES has come, one can call cache.flush()\nAll the in-memory-updated entities will be written to ES indices, and all cache data will be cleared.\n\n## **Limitations**\n\nCurrently it does not support transactions or authorization (as of today)\n\nElasticSearch also does not provide transactions or acidity. In EG, since a single update also updates rest of the graph, but first in memory, and then altogether flushed into ElasticSearch, it is possible that another process may have updated a part of updated graph in meantime. If so flushing of this subgraph update will throw an error because someone already updated part of the subgraph before. This will lead to a partial subgraph update.\n\nWhen using EG for denormalisation and dependency management, one has to be OK with possible errors in maintenance of the graph state.* If you need strict ACID behavior in your application, its best to use a transactional database as your primary datastore and use ES as your secondary datastore for read/search/analytic queries at scale and speed.*\n> Soon EG, will provide both kind of data store support out of the box.\n\n## **Deep API**\n\nYou can find the API and docs in the CRUD folder of the Postman collection shared [here](https://app.swaggerhub.com/apis/M4195/ElasticGraph/1.0.0)\n\n> A full API doc shall be made soon.\n\n## **Summing it up**\n\nThis project started with the .collect() feature sometime in 2015, from there it has evolved to include the deep API, denormalization, esql and other features. And now it is expanding to become a very powerful full fledged Microservice Platform. We have catered to four clients so far, and also built our own admin panel using the same.\n\nBuilt with deep thought from the Himalayas." }, "docs/microservices-framework/datasources/list-of-plugins.md": { "content": "# List of Plugins\n\nGodspeed framework adopts a pluggable approach that empowers you to define data sources effortlessly. Our framework graciously provides an interface that caters to diverse data source needs. Here's a glimpse into the exceptional datasource plugins crafted by our core framework team.\n\nTo seamlessly integrate these plugins into your project, simply run the command:\n\n```bash\n$ godspeed plugin add\n```\n```bash\n\n\n ,_, ╔════════════════════════════════════╗\n (o,o) ║ Welcome to Godspeed ║\n ({___}) ║ World's First Meta Framework ║\n \" \" ╚════════════════════════════════════╝\n\n\n? Please select godspeed plugin to install: (Press to select, to move rows)\n┌──────┬────────────────────────────────────────┬────────────────────────────────────────────────────────────────────────────────┐\n│ │ Name │ Description │\n├──────┼────────────────────────────────────────┼────────────────────────────────────────────────────────────────────────────────┤\n│ ❯◯ │ cron-as-eventsource │ Cron as eventsource plugin for Godspeed Framework │\n├──────┼────────────────────────────────────────┼────────────────────────────────────────────────────────────────────────────────┤\n│ ◯ │ aws-as-datasource │ aws as datasource plugin for Godspeed Framework │\n├──────┼────────────────────────────────────────┼────────────────────────────────────────────────────────────────────────────────┤\n│ ◯ │ excel-as-datasource │ excel as datasource plugin for Godspeed Framework │\n├──────┼────────────────────────────────────────┼────────────────────────────────────────────────────────────────────────────────┤\n│ ◯ │ mangoose-as-datasource │ mongoose-as-datasource as datasource plugin for Godspeed Framework │\n├──────┼────────────────────────────────────────┼────────────────────────────────────────────────────────────────────────────────┤\n│ ◯ │ mailer-as-datasource │ mailer as datasource plugin for Godspeed Framework │\n├──────┼────────────────────────────────────────┼────────────────────────────────────────────────────────────────────────────────┤\n│ ◯ │ kafka-as-datasource-as-eventsource │ kafka as datasource-as-eventsource plugin for Godspeed Framework │\n└──────┴────────────────────────────────────────┴────────────────────────────────────────────────────────────────────────────────┘\n\n```\n\nyou can specify plugin name to add directly to your project\n\n```sh\ngodspeed plugin add \n```\n## List of Datasource plugins\n\n### 1. [prisma-as-datasource](https://www.npmjs.com/package/@godspeedsystems/plugins-prisma-as-datastore)\n\nPrisma-as-datasource plugin provide functionality to access most popular databases like, PostgreSQL, MySQL, SQL Server, SQLite, MongoDB, CockroachDB, Planetscale and MariaDB through Prisma ORM.\n\n\n### 2. [axios-as-datasource](https://www.npmjs.com/package/@godspeedsystems/plugins-axios-as-datasource)\n\nAxios as a datasource: Level up your data-fetching game with Axios. Seamlessly integrate this powerful HTTP client into your app for smooth and efficient data transactions. Fetch, post, and interact with APIs effortlessly. Ready to make data requests a breeze? 🌐✨\n\n\n\n## How to Use\n- Create a godspeed project from the CLI and by default the axios plugin is integrated into your project if not, add the plugin from the CLI and select the `@godspeedsystems/plugins-axios-as-datasource` to integrate the plugin.\n\n```\ngodspeed plugin add \n\n\n ,_, ╔════════════════════════════════════╗\n (o,o) ║ Welcome to Godspeed ║\n ({___}) ║ World's First Meta Framework ║\n \" \" ╚════════════════════════════════════╝\n\n\n? Please select godspeed plugin to install: (Press to select, to move rows)\n┌──────┬───────────────────────────────────┬──────────────────────────────────────────────────────────────────┐\n│ │ Name │ Description │\n├──────┼───────────────────────────────────┼──────────────────────────────────────────────────────────────────┤\n│ ◯ │ kafka-as-datasource-as-eventsource│ kafka as datasource-as-eventsource plugin for Godspeed Framework │\n├──────┼───────────────────────────────────┼──────────────────────────────────────────────────────────────────┤\n│ ❯◯ │ axios-as-datasource │ Axios as datasource plugin for Godspeed Framework │\n└──────┴───────────────────────────────────┴──────────────────────────────────────────────────────────────────┘\n\n```\n\n- The plugin can also be directly installed by running `npm i @godspeedsystems/axios-as-datasource` command\n\n- You will find the files in your project related to the axios plugin at `src/datasources/types/axios.ts` and `src/datasources/api.yaml`.\n\n### axios.ts (src/datasources/types/axios.ts)\n\n```typescript\nimport { DataSource } from '@godspeedsystems/plugins-axios-as-datasource';\nexport default DataSource;\n```\n\n### axios config (src/datasources/api.yaml)\n\n```yaml\ntype: axios\nbase_url: http://localhost:4000\n```\n\n### Axios Workflow (src/functions/sample.yaml)\n```\nid: sample\ntasks:\n - id: first_task\n fn: datasource.api.get./api/items\n args:\n headers:\n data:\n timeout:\n params:\n```\nThe axios request configuration options, such as headers, params, data, and timeout, can be directly passed as arguments (args).\n\n```\nargs:\n headers:\n 'X-Requested-With': 'XMLHttpRequest'\n params:\n ID: 12345\n data:\n firstName: 'Fred'\n timeout: 1000\n```\n To get more clarity checkout about [Axios configuration]( https://axios-http.com/docs/req_config)\n\n\n## How It Helps\n\nThe Godspeed Axios Plugin offers the following advantages:\n\n1. **Axios Integration:** The plugin abstracts away the complexities of setting up Axios instances, making it effortless to configure and execute HTTP requests.\n\n2. **Unified DataSource:** Developers can use a uniform API to define data sources that make HTTP requests using Axios. This enhances consistency and ease of use across different parts of the application.\n\n3. **Error Handling:** The plugin includes robust error handling, allowing developers to gracefully handle various scenarios, such as server timeouts, request setup failures, and server-side errors.\n\n4. **Integration with Godspeed Core:** The plugin seamlessly integrates with the Godspeed Core library, aligning with the principles of the Godspeed framework and enabling streamlined event-driven workflows.\n\n\n## Plugin Components\n\nThe plugin consists of the following key components:\n\n### 1. `DataSource` Class\n\n- This class extends `GSDataSource`, a base class provided by the Godspeed framework for creating data sources.\n\n- It initializes an Axios instance to make HTTP requests based on the provided configuration options.\n\n- The `execute` method is used to define how the plugin should execute HTTP requests. It maps incoming parameters to Axios request properties, processes the request, and handles various response scenarios.\n\n### 2. Constants\n\n- `SourceType`: A constant representing the source type of the plugin, which is 'DS' (data source).\n\n- `Type`: A constant representing the loader file of the plugin. The final loader file will be located in the 'types' directory and named `${Type.js}`, where `Type` is 'axios' in this case.\n\n- `CONFIG_FILE_NAME`: In the context of a data source, this constant also serves as the data source name. In this plugin, it is set to 'api'.\n\n- `DEFAULT_CONFIG`: A default configuration object with Axios options like base URL and other settings.\n\n\n### Axios retry\n- Defaults set retry at datasource level within datasource config yaml file.(src/datasources/api.yaml)\n\n```yaml\ntype: axios\nbase_url: http://localhost:4000\nretry:\n when: #the condition\n status: [500, 501, 502] # an array or single value of codes\n message: my custom expected message for retry\n max_attempts: 5\n type: constant ##[constant,exponential,random]\n interval: PT15s\n```\n\nthe above config works on two conditions if status from the api is 500,501 or 502 and message value is as mentioned in the config. When condition is optional and if retry is without when condition, the retry will be made on failures of the API.\n\n- Override at task level within args object of the axios method call.\n\n```yaml\nid: some_workflow\ntasks:\n - id: post-anything\n # Fetching loan offers from rule engine for the given bank and pan card\n fn: datasource.api_datasource.post./anything\n args:\n data:\n data: <%inputs.body.data%>\n on_error:\n continue: false\n retry: # By default the datasource has constant retry set in its yaml. Here we override the retry to exponential\n when: # an and condition between status and message.\n status: [500, 503] # an array or single value of codes (optional). Default 500\n message: Retry later # Retry when response has this message\n max_attempts: 5\n type: exponential\n min_interval: PT5s\n max_internal: PT15s\n```\n\n\n\nThe sample config can be modified as per the usecase of your application.\n\n#### sample config `api.yaml`\n```yaml\ntype: axios\nbase_url: https://httpbin.org\n\n# print all api calls in curl format\ncurlifiedLogs: true \n\n# Authentication of API calls with token refresh logic\nauthn: \n fn: my_bank.authn\n refreshOn:\n statusCode: [401]\n\n# Common headers to be set in all API calls\nheaders:\n Content-Type: application/json\n Cookie: <%mappings.my_bank.auth_workflow_cookie%>\n\n# Retry logic for failed API calls for ex on Internal server errors or request timeouts\nretry:\n when: #the condition\n status: [500, 503] # an array or single value of codes (optional). Default 500\n message: my custom expected message for retry #And (optionally) when response has this message\n max_attempts: 5\n type: constant # or random, exponential\n interval: PT15s\n # type: exponential\n # min_interval: PT5s\n # max_internal: PT15s\n # type: random\n # min_interval: PT5s\n # max_internal: PT15s\n```\n\n### Authentication \n#### API calls with token refresh logic and authentication can also be configured in your datasource config file, by setting `authn` and the `fn` is called before calling the API endpoint and token will be refreshed on statusCode mentioned in the array of [`statusCode`](/docs/microservices-framework/datasources/list-of-plugins#sample-config-apiyaml).\n\n\nexample `fn` of `authn`\n```ts\n\nimport { logger } from \"@godspeedsystems/core\";\n\nconst axios = require('axios');\nconst client = axios.create({\n headers: {\n 'Content-Type': 'application/json'\n }\n});\n/**\n * Generate and return all the headers which are required to be sent\n * in the API calls which require authentication tokens\n */\nmodule.exports = async function (ctx: any) {\n try {\n const res = await client({\n method: 'get',\n url: `https://httpbin.org/anything`,\n data: {\n \"Authorization\": 'access_token'\n }\n })\n // Retrieve the authn tokens\n const headers = {\n \"Authorization\": res.data.access_token || 'access_token'\n };\n \n logger.info('Auth token successfully refreshed and following headers set: %o', Object.keys(headers));\n return headers;\n } catch (error) {\n logger.error('Error in refreshing token %o', error);\n throw error;\n }\n}\n```\n\n## Conclusion\n\nThe Godspeed Axios Plugin is a valuable addition to the Godspeed framework, providing a standardized way to make HTTP requests using the Axios library. With this plugin, you can easily integrate with external APIs, handle responses, and streamline data retrieval within your applications.\n\n\n### 3. [aws-as-datasource](https://github.com/godspeedsystems/gs-plugins/blob/main/plugins/aws-as-datasource/README.md)\n\nAWS as a datasource plugin: Turbocharge your app by tapping into Amazon Web Services. Unleash the power of cloud-based data, storage, and more to supercharge your application. 🚀 \n\n## Steps to use aws plug-in in godspeed framework:\n\n### How to install\n- Create a godspeed project from the CLI , open the created project in vscode and then add the plugin from the CLI of vscode, select the `@godspeedsystems/plugins-aws-as-datasource` to integrate the plugin.\n\n```\n> godspeed plugin add\n ,_, ╔════════════════════════════════════╗\n (o,o) ║ Welcome to Godspeed ║\n ({___}) ║ World's First Meta Framework ║\n \" \" ╚════════════════════════════════════╝\n? Please select godspeed plugin to install: (Press to select, to move rows)\n┌──────┬────────────────────────────────────┬────────────────────────────────────────────────────────────────────┐\n│ │ Name │ Description │\n├──────┼────────────────────────────────────┼────────────────────────────────────────────────────────────────────┤\n│ ◯ │ prisma-as-datastore │ Prisma as a datasource plugin for Godspeed Framework. │\n├──────┼────────────────────────────────────┼────────────────────────────────────────────────────────────────────┤\n│ ❯◯ │ aws-as-datasource │ aws as datasource plugin for Godspeed Framework │\n└──────┴────────────────────────────────────┴────────────────────────────────────────────────────────────────────┘\n```\n- The plugin can also be directly installed by running `npm i @godspeedsystems/aws-as-datasource` command\n\n### Configuration\n\nIn your .yaml file, you will need to configure\n- type: aws (type of the datasource)\n- default_client_config (optional) for initializing your clients, as per the aws config specs\n- Client type to client name mappings via the `types` key\n- `services` contains settings for the services you want to invoke via this datasource. \n - Each service has a type like s3, lamdba etc.\n - They can have their own config overriding the default under the `config` key\n - Note: There can be multiple services configured for the same type. Check `s3` and `s3_1` below\n\n```yaml\ntype: aws\ndefault_client_config: #any aws specific configurations\n credentials:\n accessKeyId: <%config.accessKeyId%>\n secretAccessKey: <%config.secretAccessKey%>\n# service type is the name of the npm module for ex. @aws-sqk/client-dynamodb or @aws-sqk/client-s3 etc\n# The `types` key can have service type to sdk's client names mappings when coding\ntypes: #mappings\n dynamodb: DynamoDB\n s3: S3\n lambda: Lambda\n ssm: SSM\n sqs: SQS\nservices:\n s3:\n type: s3\n config:\n region: <%config.anotherAccessKeyId%>\n credentials:\n accessKeyId: <%config.anotherAccessKeyId%>\n secretAccessKey: <%config.anotherSecretAccessKey%>\n s3_1: #uses default config\n type: s3\n dynamodb:\n type: dynamodb\n sqs:\n type: sqs\n ssm:\n type: ssm\n lamdba:\n type: lambda\n```\n\n### Example usage\n\nIn an event, we establish HTTP endpoint that accepts json objects in request body. When this endpoint is invoked, it triggers the `aws_list` function with the args coming from request body.\n\n#### Example event schema\n```yaml\n# event for create\n\n\"http.post./aws\":\n fn: aws_list\n body:\n type: object\n responses:\n 200:\n content:\n application/json:\n\n```\n\n#### Example TS workflow\n```ts\nimport { GSContext, GSDataSource, GSStatus } from \"@godspeedsystems/core\";\n\nexport default async function (ctx: GSContext, args: any) {\n const ds: GSDataSource = ctx.datasources.aws;\n const response = await ds.execute(ctx, {\n //Pass exactly same args as this aws service's method takes\n ...args,\n //Along with args, pass meta object\n // meta can contain {entityName, method}\n meta: {entityName: 's3', method: 'listBuckets'},\n //Or meta can contain {fnNameInWorkflow} which is same as \n //the 'fn' that we write when invoking datasource from yaml workflow\n //For example, this will also work\n //meta: {fnNameInWorkflow: 'datasource.aws.s3.listBuckets'}\n });\n return response;\n}\n```\n### Example to upload file on aws s3\n\nEvent\n```yaml\n# event for upload s3 file\n\n\"http.post./aws\":\n fn: aws_upload\n body:\n content:\n multipart/form-data:\n schema:\n $ref: '#/definitions/event/upload_file'\n responses:\n 200:\n content:\n application/json:\n\n```\nthe above event calling `aws_upload.ts` workflow from `src/functions/` \n \n```js \nimport { GSContext, GSStatus } from \"@godspeedsystems/core\";\nimport fs from 'fs'\n\nmodule.exports = async (ctx: GSContext) => {\n const { files: { myfile } } = ctx.inputs.data;\n const { datasources, logger } = ctx;\n try {\n return new Promise((resolve, reject) => {\n fs.readFile(myfile.tempFilePath, async function (err, data) {\n if (err) {\n resolve(new GSStatus(false, 500, 'S3 document upload failed', { error: { message: err.message } }));\n\n } // Something went wrong!\n const contentType = ctx.inputs.data.headers['content-type']\n var params = {\n Key: myfile.name, //file name\n Body: data,\n Bucket: 'userdocs123',\n ContentType: contentType,\n };\n\n const res = await datasources.aws.client.s3.putObject(params);\n\n resolve(new GSStatus(true, 200, 'successfully uploaded document', res, {}))\n\n })\n });\n\n } catch (e: any) {\n logger.error('S3 document upload failed %o', e)\n return new GSStatus(false, 500, 'S3 document upload failed', { error: { message: e.message } })\n }\n\n};\n```\n\n### 4. [excel-as-datasource](https://www.npmjs.com/package/@godspeedsystems/plugins-excel-as-datasource)\n\nExcel as a datasource: Turn your spreadsheets into actionable insights. Seamlessly integrate Excel into your applications to harness data, charts, and calculations. Transform static numbers into dynamic, real-time intelligence. Ready to Excel? 📊\n\n### 5. [Nodemailer-as-datasource](https://www.npmjs.com/package/@godspeedsystems/plugins-mailer-as-datasource)\n\nNodemailer as a datasource: Amp up your communication game by using a mailer as a powerful data source. Connect seamlessly to send information through emails. Transform your app into a messaging maestro, unlocking a world of possibilities. Ready to send your data soaring through the digital mail stream? 📧✨\n\nSending emails in your Node.js application has never been smoother. The Godspeed Nodemailer Plugin provides seamless integration between the robust Godspeed framework and Nodemailer, the go-to solution for email delivery in Node.js.\n\n## Features\n\n- **Effortless Setup:** Get up and running in minutes with our easy-to-follow setup guide.\n- **Dynamic Templating:** Craft personalized emails with dynamic content using popular templating engines.\n- **Error Resilience:** Robust error handling ensures reliable email delivery, even in challenging scenarios.\n- **Scalable and Secure:** Designed for scalability and security, so your email system can grow with your application.\n\nWhether you're sending transactional emails, newsletters, or notifications, this plugin empowers you to deliver messages with Godspeed. Let's elevate your email game together!\n\n## Example usage:\n\n### Mailer config ( src/datasources/mail.yaml )\n```yaml\ntype: mail\nuser: 'godspeed@gmail.com'\npass: 'rmeb bjak xcam xkub'\n```\n\n### Mailer Event for Producer ( src/events/mail_send_event.yaml )\n\n```yaml title=src/events/mail_send_event.yaml\nhttp.post./email:\n summary: Send email to user\n description: Sends an email to a user using the mailer datasource\n fn: send_email\n authn: false\n body:\n content:\n application/json:\n schema:\n type: object\n properties:\n recipient:\n type: string\n subject:\n type: string\n body:\n type: string\n required:\n - recipient\n - subject\n - body\n responses:\n '200':\n description: Email sent successfully\n content:\n application/json:\n schema:\n type: object\n properties:\n success:\n type: boolean\n example: true\n data:\n type: object # Or a more specific schema if the mailer returns structured data\n '400':\n description: Failed to send email\n content:\n application/json:\n schema:\n type: object\n properties:\n success:\n type: boolean\n error:\n type: string\n```\n\n### Typescript Workflow to send mail\n```typescript title=src/functions/send_email.ts\n\nimport { GSContext, GSStatus, GSDataSource } from \"@godspeedsystems/core\";\nexport default async function (ctx: GSContext) {\n const { body} = ctx.inputs.data;\n const mailerClient: GSDataSource = ctx.datasources.mailer;\n\n try {\n const response = await mailerClient.execute(ctx, {\n to: body.recipient,\n subject: body.subject,\n text: body.body\n });\n return new GSStatus(true, 200, \"Email Sent successfully\", response);\n } catch (error: any) {\n const errorData = error.stack || error;\n return new GSStatus(false, 400, \"Failed to send email\", errorData);\n }\n}\n```\n\n### 6. [redis-as-datasource](https://www.npmjs.com/package/@godspeedsystems/plugins-redis-as-datasource)\n\nRedis as a datasource: Elevate your data game with the speed and efficiency of Redis. Use it as a powerhouse for caching, real-time analytics, and lightning-fast data retrieval. Embrace the key-value magic to supercharge your application's performance. Ready to Rediscover efficiency? 🚀\n\n\n### 7. [mongoose-as-datasource](https://github.com/godspeedsystems/gs-plugins/tree/main/plugins/mongoose-as-datasource)\nMongoose as a datasource: It provides seamless integration with MongoDB through the Mongoose library. MongoDB is a popular NoSQL database, and with this plugin, you can harness the power of Mongoose to model your data, perform queries, and interact with MongoDB in a structured and efficient manner.\n\n## How to Use\n- Open the godspeed project in vscode and then add the plugin from the CLI of vscode, select the `@godspeedsystems/plugins-mongoose-as-datastore` to integrate the plugin.\n```\n> godspeed plugin add\n ,_, ╔════════════════════════════════════╗\n (o,o) ║ Welcome to Godspeed ║\n ({___}) ║ World's First Meta Framework ║\n \" \" ╚════════════════════════════════════╝\n? Please select godspeed plugin to install: (Press to select, to move rows)\n┌──────┬────────────────────────────────────┬────────────────────────────────────────────────────────────────────┐\n│ │ Name │ Description │\n├──────┼────────────────────────────────────┼────────────────────────────────────────────────────────────────────┤\n│ ❯◯ │ mongoose-as-datastore │ Mongoose as a datasource plugin for Godspeed Framework. │\n└──────┴────────────────────────────────────┴────────────────────────────────────────────────────────────────────┘\n```\n- The plugin can also be directly installed by running `npm i @godspeedsystems/plugins-mongoose-as-datastore` command\n\n\n- This will create some files.\n - You will find a file in your project related to the Prisma plugin at `src/datasources/types/mongoose.ts` \n\n ### Contents of types/mongoose.ts\n The file will look something like this\n ```typescript\n import { DataSource } from '@godspeedsystems/plugins-mongoose-as-datastore';\n export default DataSource;\n ```\n\n### .yaml file\n ![Alt text](../../../static/img/mongoose_folder_structure.png)\n- You can keep the file by any name. This file is used to initialize a mongoose datasource instance. Whatever is the name of the file, you will need to invoke\nthe mongoose datasource commands by the same name. Also your models will be needed to be kept in a folder with the same name as your yaml file (i.e. your datasource instance name). For example mongoose1.yaml would mean\ncalling `fn:datasources.mongoose1..` from yaml workflows and \n`ctx.datasources.mongoose1..` from TS/JS files. Also you will need to create a folder `datasources/mongoose1/models` and keep your models there as detailed below.\n\n- You can override the default response codes for success cases for different methods by putting them in the datasource instance's yaml file\n\n```yaml\ntype: mongoose\nsuccessResponseCodes: #default response codes for success responses\n create: 201\n find: 200\n findOne: 200\n aggregate: 200\n findOneAndUpdate: 201\n findOneAndDelete: 202\n```\n\n#### Error response\nWhen a call has an error the datasource returns following `GSStatus`\n\n```yaml\n code: 500\n success: false\n data: \n message: Internal Server Error\n```\n\n### Setting up Mongoose models\nThis datasource loads the [Mongoose models](https://mongoosejs.com/docs/models.html) from `datasources//models` folder.\n\n![Alt text](../../../static/img/mongoose_folder_structure.png)\n#### Example Mongoose model file\nThese files are stored in `datasources//models` folder Your TS or JS file should export as following\n```typescript\nmodule.exports = {\n type: 'SomeModel', //The name by which you will access methods of this collection/model\n model: SomeModel //The Mongoose Model\n};\n```\nAn example Mongoose model file\n```typescript\nconst { model, Schema, Document } =require('mongoose');\n\nconst SomeModelSchema = new Schema(\n {\n partnerName: {\n type: String,\n required: true,\n },\n productType: {\n type: String,\n required: true,\n },\n apiType: {\n type: String,\n required: true,\n },\n method: {\n type: String,\n required: true,\n },\n api: {\n type: String,\n required: true,\n },\n code: String,\n headers: Schema.Types.Mixed,\n payload: Schema.Types.Mixed,\n response: Schema.Types.Mixed,\n isDynamicUrl: Boolean,\n expectedResponseStatusCode: Number,\n },\n { timestamps: true }\n);\n\nconst SomeModel = model('some-model', SomeModelSchema, 'some-model');\nmodule.exports = {\n type: 'SomeModel', //The name by which you will access methods of this collection/model\n model: SomeModel\n};\n```\n\n### Sample workflow for Mongoose API\nWhen calling any api function it will be called as `fn:datasources.mongoose1..` from yaml workflows and \n`ctx.datasources.mongoose1..` from TS/JS files.\nThe arguments to any `Function_Name` are to be passed in two ways\n- Only the first arg of the function as accepted by the API\n ```yaml\n id: mongoose_workflow\n tasks:\n - id: first_task\n fn: datasource.mongoose.SomeModel.findOne\n args: {\"name\": \"mastersilv3r\"} #Fun fact: YAML acceptes JSON as well. \n ```\n- Most Mongoose functions accept multiple args. To pass all args to an API call, send an array of the acceptable args. This array is spread and passed to the API call\n ```yaml\n id: helloworld2_workflow\n tasks:\n - id: helloworld2_workflow_first_task\n fn: datasource.mongoose.SomeModel.findOne\n args: #as an array\n - name: mastersilv3r #search clause: First argument\n - 'name age' #The projection: second argument\n - {} # Options: the third argument\n ```\n- Calling from a TS/JS workflow works same as any other datasource\n```typescript\nimport { GSContext, GSDataSource, GSStatus } from \"@godspeedsystems/core\";\n\nexport default async function (ctx: GSContext, args: any) {\n const ds: GSDataSource = ctx.datasources.mongoose1;\n //Will need to set a meta object in the args to pass entitType and method\n args.meta = {entityType: 'SomeModel', method: 'findOne'};\n const response = await ds.execute(ctx, args);\n return response;\n}\n```\n### Run the service\n- Set an environment variable `MONGO_URL` as your connection string to running mongodb instance.\n For example, setting via a unix shell.\n ```shell\n export MONGO_URL='mongodb+srv://:@cluster0.xyzabc.mongodb.net/?retryWrites=true&w=majority'\n ```\n- From your command line run your service in the auto-watch mode\n ```bash\n godspeed serve\n ```\n\n### 8. [kafka-as-datasource-as-eventsource](https://www.npmjs.com/package/@godspeedsystems/plugins-kafka-as-datasource-as-eventsource)\n\nKafka is your dynamic data stream and event maestro! As a data source, it floods your systems with real-time insights, turning data into decision-making gold. And when it comes to event sourcing, Kafka orchestrates a symphony of real-time events that power your applications and spark innovation. Experience the future of data and event handling with Kafka. \n\n\n" }, "docs/microservices-framework/datasources/overview.md": { "content": "# Data Sources\n## Overview\n\nData sources play a central role in the Godspeed Framework, serving as the origins or locations from which data can be collected and stored. This documentation provides an overview of data sources within the framework, their usage, and how to invoke them from within workflow tasks.\n\nIn the Godspeed Framework, data sources are fundamental components that enable users to access and manipulate data from various origins. Examples of data sources include databases,message bus, cache, file systems, and third-party APIs.\n\nData sources can be seamlessly integrated into your workflow tasks using a standardized syntax. The key element for invoking data sources is the fn attribute, which is namespaced under datasource. Here's an example of how data sources are used within a workflow task:\n\n### Demonstration\n\n
\n\n
\n\n\n## Datasource Types\n\nDatasources can be divided into two types, \"Datastore as datasource\" and \"API as a datasource\"\n\n\n\"event\n\n\nExample 1: Datastore as Datasource [prisma-as-datastore](/docs/microservices-framework/datasources/list-of-plugins#1-prisma-as-datasource)\n\n\n```ts\nimport { GSContext, GSStatus } from \"@godspeedsystems/core\";\n\nmodule.exports = async (ctx: GSContext) => {\n const { inputs: { data: { params } }, logger, datasources } = ctx;\n\n const response = await datasources.mongo.client.Post.findUnique({\n where: { id: params.id }\n });\n return new GSStatus(true, 200, \"Post fetched\", response );\n}\n```\n\nIn this example:\n\n`datasources.mongo.client.Post.findUnique` is the datasource function, which can be described as below:\n\n - `datasources`: fixed namespace for all data sources for typescript functions\n - `mongo`: name of datasource (name of prisma schema in case of prisma datasource), you can use any database provider, checkout [supported databases list](/docs/getting-started/advance-guide#prisma-supports-wide-range-of-databases)\n - `Post`: entity name\n - `findUnique`: method to be invoked in entity name\n\nthe workflow is consuming the datasource `mongo` and finding one document from Post entity.\n\n:::tip **Godspeed has \"Prisma as datastore plugin\", which provides a uniform access to all databases**\n:::\n\nTo enable this seamless interaction with datasources, the Godspeed Framework allows you to configure data sources within your project. For instance, the example mentions the use of the \"prisma-as-datastore\" plugin to define the \"mongo\" data source. This configuration step ensures that the framework can establish connections and communicate effectively with the specified data source.\n\nIn the above example there is a `mongo` datasource defined in the project, you are free to name your datasource as you like. A default config of your datasource is present in `src/datasources` folder.\n\n\nExample 2: API Datasource [axios-as-datasource](/docs/microservices-framework/datasources/list-of-plugins#2-axios-as-datasource)\n\n\n```ts\nimport { GSContext, GSDataSource, logger, PlainObject } from \"@godspeedsystems/core\";\n\nexport default async function (ctx: GSContext) {\n const res = await ctx.datasources.api.execute(ctx, {\n meta: {\n method: 'get',\n url: '/anything',\n },\n });\n return res;\n};\n```\nIn the above example:\n - `datasources`: fixed namespace for all data sources\n - `api`: name of data source,\n - `get`: API method\n - `./anything`: API endpoint\n\n" }, "docs/microservices-framework/definitions.md": { "content": "---\ntitle: Definitions\ndescription: Use `Definitions/` to create reusable YAML-based schema parts. Reference them via `$ref` using JSON Pointer notation. They function similarly to OpenAPI `components`. They reduce duplication and simplify maintenance across large API designs.\nkeywords: [API schema definitions]\n---\n\n#s Reuse `Definitions` using $ref\n\nIn **Godspeed**, when defining your HTTP events and their payload schemas, you often come across syntax like this:\n\n```yaml\nbody:\n content:\n application/json:\n schema:\n $ref: \"#/definitions/mongo/BusinessProfile\"\n```\nThis usage aligns with how **OpenAPI** refers to **component schemas**, but with Godspeed’s internal structure for reusability.\n\n\n## What is `#/definitions/...`?\n\nThe '#/definitions/...' notation is a **JSON Pointer** used for referencing reusable schemas within your project. It behaves just like '#/components/schemas/...' in OpenAPI 3.0 and enables you to keep your API definitions modular and DRY (Don’t Repeat Yourself).\n\n\n## Location: Where Definitions Live\n\nGodspeed uses the `src/definitions` folder to store all reusable components:\n\n```\nsrc/\n├── definitions/\n│ ├── mongo/\n│ │ └── BusinessProfile.yaml\n│ └── shared/\n│ └── ErrorResponse.yaml\n```\n\nEach `.yaml` file under this folder defines a schema, parameter, response, or other reusable piece of the OpenAPI spec.\n\n---\n\n## Sample Usage\n\nGiven a schema like this in `definitions/mongo/BusinessProfile.yaml`:\n\n```yaml\ntype: object\nproperties:\n businessId:\n type: string\n name:\n type: string\nrequired:\n - businessId\n - name\n```\n\nYou can reference it in an event:\n\n```yaml\nhttp.post./business:\n fn: createBusiness\n body:\n content:\n application/json:\n schema:\n $ref: \"#/definitions/mongo/BusinessProfile\"\n```\n\nThis connects your event’s body schema to the reusable definition.\n\nHowever, **definitions themselves do not affect your API** unless they are explicitly referenced in event or function specifications.\n\n---\n\n* Think of `definitions` as a toolbox — its content only matters when you pick and use the tools elsewhere.\n* Definitions are **not automatically included** in operations.\n* They **must be referenced** via `$ref`.\n* They are useful to prevent duplication and improve maintainability.\n---\n\n## Analogy with OpenAPI `components`\n\n| OpenAPI Section | Godspeed Equivalent |\n| -------------------------- | --------------------------- |\n| `components.schemas` | `/definitions/...` files |\n| `$ref: \"#/components/...\"` | `$ref: \"#/definitions/...\"` |\n\n\n\n\n" }, "docs/microservices-framework/deploy-on-render.md": { "content": "---\ntitle: Deploying Godspeed Service on Render\ndescription: A guide to deploying Godspeed applications on Render, covering project configuration, build commands, start commands, and Render dashboard setup.\nkeywords: [godspeed, render, deployment, cloud platform, node.js]\n---\n# Deploying Godspeed Service on Render\n\nRender is a modern cloud platform that makes it easy to deploy and scale Godspeed applications. This guide will walk you through the steps to get your Godspeed project up and running on Render.\n\n---\n\n## Prerequisites\n\nBefore deploying, ensure you have:\n\n- A working Godspeed project.\n- `pnpm` configured as your package manager.\n- The project built and running successfully locally (`godspeed serve`).\n- A GitHub, GitLab, or Bitbucket repository with your Godspeed project code.\n\n---\n\n## Project Configuration on Render\n\nWhen setting up your Render service:\n\n### Root Directory\n\nThis helps Render locate the `dist` folder after build and correctly start your service.\n\n\nSet the root directory to the base of your Godspeed project.\n\n```plaintext\n.\n```\n---\n\n### Build Command\n\nRender uses this command to install dependencies and build your Godspeed app:\n\n```bash\npnpm install && pnpm run build\n```\n\n---\n\n### Start Command\n\nTo run your app after build:\n\n```bash\nnode dist/index.js\n```\n\nThis starts the compiled Node.js service.\n\n---\n\n## Set Up on Render Dashboard\n\n1. **Go to [Render.com](https://render.com) and log in.**\n2. **Click \"New Web Service\".**\n3. **Connect your repository** that contains the Godspeed project.\n4. **Configure your service:**\n - **Environment:** `Node`\n - **Build Command:** `pnpm install && pnpm run build`\n - **Start Command:** `node dist/index.js`\n - **Root Directory:** `.`\n - **Branch:** Choose your deployment branch (e.g., `main`)\n5. **Add Environment Variables** (from your `.env` or config):\n - Example: `DATABASE_URL`, `PORT`, `NODE_ENV`, etc.\n6. **Click \"Create Web Service\"** and Render will start the build and deployment process.\n\n---\n\n## Tips\n\n- Ensure `dist/` is generated in your local build with `pnpm run build`.\n- Logs and deploy history are accessible from Render’s dashboard.\n- Enable auto-deploy for continuous integration from your Git repo.\n- Configure domains, HTTPS, and scaling from the settings.\n" }, "docs/microservices-framework/event-sources/create-custom-event-source.md": { "content": "---\ntitle: Create a custom Eventsource\ndescription: A comprehensive guide to Godspeed's pluggable event source system. Learn about creating your own custom event source, how to integrate a new event source into your Godspeed service or project.\nkeywords: [custom event sources, new event source, custom plugin, event-driven, godspeed plugin, plugin management, event integration, event source plugins]\n---\n# Create a custom Eventsource\n\n## About Eventsources\n\nAn eventsource is any entity or technology responsible for capturing events or notifications when specific events or conditions occur. These events are consumed by event handlers or processors for real-time or near-real-time responses. Eventsources can include Sync and Async event sources like Message brokers, Webhooks etc.The settings for each datasource lies in src/eventsources directory.\n\n### Steps to create your custom eventsource :\n\n1. Inside the `eventsources` directory, create a `YAML` file with a specific name. In this YAML file, ensure you specify a `type` field, and there must be a corresponding `TypeScript` file in the `types` directory that shares the same name as the `type` you defined.\n\n```\n ├── src\n ├── datasources\n ├── events\n |\n ├── eventsources\n │ ├── types\n │ | └── custom_eventsource.ts\n | |\n │ └── custom_eventsource.yaml\n |\n └── functions\n```\n2. In your TypeScript file, use an import statement to bring in `GSEventSource` from the `@godspeedsystems/core` package. Then, create a class that inherits from `GSEventSource`.\n\n3. Afterward, you can access the methods provided by `GSEventSource`. Initialize your client by calling the `initClient()` function.\n\n4. Once your client is initialized, you can execute its subscription using the `subscribeToEvent` function.\n\n### Sample ts file of a custom event source in Godspeed:\n\n```typescript\n import { PlainObject, GSActor, GSCloudEvent, GSStatus, GSEventSource, GSDataSource, GSContext } from \"@godspeedsystems/core\";\n\n class EventSource extends GSEventSource {\n protected initClient(): Promise {\n // initialize your client\n }\n async subscribeToEvent(eventRoute: string, eventConfig: PlainObject, processEvent: (event: GSCloudEvent, eventConfig: PlainObject) => Promise): Promise {\n try {\n // subscribeToEvent \n } catch (error) {\n throw error;\n }\n }\n }\n const SourceType = 'ES';\n const Type = \"new_es_name\"; // this is the loader file of the plugin, So the final loader file will be `types/${Type.js}`\n const CONFIG_FILE_NAME = \"new_es_name\"; // in case of event source, this also works as event identifier, and in case of datasource works as datasource name\n const DEFAULT_CONFIG = {};\n\n export {\n EventSource,\n SourceType,\n Type,\n CONFIG_FILE_NAME,\n DEFAULT_CONFIG\n }\n ```\n\n:::tip \n**To customize any existing event source, go through the respective plugin's ts file and customize. Use this [repo](https://github.com/godspeedsystems/gs-plugins.git) for better understanding**\n:::\n\n
\n let's use Express as an example of eventsource :\n\n#### Project structure\n\n```bash\n ├── src\n ├── datasources\n │\n ├── events\n | |\n │ └── sample.yaml\n | \n ├── eventsources\n | |\n │ ├── types\n | | |\n │ | └── express.ts\n | |\n │ └── http.yaml\n |\n └── functions\n |\n └── sample.yaml\n```\n\n#### Express config (src/eventsources/http.yaml)\n```\ntype: express\nport: 3000\n```\n\n#### initializing client and execution ( src/eventsources/types/express.ts ) :\n\n```javascript\nimport { PlainObject, GSActor, GSCloudEvent, GSStatus, GSEventSource } from \"@godspeedsystems/core\";\nimport express from \"express\";\nimport bodyParser from 'body-parser';\nimport _ from \"lodash\";\nimport promClient from '@godspeedsystems/metrics';\n//@ts-ignore\nimport promMid from '@mindgrep/express-prometheus-middleware';\nimport passport from \"passport\";\nimport fileUpload from \"express-fileupload\"\nimport { Strategy as JwtStrategy, ExtractJwt } from 'passport-jwt';\n\nexport default class EventSource extends GSEventSource {\n async initClient(): Promise {\n const app = express();\n const {\n port = 3000,\n request_body_limit = 50 * 1024 * 1024,\n file_size_limit = 50 * 1024 * 1024,\n jwt: jwtConfig\n } = this.config;\n\n app.use(bodyParser.urlencoded({ extended: true, limit: request_body_limit }));\n app.use(bodyParser.json({ limit: file_size_limit }));\n app.use(\n fileUpload({\n useTempFiles: true,\n //@ts-ignore\n limits: { fileSize: file_size_limit },\n })\n );\n \n if (jwtConfig) {\n app.use(passport.initialize());\n passport.use(\n new JwtStrategy(\n {\n jwtFromRequest: ExtractJwt.fromAuthHeaderAsBearerToken(),\n secretOrKey: jwtConfig.secretOrKey,\n ignoreExpiration: true,\n jsonWebTokenOptions: {\n audience: jwtConfig.audience,\n issuer: jwtConfig.issuer,\n },\n },\n function (jwtPayload, done) {\n return done(null, jwtPayload);\n },\n ),\n );\n };\n\n app.listen(port);\n\n if (process.env.OTEL_ENABLED == 'true') {\n app.use(\n promMid({\n metricsPath: false,\n collectDefaultMetrics: true,\n requestDurationBuckets: promClient.exponentialBuckets(0.2, 3, 6),\n requestLengthBuckets: promClient.exponentialBuckets(512, 2, 10),\n responseLengthBuckets: promClient.exponentialBuckets(512, 2, 10),\n })\n );\n }\n\n return app;\n }\n\n private authnHOF(authn: boolean) {\n return (req: express.Request, res: express.Response, next: express.NextFunction) => {\n if (authn) {\n return passport.authenticate('jwt', { session: false })(req, res, next)\n } else {\n next();\n }\n };\n };\n\n subscribeToEvent(eventRoute: string, eventConfig: PlainObject, processEvent: (event: GSCloudEvent, eventConfig: PlainObject) => Promise, event?: PlainObject): Promise {\n const routeSplit = eventRoute.split('.');\n const httpMethod: string = routeSplit[1];\n const endpoint = routeSplit[2].replace(/{(.*?)}/g, ':$1');\n const app: express.Express = this.client as express.Express;\n //@ts-ignore\n app[httpMethod](endpoint, this.authnHOF(event.authn), async (req: express.Request, res: express.Response) => {\n const gsEvent: GSCloudEvent = EventSource.createGSEvent(req, endpoint)\n const status: GSStatus = await processEvent(gsEvent, { key: eventRoute, ...eventConfig });\n res\n .status(status.code || 200)\n // if data is a integer, it takes it as statusCode, so explicitly converting it to string\n .send(Number.isInteger(status.data) ? String(status.data) : status.data);\n });\n return Promise.resolve();\n }\n\n static createGSEvent(req: express.Request, endpoint: string) {\n const reqProp = _.omit(req, [\n '_readableState',\n 'socket',\n 'client',\n '_parsedUrl',\n 'res',\n 'app'\n ]);\n const reqHeaders = _.pick(req, ['headers']);\n let data = { ...reqProp, ...reqHeaders };\n\n const event: GSCloudEvent = new GSCloudEvent(\n 'id',\n endpoint,\n new Date(),\n 'http',\n '1.0',\n data,\n 'REST',\n new GSActor('user'),\n {}\n );\n\n return event;\n }\n}\n\nconst SourceType = 'ES';\nconst Type = 'express'; // this is the loader file of the plugin, So the final loader file will be `types/${Type.js}`\nconst CONFIG_FILE_NAME = 'http'; // in case of event source, this also works as event identifier, and in case of datasource works as datasource name\nconst DEFAULT_CONFIG = { port: 3000, docs: { endpoint: '/api-docs' } };\n\nexport {\n EventSource,\n SourceType,\n Type,\n CONFIG_FILE_NAME,\n DEFAULT_CONFIG\n};\n```\n\n\n\n#### Express event (src/events/sample.yaml)\n\n```\nhttp.get./sample_api:\n fn: sample # will invoke sample.ts function located at src/functions/\n body: \n content:\n application/json:\n schema:\n type: object\n properties:\n name: \n type: string\n message: \n type: string \n params: \n - in: query\n name: user\n required: true \n schema: \n type: string \n responses: \n 200:\n content:\n application/json:\n schema:\n type: string\n```\n\n#### Function to be called (src/functions/sample.ts)\n\n```\nimport { GSContext, GSStatus } from \"@godspeedsystems/core\";\n\nexport default function (ctx: GSContext): GSStatus {\n const {\n inputs: {\n data: { query, body }\n },\n logger,\n } = ctx;\n\n const user = query.user || \"Guest\";\n const message = body.message || \"No message provided\";\n\n const result = `Hello ${user}. This is a message from body ${message}`;\n\n logger.info(\"Returning message: %s\", result);\n\n return new GSStatus(true, 200, 'OK', result);\n}\n```\n
\n\n\n
\n let's use cron as an example of eventsource :\n\n#### Project structure\n\n```bash\n ├── src\n ├── datasources\n │\n ├── events\n | |\n │ └── every_minute_task.yaml\n | \n ├── eventsources\n | |\n │ ├── types\n | | |\n │ | └── cron.ts\n | |\n │ └── cron.yaml\n |\n └── functions\n |\n └── every_minute.yaml\n```\n\n#### Cron config ( src/eventsources/cron.yaml )\n```yaml\ntype: cron\n```\n\n#### Initializing client and execution ( src/eventsources/types/cron.ts ) :\n\n```ts\nimport {GSEventSource, GSCloudEvent,PlainObject, GSStatus, GSActor } from \"@godspeedsystems/core\";\nimport cron from \"node-cron\";\n\nexport default class EventSource extends GSEventSource {\nprotected initClient(): Promise {\n return Promise.resolve(cron);\n}\nsubscribeToEvent(\n eventKey: string,\n eventConfig: PlainObject,\n processEvent: (\n event: GSCloudEvent,\n eventConfig: PlainObject\n ) => Promise\n): Promise {\n let [,schedule, timezone] = eventKey.split(\".\");\n let client = this.client;\n if (client) {\n try {\n client.schedule(\n schedule,\n async () => {\n const event = new GSCloudEvent(\n \"id\",\n eventKey,\n new Date(),\n \"cron\",\n \"1.0\",\n {},\n \"cron\",\n new GSActor(\"user\"),\n {}\n );\n await processEvent(event, eventConfig);\n return Promise.resolve()\n },\n {\n timezone,\n }\n );\n } catch (err) {\n console.error(err);\n throw err;\n }\n }\n return Promise.resolve(); \n }\n};\n```\n\n#### cron event ( src/events/every_minute_task.yaml )\n\n```yaml\n# event for Shedule a task for every minute.\ncron.* * * * *.Asia/Kolkata:\n fn: every_minute\n\n```\nFor cron expressions `https://crontab.cronhub.io/`\n\n#### cron workflow to schedule ( src/functions/every_minute.yaml )\n\n```\nimport { GSContext, GSStatus } from \"@godspeedsystems/core\";\n\nexport default function (ctx: GSContext): GSStatus {\n const message = \"HELLO from CRON\";\n ctx.childLogger.info(\"Cron message: %s\", message);\n return new GSStatus(true, 200, undefined, message);\n}\n```\n\n
\n\n### Datasource as eventsource \n\nThere are special cases when datasource can also act as an eventsource.\nFor eg: Kafka can be used both datasource as well as eventsource. When we are publishing message to kafka, it can work as a datasouce .But when we are listening to events on kafka, then it is event source also, then the same client can serve as both.\n\n
\n Let's use kafka as an example of a datasource as an eventsource:\n\n#### Project structure\n\n```bash\n ├── src\n ├── datasources\n │ ├── types\n │ | └── kafka.ts\n | |\n │ └── kafka.yaml\n │\n ├── events\n | |\n │ ├── kafka_publish_event.yaml\n | |\n | └── kafka_consumer_event.yaml\n\n ├── eventsources\n │ ├── types\n │ | └── kafka.ts\n | |\n │ └── kafka.yaml\n |\n └── functions\n |\n ├── kafka-publish.yaml\n |\n └── kafka-consume.yaml\n```\n\n#### Kafka config ( src/eventsources/kafka.yaml )\n```yaml\ntype: kafka\ngroupId: \"kafka_proj\"\n```\n\n#### subscribeToEvent ( src/eventsources/types/Kafka.ts ) :\n\n```javascript\nimport { GSCloudEvent, GSStatus, GSActor, GSDataSourceAsEventSource, PlainObject} from \"@godspeedsystems/core\";\n\nexport default class EventSource extends GSDataSourceAsEventSource {\n async subscribeToEvent(\n eventKey: string,\n eventConfig: PlainObject,\n processEvent: (\n event: GSCloudEvent,\n eventConfig: PlainObject\n ) => Promise\n ): Promise {\n const client = this.client;\n const ds = eventKey.split(\".\")[0];\n const groupId = eventKey.split(\".\")[2]\n const _topic = eventKey.split('.')[1];\n interface mesresp {\n topic: string;\n partition: number;\n message: any;\n }\n\n if (client) {\n const consumer = client.consumer({ groupId: groupId });\n await consumer.subscribe({\n topic: _topic,\n fromBeginning: true,\n });\n\n await consumer.run({\n eachMessage: async (messagePayload: mesresp) => {\n const { message } = messagePayload;\n let msgValue;\n let status;\n let data;\n try {\n msgValue = message?.value?.toString();\n data = {\n body: msgValue,\n };\n status = 200;\n } catch (ex) {\n status = 500;\n return new GSStatus(\n false,\n 500,\n `Error in parsing kafka event data ${msgValue}`,\n ex\n );\n }\n const event = new GSCloudEvent(\n \"id\",\n `${ds}.${_topic}.${groupId}`,\n new Date(message.timestamp),\n \"kafka\",\n \"1.0\",\n data,\n \"messagebus\",\n new GSActor(\"user\"),\n \"\"\n );\n const res = await processEvent(event, eventConfig);\n\n if (!res) {\n status = 500;\n } else {\n status = 200;\n }\n return res;\n },\n });\n }\n }\n}\n```\n\n#### Example event for consume ( src/events/kafka_consumer_event.yaml ) :\n\n```yaml\nkafka.publish-producer1.kafka_proj:\n id: kafka__consumer\n fn: kafka_consume\n body:\n description: The body of the query\n content:\n application/json:\n schema:\n type: string\n```\n\n#### Example workflow for consumer ( src/functions/kafka-consume.yaml ) :\n\n```ts\nimport { GSContext, GSStatus } from \"@godspeedsystems/core\";\n\nexport default function (ctx: GSContext): GSStatus {\n const {\n inputs: {\n data: \n { body }\n },\n } = ctx;\n return new GSStatus(true, 200, 'OK', body);\n}\n```\n
\n\n\n1. To create datasource follow [How to create custom datasource](/docs/microservices-framework/datasources/create-custom-datasource.md)\n\n```\n .\n ├── src\n ├── datasources\n │ ├── types\n │ | └── custom_datasource.ts\n | |\n │ └── custom_datasource.yaml\n │\n ├── events\n | |\n │ └── helloworld.yaml\n |\n ├── eventsources\n │ ├── types\n │ | └── custom_eventsource.ts\n | |\n │ └── custom_eventsource.yaml\n |\n └── functions\n |\n └── helloworld.yaml\n```\n\n2. To create eventsource, Inside the `eventsources` directory, create a `YAML` file with a specific name. In this YAML file, ensure you specify a `type` field, and there must be a corresponding `TypeScript` file in the `types` directory that shares the same name as the `type` you defined.\n\n3. In your TypeScript file, use an import statement to bring in `GSEventSource` from the `@godspeedsystems/core` package. Then, create a class that inherits from `GSEventSource`.\n\n4. Your client is initialized already in datasource so you can execute its subscription using the `subscribeToEvent` function.\n\n\n" }, "docs/microservices-framework/event-sources/create-eventsource-plugin.md": { "content": "# Create npm plugin for your custom eventsource\n\nGodspeed has a [plugin based](https://github.com/godspeedsystems/gs-plugins.git) ecosystem and you can install the plugins using [CLI](/docs/microservices-framework/CLI.md).\n\nIn this section, let us understand how a developer can contribute a plugin to godspeed [plugin repo](https://github.com/godspeedsystems/gs-plugins.git) or for internal use. \n\n### Steps to create new plug-in:\n\n1. Begin by installing the `godspeed-plugin-generator` globally using the following commands:\n\n ```bash\n npm install -g generator-godspeed-plugin\n npm install -g yo\n ```\n\n2. To initiate the creation of your plugin, execute the following command in your terminal:\n\n ```bash\n yo godspeed-plugin\n ```\n\n3. After running the above command, you'll be prompted to enter your desired plugin name. Proceed by typing it in:\n\n ```bash\n ? Enter your plugin name: (your-plugin-name)\n ```\n\n4. Select the type of plugin that aligns with your project's requirements. You can choose from the following options:\n\n ```bash\n ? Select the type of plugin: (Use arrow keys)\n ❯ DataSource \n EventSource \n DataSource-As-EventSource \n ```\n\n ⚠️ **Note:** In certain network environments, particularly on WiFi connections, you may encounter a `Request failed with status code 403` error due to network security protocols or firewall restrictions that \n block specific API requests. To bypass this limitation, consider switching to a **mobile data hotspot**, which often has fewer restrictions. This workaround can help ensure successful command execution and \n avoid network-related request failures.\n\n5. Depending on your selection, the plugin generator will generate a template with your chosen plugin name, such as \"your-plugin-name-as-datasource.\" The structure of the generated files will be as follows:\n\n ```\n .\n ├── src\n | └── index.ts\n |\n ├── package.json\n |\n ├── README.md\n |\n ├── tsconfig.json\n |\n ├── .gitignore\n |\n └── .npmignore\n ```\n\n6. To customize your plugin, navigate to the `index.ts` file located in the `src` directory. You can modify the content within this file to meet your specific plugin requirements. There's no need to make changes to any other files outside of `index.ts`.\n\n**If you opt for `EventSource`, the `index.ts` file appears as follows:**\n ```\n import { PlainObject, GSActor, GSCloudEvent, GSStatus, GSEventSource, GSDataSource, GSContext } from \"@godspeedsystems/core\";\n\n\n class EventSource extends GSEventSource {\n\n protected initClient(): Promise {\n // initialize your client\n }\n async subscribeToEvent(eventRoute: string, eventConfig: PlainObject, processEvent: (event: GSCloudEvent, eventConfig: PlainObject) => Promise): Promise {\n try {\n // subscribeToEvent\n \n } catch (error) {\n throw error;\n }\n }\n }\n\n const SourceType = 'ES';\n const Type = \"p\"; // this is the loader file of the plugin, So the final loader file will be `types/${Type.js}`\n const CONFIG_FILE_NAME = \"p\"; // in case of event source, this also works as event identifier, and in case of datasource works as datasource name\n const DEFAULT_CONFIG = {};\n\n export {\n EventSource,\n SourceType,\n Type,\n CONFIG_FILE_NAME,\n DEFAULT_CONFIG\n }\n ```\n**If you opt for `DataSource-As-EventSource `, the `index.ts` file appears as follows:**\n\n ```\n\n import { GSContext, GSDataSource, PlainObject, GSDataSourceAsEventSource, GSCloudEvent, GSStatus, GSActor} from \"@godspeedsystems/core\";\n\n class DataSource extends GSDataSource {\n protected async initClient(): Promise {\n try {\n \n // initialize your client\n } catch (error) {\n throw error;\n }\n\n }\n\n async execute(ctx: GSContext): Promise {\n try {\n // execute methods\n \n } catch (error) {\n throw error;\n }\n }\n }\n\n class EventSource extends GSDataSourceAsEventSource {\n async subscribeToEvent(\n eventKey: string,\n eventConfig: PlainObject,\n processEvent: (\n event: GSCloudEvent,\n eventConfig: PlainObject\n ) => Promise\n ): Promise {\n\n // subscribeToEvent\n }\n }\n const SourceType = 'BOTH';\n const Type = \"shirisha\"; // this is the loader file of the plugin, So the final loader file will be `types/${Type.js}`\n const CONFIG_FILE_NAME = \"shirisha\"; // in case of event source, this also works as event identifier, and in case of datasource works as datasource name\n const DEFAULT_CONFIG = {};\n\n export {\n DataSource,\n EventSource,\n SourceType,\n Type,\n CONFIG_FILE_NAME,\n DEFAULT_CONFIG\n }\n ``` \n7. For better understanding checkout Examples and watch following videos\n\n [AWS](https://github.com/godspeedsystems/gs-plugins/tree/main/plugins/aws-as-datasource/src/index.ts) (DataSource)\n\n [CRON](https://github.com/godspeedsystems/gs-plugins/tree/main/plugins/cron-as-eventsource/src/index.ts) (EventSource)\n\n [KAFKA](https://github.com/godspeedsystems/gs-plugins/blob/main/plugins/kafka-as-datasource-as-eventsource/src/index.ts) (DataSource-As-EventSource)\n\n### How to create plugins using godspeed?\n\nHere is a video which helps you create a plugin using the godspeed. Do watch for better understanding\n\n
\n\n
\n\n\n### How to get started with a godspeed plugin if installed from npm\n\n
\n \n
\n\n" }, "docs/microservices-framework/event-sources/event-schema.md": { "content": "# Event Schema\n\nTo define an event in Godspeed, you need to write an Event Schema. This schema is a structured YAML configuration that follows the OpenAPI specification, allowing you to define every detail of how the event should behave. All events adhere to a standard structure, which is one of the core design principles of Godspeed, regardless of their source or protocol.\n\n## Writing an event schema \n\nIt involves specifying:\n\n- The name/topic/URL of the event\n- The event handler Workflow (fn)\n- Input and Output schema\n- [Validation error handling](/docs/microservices-framework/event-sources/validations/schema-validation)\n- [Authorization checks](/docs/microservices-framework/authorization/overview.md)\n\nBy writing an event schema, you provide a blueprint that defines how an incoming request or message should be handled, making your API endpoints easy to manage and highly configurable.\n\n\n## The generic Event Schema\nGodspeed follows [Schema Driven Development & Single Source of Truth](../introduction/guard-rails.md#1schema-driven-development), [Configure Over Code](../introduction/guard-rails.md#2configure-over-code) and [Modular Architecture](../introduction/guard-rails.md#4-decoupled-architecture) approach in \n- Listening to events from various sources and acting upon them.\n- Generating API documentation (Swagger) and other schemas like Graphql\n\nThe meta-framework follows a generic schema for defining events - their inputs, outputs, swagger specs, with validations, authentication, authorization etc. \n\n```yaml\nhttp.get./greet: #The initial line depicts a fusion of the event, the employed method, and the path associated with the event.\n fn: function_greet #Required. The 'fn' key receives the function name located in 'src/functions' and forwards the accompanying parameters. \n \n #optional configurations\n #Swagger components\n body: #same as requestBody in Swagger world\n params: #same as swagger `parameters` schema\n responses: #same as swagger `responses` schema\n id: # swagger. if not provided, when generating swagger, this is generated from the URI of the event by default\n operationId: # swagger if not provided, check if `id` is set. If that is also not set, use the summary to generate the operationId\n summary:\n tags: # swagger if you give `tags` array in schema of event, framework uses that to add tags to your generated spec. Else it uses the path and name of the file containing the event as tags. For ex. _ \n \n #Other non-swagger components (optional)\n authn: #custom authentication. Currently plugins support JWT. Can be customized\n authz: #your custom authz workflow\n on_request_validation_error: #when validation fails\n on_response_validation_error: #when validation fails\n log: #Open Telemetry compliant log attributes which help debug and search through logs better\n attributes:\n```\nLets understand the first line from the above snippet `http.get./greet`.\n\n`http`: Protocol http eventsource (can be any)\n\n`get` : method (depends on the eventsource used. Can be topic for Kafka)\n\n`/helloworld`: endpoint (In case of http and graphql sources. Can be groupId in case of Kafka for ex.)\n\nWe are exposing an endpoint with a `get` method on `http` protocol. This endpoint is calling an eventhandler called `helloworld` in the second line. Event handlers can be functions written in typescript, javascript or yaml workflows in Godspeed's DSL format. In the above example the helloworld function exists in `src/functions` directory. \n\n## Key Differences between a Sync and Async Event Schema\n\nWhen switching between eventsources, the event schema undergoes significant changes.\n- The first line is changed for each protocol:\n\n In the case of sync events or HTTP events, the start line includes the eventsource name, method, and path. \n\n However, for async events, the start line combines the source name, topic and group ID (for Kafka), or schedule (for Cron).\n\n- Async events like Kafka do not have responses, authentication and authorization fields in schema.\n\n- Cron events do not have any input.\n\n:::tip Note\nYou can apply multiple compatible eventsource instances in a URI for ex. `graphql & http.get./greeting`\n:::\n\n
\n Example HTTP Schema \n\n```yaml\nhttp.get./greet: #The initial line depicts a fusion of the event, the employed method, and the path associated with the event.\n fn: function_greet #The 'fn' key receives the function name located in 'src/functions' and forwards the accompanying parameters.\n on_request_validation_error: on_request_validation\n params: #It is also possible to define inputs such as 'params,' 'body,' 'headers,' and 'query parameters.'\n - name: greet_message\n in: query\n required: true\n body:\n content:\n application/json:\n schema:\n type: object\n properties:\n name: \n type: string\n responses:\n 500:\n content:\n application/json: \n schema:\n type: object\n 200:\n content:\n application/json:\n schema:\n type: object\n```\n
\n\n**To get a quick understanding of Event schema, please watch the video provided below…**\n\n
\n\n
\n\n\n\n" }, "docs/microservices-framework/event-sources/event-source-plugins/Apollo GraphQl Eventsource.md": { "content": "---\ntitle: Apollo GraphQL Plugin for Godspeed\ndescription: An event source plugin that enables seamless integration of GraphQL APIs using Apollo Server in Godspeed applications. Simplifies schema generation and integration with diverse data sources for scalable and high-performance GraphQL applications.\nkeywords: [graphql, apollo server, event source, api development, godspeed plugin, query language, schema generation, data integration, scalable applications, high performance]\n---\n\n**-** [Plugin Source Code](https://github.com/godspeedsystems/gs-plugins/tree/main/plugins/graphql-as-eventsource) \n\nGraphQL is a query language and runtime for APIs that enables clients to request precisely the data they need. It allows developers to describe the structure of the data they require, providing a more efficient and flexible alternative to traditional REST APIs. GraphQL empowers front-end developers to shape their API requests, minimizing over-fetching and under-fetching of data.\n\n\n**Godspeed** leverages Apollo Server, a powerful and extensible open-source server built on GraphQL, to streamline the creation of GraphQL APIs. Apollo Server excels in automatic schema generation and seamless integration with diverse data sources, providing a robust foundation for scalable and high-performance GraphQL applications.\n\nThis guide offers a concise overview of integrating the GraphQL plugin into the Godspeed framework as an Event Source.\n\n### Setting up GraphQL\n\n### 1. Add the GraphQL plugin\n\n```\n godspeed plugin add @godspeedsystems/plugins-graphql-as-eventsource\n```\n\n### 2. Tailor the configuration file according to your needs.\n\n File: (src/eventsources/grahpql.yaml)\n\n This configuration is same as Express or Fastify configuration, except that currently file upload is not supported and so is not swagger (since Swagger is not relevant for Graphql). \n\n Everything else like authn, authz, port, log attributes is same. \n\n ```yaml\n type: graphql\n port: 4000\n\n #jwt settings to run by default on every event (endpoint)\n authn:\n jwt:\n secretOrKey: mysecret #the secret\n audience: mycompany #aud in jwt token\n issuer: mycompany #iss in jwt token\n\n # authorization policies to run by default on every event\n # Uncomment this to start checking user roles.\n # This will require jwt to be setup, or another middleware to parse the user information in inputs. Currently Graphql, Epress and Fastify support creating user object from JWT token in incoming request.\n authz:\n - id: check_user_role\n fn: com.gs.transform\n args: <%inputs.user.role === 'admin'%> #an inline JS based check of user role\n\n # validation error handling, to transform error responses on wrong input or response\n on_request_validation_error: validations.request.standardResponse\n on_response_validation_error:\n - id: response_validation_error_handler\n fn: com.gs.return\n args: <%inputs%>\n ```\n\n### 3. Setup Graphql event & event handlers \n\n Ensure the event key prefix aligns with the name of the configuration YAML file. In this example, the prefix for the Event key is `graphql`. The event schema follows REST standards, resembling HTTP events.\n\n #### GraphQL Event \n Its same as http event format. When creating Graphql schema, all `get` events are created as queries and rest are mutations.\n File: (src/events/create_category.yaml)\n\n ```yaml\n graphql.post./mongo/category: #this will become a mutation\n summary: Create a new Category\n description: Create Category from the database\n fn: create\n body:\n content:\n application/json:\n schema:\n type: object\n properties:\n name:\n type: string\n responses:\n content:\n application/json:\n schema:\n type: object\n ```\n\n #### GraphQL event handler \n\n Every Graphql event has a `body`, `headers` and `user`. You can access the same in your Typescript, Javascript or YAML functions. Below is an example of a TS and YAML workflow.\n\n #### src/functions/create.ts\n\n ```ts\n export default function (ctx: GSContext) {\n const ds = ctx.datasources.mongoose;\n const response = ds.Category.create(ctx.inputs.data.body);\n return {\n code: 201,\n data: response\n }\n }\n ```\n\n\n\n### 4. Auto generate graphql schema\n```\n godspeed gen-graphql-schema\n```\n\n### 5. Start Server\n\nRun `godspeed serve` and open `https://studio.apollographql.com/sandbox/explorer` on your browser and connect with the correct port on which you are running Apollo service.\n\nThis configuration emphasizes the simplicity of implementing GraphQL within the Godspeed framework, promoting efficiency and clarity in API development.\n\n## Reference links\n**-** [Plugin Repository](https://github.com/godspeedsystems/gs-plugins/tree/main/plugins/graphql-as-eventsource) \n**-** [Issue Tracker](https://github.com/godspeedsystems/gs-plugins/issues) \n**-** [npm package](https://www.npmjs.com/package/@godspeedsystems/plugins-graphql-as-eventsource)\n" }, "docs/microservices-framework/event-sources/event-source-plugins/Cron Eventsource.md": { "content": "---\ntitle: Cron Plugin for Godspeed Framework\ndescription: An event source plugin that enables scheduling tasks using Cron jobs in Godspeed applications. Simplifies the creation of scheduled events with Cron expressions for automated task execution.\nkeywords: [cron jobs, task scheduling, event source, scheduled tasks, godspeed plugin, cron expressions, automated tasks, background service, crontab, task automation]\n---\n\n# Cron Plugin\n\n**-** [Plugin Source Code](https://github.com/godspeedsystems/gs-plugins/tree/main/plugins/cron-as-eventsource)\n\nCron jobs are a standard method of scheduling tasks to run on your server. Cron is a service running in the background that will execute commands (jobs) at a specified time, or at a regular interval.\n\n### How to Add\n- Create a godspeed project from the CLI and add the plugin using following command:\n\n```\n godspeed plugin add @godspeedsystems/plugins-cron-as-eventsource\n```\n### Related files\n\n- You will find two files added in your project related to the plugin at `src/eventsources/types/cron.ts` - the type file and `src/eventsources/cron.yaml` - the instance file.\n\n### Type File\n\n`eventsources/types/cron.ts`\n\n> You generally will not need to touch this file, unless you want to extend or customize the cron functionality.\n\n```typescript\nimport { EventSource } from '@godspeedsystems/plugins-cron-as-eventsource';\nexport default EventSource; \n```\nThis is the file generated by the plugin add command. In case you want to customise the Cron plugin to add new features, you can modify this `type` file in your project. Checkout the section [How to create custom event source](../create-custom-event-source.md)\n\n### Instance file\n\n`src/eventsources/cron.yaml`\n\nYou can create as many Cron instances in your project as cron1, cron2 and so on. The instance files keep configurations of each instance for the given type of plugin (in following example it is of `type: cron`)\n\n```yaml\ntype: cron\n```\n\n### How to Use\n\n#### 1. Define a cron event ( src/events/every_minute_task.yaml )\n\n```yaml\n# event to schedule a task for every minute.\n\ncron.* * * * *.Asia/Kolkata: #this eventkey prefix should be the `type` mentioned in the config `yaml` file, here cron\n fn: every_minute\n```\nKnow more about cron expressions `https://crontab.cronhub.io/`\n\n#### 2. Cron workflow to schedule ( src/functions/every_minute.ts )\n\n```ts\nimport { GSCloudEvent, GSContext, PlainObject } from \"@godspeedsystems/core\";\n\n/**\n * Prints a message every minute via cron\n */\nexport default function (ctx: GSContext) {\n const { logger, childLogger } = ctx;\n\n // Log using both loggers\n logger.info(\"HELLO from CRON\");\n childLogger.info(\"HELLO from CRON\");\n\n // Return the data\n return new GSStatus(true, 200, \"message\", \"HELLO from CRON\");\n}\n\n```\n\n\n## Reference links\n**-** [Plugin Repository](https://github.com/godspeedsystems/gs-plugins/tree/main/plugins/cron-as-eventsource) \n**-** [Issue Tracker](https://github.com/godspeedsystems/gs-plugins/issues) \n**-** [npm package](https://www.npmjs.com/package/@godspeedsystems/plugins-cron-as-eventsource)\n" }, "docs/microservices-framework/event-sources/event-source-plugins/Express Http Eventsource.md": { "content": "---\ntitle: Express Plugin for Godspeed\ndescription: An event source plugin that enables seamless integration of Express.js for handling HTTP events in Godspeed applications. Provides schema-driven development, input/output validation, authentication, and Swagger/GraphQL spec generation for building robust server-side applications.\nkeywords: [express.js, http events, event source, middleware, api development, godspeed plugin, node.js, server-side, schema driven, input validation, output validation, authentication, swagger, graphql]\n---\n\n**-** [Plugin Source Code](https://github.com/godspeedsystems/gs-plugins/tree/main/plugins/express-as-http)\n\nThe Godspeed Express Plugin is a core component of the Godspeed framework, integrating with the popular Express.js framework to handle HTTP events. This plugin provides seamless, schema-driven development for HTTP-based event handling, offering features like input/output validation, authentication, and Swagger/GraphQL spec generation. This flexible tool allows developers to build robust, modular, and configurable server-side applications in Node.js.\n\n\n### Plugin Features\nRead More about the Features of Express Plugin [here](/docs/microservices-framework/event-sources/event-source-plugins/express-plugin.md)\n\n## How to Add Plugin\n- Create a godspeed project from the CLI and by default the Express plugin is integrated into your project, if not then install the plugin from CLI as:\n\n```bash\n$ godspeed plugin add @godspeedsystems/plugins-express-as-http\n```\n### Related Files \n\nAfter installation, you will find two auto generated files in your project related to the Express plugin at `src/eventsources/types/express.ts` - the type file and `src/eventsources/http.yaml` - the instance file.\n\n### Type File\n\n`eventsources/types/express.ts`\n\n> You generally will not need to touch this file, unless you want to extend or customize the Express functionality.\n\n```typescript\nimport { ExpressEventSource } from '@godspeedsystems/plugins-express-as-http';\nexport default ExpressEventSource;\n```\nThis is the file generated by the plugin add command. In case you want to customise the Express plugin to add new features like some middlewares not currently supported by the default plugin implementation, you can modify this `type` file in your project. Checkout the section [How to create custom event source](../create-custom-event-source.md)\n\n\n### Instance file\n\n`src/eventsources/http.yaml`\n\nYou can create as many Express instances in your project as http1, http2 and so on. The instance files keep configurations of each instance for the given type of plugin (in following example it is of `type: express`)\n\n
\n Default Instance (yaml) File of Express Plugin \n\n```yaml\ntype: express\nport: 3000\nbase_url: /api/v1 #the base url of the http service\n\n#Bassic swagger setup\ndocs:\n endpoint: /api-docs # the url on which the service will start\n info: # info object as per swagger 3.0 spec\n title: Sample Godspeed App\n version: 1.1.0\n summary: some http calls\n description: lets play with Godspeed\n contact:\n name: API Support\n url: 'http://www.myfintech.com/support'\n email: support@myfintech.com\n license:\n name: Apache 2.0\n url: 'https://www.apache.org/licenses/LICENSE-2.0.html'\n servers:\n - url: 'http://localhost:3000'\n description: Public API server\n - url: 'http://localhost:3001'\n description: Internal API server\n\nrequest_body_limit: 20000 # maximum size of the request body (in bytes)\nfile_size_limit: 50000 # How big a file can be uploaded (in bytes) Default is 50 MB.\n\n#jwt settings to run by default on every event (endpoint)\nauthn:\n jwt:\n secretOrKey: mysecret #the secret\n audience: mycompany #aud in jwt token\n issuer: mycompany #iss in jwt token\n\n# authorization policies to run by default on every event\n# Uncomment this to start checking user roles.\n# This will require jwt to be setup, or another middleware to setup the user information in inputs\nauthz:\n - id: check_user_role\n fn: com.gs.transform\n args: <%inputs.user.role === 'admin'%> #an inline JS based check of user role\n\n# validation error handling, to transform error responses on wrong input or response\non_request_validation_error: validations.request.standardResponse\non_response_validation_error:\n - id: response_validation_error_handler\n fn: com.gs.return\n args: <%inputs%>\n```\n
\n\n## Get Started by writing an event and workflow for Express \n\nYou can store one or more events in each YAML file stored in the `src/events/` folder. The files can be organized and stored in any folder structure. The functions are created under `src/functions/`.\n\n### HTTP Event Schema Format\n\nAll types of events in Godspeed follow a standard format, with only the first line of the event definition changing as per the eventsource type. In case of http, the following structure represents the applicable composition of an event.\nSpecifying the event schema here, not only validates your input and response, handles authentication and authorization, but also generates your swagger spec and Graphql schema.\n\n```yaml\n http../: #the base_url is prepended to this endpoint path when the service runs\n fn: path.to.function #the event handler: could be a typescript, javascript or YAML function\n \n # Swagger params\n body: #requestBody - applicable for POST, PUT requests\n params: #headers, query, path params as per swagger spec\n responses: #response structure as per swagger spec\n id:\n operationId:\n tags:\n summary:\n description:\n \n #OTHER PARAMS\n # more common parameters to all event types across all event sources, as well applicable to http events\n on_request_validation_error:\n on_response_validation_error:\n authn: \n authz:\n log:\n attributes:\n```\n- The event YAML defines properties for handling specific HTTP requests within the Express app. In the YAML, `` should be replaced with actual HTTP methods such as `GET, POST, PUT, or DELETE`, specifying how the app handles those requests. The `` field should contain the API URL for the respective HTTP route.\n- A function (event handler) will be triggered on sending a request to the respective url. \n\n### Sample Event \n\n```yaml title=src/events/sample.yaml\n http.get./sample_api:\n fn: sample #redirects to src/functions/sample.yaml\n authn: false #to disable global default setting of JWT authn, say authn: false.\n # authz: overriden.custom.authz_fn #here you can add path to a JS/TS/YAML function file or put inline YAML workflow for custom authz\n id: # Swagger id. by default calculated from the event URI\n operationId: #Swagger if not set explicitly, the `id` is used. if `id` not set `summary` is used. If that is also not set, `${method}_${apiEndPoint}` with whitespace replaced by `_` is used\n tags: #swagger tags. by default, a tag is generated from the `folder_path+event_file_name`\n summary: #swagger description\n description: #swaggers summary \n body: #swagger spec equivalent to swagger's requestBody\n content:\n application/json:\n schema:\n type: object\n properties:\n name: \n type: string\n message: \n type: string \n params: #swagger params\n - in: query\n name: user\n required: true \n schema: \n type: string \n responses: #swagger spec of responses \n 200:\n content:\n application/json:\n schema:\n type: string\n log: #custom attributes to add with log statements wherever they are printed by the handlers called by this event\n attributes:\n event_name: sample\n\n # on_request_validation_error:\n # on_response_validation_error:\n```\n\n### Event Handler or Workflow: (TS/JS)\n\nThe meta-framework supports pure functions. This means they take JSON as input and return JSON as output, irrespective of the eventsource from where the event is captured and response returned.\nHereby sharing a typescript function which shows all that you get in your event handler workflow\nwhen an event is captured by _any event source_. The generic input structure is constant whether for Express, Fastify, Kafka, Salesforce, Socket etc.\n\n```typescript\nimport { GSContext, PlainObject } from \"@godspeedsystems/core\";\n\nexport default function (ctx: GSContext) {\n //@ts-ignore\n const {\n inputs: {\n data: {\n params, body, query, user, headers, files\n }\n }, \n childLogger, \n logger,\n outputs,\n functions, \n datasources,\n config,\n mappings\n }: {\n inputs: {\n data: {\n params: PlainObject,\n body: PlainObject,\n query: PlainObject,\n user: PlainObject,\n headers: PlainObject,\n files: any\n }\n }, \n childLogger: any, // Pino logger with log.attributes set (which you saw in eventsource, event configurations as well)\n outputs: PlainObject,\n logger: any, //Pino logger (Plain jain Pino logger without any custom log attributes)\n functions: PlainObject, // The functions in the `src/functions` folder\n datasources: PlainObject, // The clients of the datasources you have configured in this proejct \n config: PlainObject, //Plain JSON of the config folder as per the node-config module\n mappings: PlainObject //Plain JSON loaded from the mappings folder\n } = ctx;\n \n // Will print with workflow_name and task_id attributes\n childLogger.info('Server is running healthy');\n // Will print without workflow_name and task_id attributes\n logger.info('Inputs object \\n user %o query %o body %o headers %o params %o', user, query, body, headers, params);\n logger.info('Outputs object has outputs from previous tasks with given ids %o', Object.keys(outputs));\n logger.info('Datasources object has following datasource clients %o', Object.keys(datasources));\n logger.info('Total functions found in the project %s', Object.keys(functions).length)\n \n return {\n data: 'Its working! ' + body.name,\n code: 200,\n success: true,\n headers: {\n custom_response_header: 'something'\n }\n }\n}\n```\n\n\n## Setting Base Url\nThe base url is set in datasources/api.yaml\n\n```\ntype: axios\nbase_url: /api/v1\n```\n\n## Configuring JWT\n\nYou can configure JWT settings within the eventsources/http.yaml. Here's an example of such a configuration:\n\n```\ntype: express\nauthn:\n jwt:\n issuer: <% config.issuer %> # must be equal to the key iss in your jwt token\n audience: <% config.audience %> # must be equal to the key aud in your jwt token\n secretOrKey: <% config.secret %>\n```\n\n## Configuring OAuth2\n\nYou can configure OAUth2 settings within the eventsources/http.yaml. [Here's an example of such a configuration](/docs/microservices-framework/authentication/oauth2-authentication#3-configure-your-eventsource)\n\n\n### Uploading files\n\nThe Express plugin allows you to upload your files\n\nThe default file size accepted is 50MB. If you wish to specify a custom file size, you can modify the value in `./src/eventsources/http.yaml`.\n\n### Example- Event to Upload file\n\n```yaml\nhttp.post./helloworld:\n fn: helloworld\n body:\n content:\n multipart/form-data:\n schema:\n type: object\n properties:\n fileName:\n type: string\n format: binary\n responses:\n 200:\n content:\n application/json:\n schema:\n type: object\n```\n\n\n### Example Workflow to handle the uploaded files \n\n```typescript\n export default function (ctx: GSContext) {\n // You will get files in the ctx.inputs, along with params, body, query, user, headers\n const { files: { panCardFile } } = ctx.inputs.data;\n //do something like upload to S3\n return {\n data: 'Uploaded',\n success: true,\n code: 201,\n //headers: {}\n }\n }\n```\n\n### Example success response\n\n![image](https://res.cloudinary.com/dzdcjchdc/image/upload/v1704369051/Screenshot_from_2024-01-04_17-20-32_dfzirt.png)\n\nUpon successful upload of the file in Postman, an autogenerated **`tmp`** folder is created within the project directory, containing the uploaded file.\n\n## Plugin Components\n\n[Click here](/docs/microservices-framework/event-sources/event-source-plugins/express-plugin.md) to read more about the Key Components of Express Plugin.\n\n\n## Reference links\n**-** [Plugin Repository](https://github.com/godspeedsystems/gs-plugins/tree/main/plugins/express-as-http) \n**-** [npm package](https://www.npmjs.com/package/@godspeedsystems/plugins-express-as-http)\n" }, "docs/microservices-framework/event-sources/event-source-plugins/Fastify Eventsource.md": { "content": "---\ntitle: Fastify Plugin for Godspeed Framework\ndescription: An event source plugin that enables seamless integration of Fastify, a fast and low overhead web framework, for handling HTTP events in Godspeed applications. Simplifies event subscriptions and processing for building responsive and scalable applications.\nkeywords: [fastify, event source, http events, serverless, godspeed plugin, node.js, web framework, event subscription, scalable applications, event processing]\n---\n\n**-** [Plugin Source Code](https://github.com/godspeedsystems/gs-plugins/tree/main/plugins/fastify-as-http)\n\nThe Godspeed Fastify Plugin is an essential component of the Godspeed framework, designed to streamline the integration of event-driven and serverless functionalities into your projects. This plugin harnesses the power of Fastify, a fast and low overhead web framework for Node.js, to handle HTTP events. It simplifies the process of defining event subscriptions and processing incoming events, providing a robust foundation for building responsive and scalable applications.\n\n## How to Use\n\n### Add the Plugin\nWhen creating a Godspeed project using the CLI, you can add the plugin using following command.\n```bash\ngodspeed plugin add @godspeedsystems/plugins-fastify-as-http\n```\n### Related Files\n- You will find two files in your project related to the Fastify plugin at `src/eventsources/types/fastify.ts` and `src/eventsources/http.yaml`.\n\n#### fastify.ts\n\n```typescript\nimport { FastifyEventSource } from '@godspeedsystems/plugins-fastify-as-http';\nexport default FastifyEventSource;\n```\n\n#### Fastify config (src/eventsources/fastify.yaml)\n\n```yaml\ntype: fastify\nport: 3000\n```\n\n#### Fastify event \n```yaml\n# Event Schema Structure\nhttp../:\n fn: \n body:\n params:\n responses:\n```\n(src/events/sample.yaml)\n```yaml\nhttp.post./sample_api:\n fn: sample #redirects to src/functions/sample.yaml\n body: \n content:\n application/json:\n schema:\n type: object\n properties:\n name: \n type: string\n message: \n type: string \n params: \n - in: query\n name: user\n required: true \n schema: \n type: string \n responses: \n 200:\n content:\n application/json:\n schema:\n type: string\n```\n\n- The event YAML defines properties for handling specific HTTP requests within the Fastify app. In the YAML, `` should be replaced with actual HTTP methods such as `GET, POST, PUT, or DELETE`, specifying how the app handles those requests. The `` field should contain the API URL for the respective HTTP route.\n- A function will be triggered on sending a request to the respective url. The functions are created under `src/functions/`.\n\n#### Function to be called (src/functions/sample.ts)\n```ts\nimport { GSContext, PlainObject, GSStatus } from \"@godspeedsystems/core\";\n\n // Returns a personalized message using query and body input\n\nexport default function (ctx: GSContext): GSStatus {\n const { inputs } = ctx;\n const { query, body } = inputs.data;\n\n const user = query?.user || \"Guest\";\n const message = body?.message || \"No message provided\";\n\n const response = `Hello ${user}. This is a message from body ${message}`;\n\n return new GSStatus(true, 200, \"message received\", response);\n}\n```\n\n\n\n## How It Helps\n\nThe Godspeed Fastify Plugin provides the following benefits:\n\n1. **Fastify Integration:** The plugin simplifies the setup of a Fastify application, enabling easy definition of routes and handlers for incoming HTTP events.\n\n2. **Event Subscription:** Developers can effortlessly subscribe to specific HTTP events by defining routes and handlers using a consistent API.\n\n3. **Customizable Configuration:** The plugin provides simple configuration options for Fastify settings, including port, request body limits, and file size limits.\n\n4. **Integration with Godspeed Core:** Seamlessly works with the Godspeed Core library, facilitating the processing of cloud events and supporting event-driven architecture.\n\n## Plugin Explaination\n\nThis plugin is designed to seamlessly integrate with the Godspeed framework, offering functionality related to event sources using Fastify, a performant web framework for Node.js. It allows you to create event sources that listen for incoming HTTP requests, triggering actions based on those requests.\n\n## Plugin Components\n\nThe [plugin](./src/index.ts) consists of several key components:\n\n### 1. `EventSource` Class\n\n- This class extends `GSEventSource`, a base class provided by the Godspeed framework for creating event sources.\n\n- It initializes an Fastify server to listen for incoming HTTP requests based on the provided configuration options.\n\n- The `subscribeToEvent` method is used to define how the plugin should respond to specific HTTP routes. It maps incoming HTTP requests to Godspeed Cloud Events, processes them using a provided function, and sends a response.\n\n- The `createGSEvent` static method is used to create a Godspeed Cloud Event from an incoming Fastify request.\n\n### 2. Constants\n\n- `SourceType`: A constant representing the source type of the plugin, which is 'ES' (event source).\n\n- `Type`: A constant representing the loader file of the plugin. The final loader file will be located in the 'types' directory and named `${Type.js}`, where `Type` is 'fastify' in this case.\n\n- `CONFIG_FILE_NAME`: In the context of an event source, this constant also serves as the event identifier. For a data source, it works as the data source name. In this plugin, it is set to 'http'.\n\n- `DEFAULT_CONFIG`: A default configuration object with options like the port number for the Fastify server.\n\n\n## Conclusion\n\nThe Godspeed Fastify Plugin is a valuable addition to the Godspeed framework, providing a seamless solution for integrating and managing event sources within your applications. With this plugin, handling HTTP requests, defining routes, and triggering actions become effortless, making it an indispensable tool for building responsive and scalable applications.\n\nWe welcome your feedback and contributions. If you have questions, suggestions, or encounter any issues while using the plugin, please reach out to us. Your insights and ideas can help us improve and enhance this plugin for the entire community.\n\nWe are excited to see the innovative projects you will create with the Fastify plugin, and we look forward to collaborating with you to make your projects even more successful. Happy coding!\n\n## Reference links\n**-** [Plugin Repository](https://github.com/godspeedsystems/gs-plugins/tree/main/plugins/fastify-as-http) \n**-** [Issue Tracker](https://github.com/godspeedsystems/gs-plugins/issues) \n**-** [npm package](https://www.npmjs.com/package/@godspeedsystems/plugins-fastify-as-http)\n" }, "docs/microservices-framework/event-sources/event-source-plugins/Kafka Eventsource.md": { "content": "**-** [Plugin Source Code](https://github.com/godspeedsystems/gs-plugins/tree/main/plugins/kafka-as-datasource-as-eventsource) \n\nKafka is a versatile messaging system designed to securely transfer data between various systems. Its functionality can be tailored through configuration, allowing it to serve as a reliable conduit for real-time event tracking or even function as a replicated distributed database. While it's often colloquially labeled as a queue, it's more precisely described as a hybrid system that combines characteristics and trade-offs from both queue and database systems.\n\nA brief description of how to use Kafka plug-in in our godspeed framework as Data Source as Event Source. \n\n### How to Add\n- Open a godspeed project and add the plugin using cli:\n```\n godspeed plugin add @godspeedsystems/plugins-kafka-as-datasource-as-eventsource\n```\n\n### Related files\n\nAfter installation, you will find auto-generated files in your project related to the plugin at `src/datasources/types/kafka.ts` and `src/datasources/kafka.yaml`and `src/eventsources/types/kafka.ts` and `src/datasources/kafka.yaml`.\n\n\n### Update configuration file based on your requirements in \n`src/eventsources/kafka.yaml`\n\nkafka config ( src/datasources/kafka.yaml )\n```yaml\ntype: kafka\nclientId: \"kafka_proj\"\nbrokers: [\"kafka:9092\"]\n```\n\n### Define kafka event for Producer ( src/events/kafka_pub.yaml )\n\nIn the event, we establish an HTTP endpoint that accepts parameters such as the topic name and message content. When this endpoint is invoked, it triggers the `datasource.kafka.producer` function. This function, in turn, takes the provided topic name and message as input arguments and performs the task of publishing the message to the specified Kafka topic.\n```yaml\n# event to Publish\n\nhttp.post./kafka-pub:\n fn: kafka-publish\n body:\n content:\n application/json:\n schema:\n type: object\n properties:\n message:\n type: string\n required: ['message']\n\n```\n\n### kafka workflow for Producer ( src/functions/kafka-publish.ts )\n\n```ts\nimport { GSContext, PlainObject, GSStatus } from \"@godspeedsystems/core\";\n /**\n * Kafka producer function - publishes message from request body to a Kafka topic\n */\nexport default async function (ctx: GSContext): Promise {\n const { datasources, inputs, logger } = ctx;\n const { message } = inputs.data.body;\n\n if (!message) {\n return new GSStatus(false, 400, 'Failed', \"Message is required\");\n }\n\n const kafkaProducer = datasources.kafka?.producer;\n\n if (typeof kafkaProducer !== \"function\") {\n return new GSStatus(false, 500, 'Failed', \"Kafka producer function not found\");\n }\n\n try {\n const result = await kafkaProducer({\n topic: \"publish-producer1\",\n message\n });\n\n logger.info(\"Kafka message published successfully: %o\", result);\n\n return new GSStatus(true, 200, 'OK', \"Message published to Kafka\");\n } catch (err) {\n logger.error(\"Kafka publish failed: %o\", err);\n return new GSStatus(false, 500, 'Failed', \"Failed to publish to Kafka\");\n }\n}\n```\n### Example usage EventSource (Consumer):\n\nUpdate configuration file based on your requirements in `Eventsources/kafka.yaml`.\n\n### kafka config ( kafka.yaml )\n```yaml\ntype: kafka\ngroupId: \"kafka_proj\"\n```\n\n### kafka event for consumer (src/events/kafka_pub.yaml)\n\nTo use Consumer we need to follow the below event key format.\n\n```\n kafka.{Topic}.{GroupId}\n```\nThe consumer event is triggered whenever a new message arrives on the specified topic. Upon triggering, it retrieves the incoming message and forwards it to the `kafka_consume` function. Inside this function, the incoming message is processed, and the result is then returned.\n\n``` yaml\n# event to consume data from Topic\nkafka.publish-producer1.kafka_proj: # event key\n id: kafka_consumer\n fn: kafka_consume\n body:\n description: The body of the query\n content:\n application/json: \n schema:\n type: string\n```\n### kafka workflow for Consumer \n(src/functions/kafka_consume.ts)\n```ts\nimport { GSContext, PlainObject, GSStatus } from \"@godspeedsystems/core\";\n/**\n * Kafka consumer function - returns the consumed Kafka message\n */\nexport default function (ctx: GSContext): GSStatus {\n const { inputs } = ctx;\n\n return new GSStatus(true, 200, 'OK', inputs.data);\n}\n```\n\n\n## Reference links\n**-** [Plugin Repository](https://github.com/godspeedsystems/gs-plugins/tree/main/plugins/kafka-as-datasource-as-eventsource) \n**-** [Issue Tracker](https://github.com/godspeedsystems/gs-plugins/issues) \n**-** [npm package](https://www.npmjs.com/package/@godspeedsystems/plugins-kafka-as-datasource-as-eventsource)\n\n" }, "docs/microservices-framework/event-sources/event-source-plugins/Overview.md": { "content": "---\ntitle: Event Source Plugins Overview for Godspeed Framework\ndescription: A comprehensive guide to Godspeed's pluggable event source system. Learn about available plugins, installation process, and how to integrate various event sources into your Godspeed applications with ease.\nkeywords: [event sources, plugins, event-driven architecture, godspeed plugin, plugin management, event integration, plugin installation, framework plugins, event source plugins, plugin architecture]\n---\n\n# Event Source Plugins Overview\n\nGodspeed framework adopts a pluggable approach that empowers you to define eventsources effortlessly. Our framework provides an interface that caters to diverse eventsource needs. Here's a glimpse into the exceptional eventsource plugins crafted by our core framework team.\n\n## List of event source plugins\n\n### 1. [express-as-http](https://www.npmjs.com/package/@godspeedsystems/plugins-express-as-http)\n\nThe Godspeed Express Plugin is a core component of the Godspeed framework, integrating with the popular Express.js framework to handle HTTP events. This plugin provides seamless, schema-driven development for HTTP-based event handling, offering features like input/output validation, authentication, and Swagger/GraphQL spec generation.\n\n### 2. [cron-as-eventsource](https://www.npmjs.com/package/@godspeedsystems/plugins-cron-as-eventsource)\n\nCron eventsource is a plugin which runs in the background and execute events at specified time or regular intervals.\n\n\n### 3. [kafka-as-datasource-as-eventsource](https://www.npmjs.com/package/@godspeedsystems/plugins-kafka-as-datasource-as-eventsource)\n\nKafka is a versatile messaging system designed to securely transfer data between various systems. Its functionality can be tailored through configuration, allowing it to serve as a reliable conduit for real-time event tracking or even function as a replicated distributed database. And when it comes to event sourcing, Kafka orchestrates a symphony of real-time events that power your applications and spark innovation. Experience the future of data and event handling with Kafka. \n\n### 4. [apollographql-as-eventsource](https://www.npmjs.com/package/@godspeedsystems/plugins-graphql-as-eventsource)\n\nGraphQL is a query language and runtime for APIs that enables clients to request precisely the data they need. It allows developers to describe the structure of the data they require, providing a more efficient and flexible alternative to traditional REST APIs. GraphQL empowers front-end developers to shape their API requests, minimizing over-fetching and under-fetching of data.\n\n### 5. [fastify-as-eventsource](https://www.npmjs.com/package/@godspeedsystems/plugins-fastify-as-http)\n\nThe Godspeed Fastify Plugin is designed to streamline the integration of event-driven and serverless functionalities into your projects. This plugin harnesses the power of Fastify, a fast and low overhead web framework for Node.js, to handle HTTP events. It simplifies the process of defining event subscriptions and processing incoming events, providing a robust foundation for building responsive and scalable applications.\n\n\n## How to add plugin in your project?\n\nTo browse available plugins in Godspeed, simply use this command:\n\n```bash\n$ godspeed plugin add\n```\nYou can directly install a plugin by specifying its package name:\n\nExample:\n\n```bash\n$ godspeed plugin add @godspeedsystems/plugins-cron-as-eventsource\n```\n\n```bash\n$ godspeed plugin add @godspeedsystems/plugins-kafka-as-datasource-as-eventsource \n```\n\n\n" }, "docs/microservices-framework/event-sources/event-source-plugins/Setup Middleware.md": { "content": "# Writing Custom Middleware in EventSource or DataSource\n\nGodspeed's plugin architecture allows you to extend behavior by injecting **custom middleware** into your **EventSource** (e.g., Express).\n\nThis is useful for adding: Logging, Rate limiting, Authentication, Metrics, Request/response mutation, Any side-effect logic before or after processing an api or event.\n\n\n### Where Can You Add Middleware?\n\nMiddleware can be added inside:\n\n* `src/eventsources/types/.ts`\n\nYou typically do this by **subclassing the base plugin** and injecting middleware in the `initClient()` method.\n\n---\n\n### Example: Adding Middleware in Express EventSource\n\nHere’s how to add a middleware that:\n\n* Overrides the `/metrics` endpoint\n* Combines Express metrics with Prisma metrics\n\n#### File: `src/eventsources/types/express.ts`\n\n```ts\nimport { PlainObject } from \"@godspeedsystems/core\";\nimport { EventSource } from \"@godspeedsystems/plugins-express-as-http\";\nimport promClient from '@godspeedsystems/metrics';\nimport { PrismaClient } from \"../../datasources/prisma-clients/schema\";\n\nclass MyEventSource extends EventSource {\n async initClient(): Promise {\n const client = await super.initClient();\n\n // Add your custom middleware here\n client.get('/metrics', async (req, res) => {\n try {\n const expressMetrics = await promClient.register.metrics();\n const prismaMetrics = await new PrismaClient().$metrics.prometheus();\n res.set('Content-Type', promClient.register.contentType);\n res.end(`${expressMetrics}\\n${prismaMetrics}`);\n } catch (err: any) {\n res.status(500).send({ error: err.message });\n }\n });\n\n return client;\n }\n // (Optional) override setupMetrics to disable default `/metrics` middleware\n setupMetrics(app: any) {\n promClient.collectDefaultMetrics(); // only collects, doesn't auto-bind endpoint\n }\n}\nexport default MyEventSource;\n```\n\n---\n\n### General Pattern\n\n1. **Subclass the plugin’s `EventSource` class.**\n2. **Override `initClient()` to gain access to the underlying client (e.g., `express()` app, PrismaClient, Axios instance).**\n3. **Inject your middleware using standard APIs** (e.g., `app.use()`, `app.get()`, or interceptors for Axios).\n\n---\n\n### Example Use Cases\n\n#### Logging Every Request\n\n```ts\nclient.use((req, res, next) => {\n console.log(`[${req.method}] ${req.originalUrl}`);\n next();\n});\n```\n\n#### Injecting Custom Headers\n\n```ts\nclient.use((req, res, next) => {\n res.set('X-Godspeed-Version', 'v2.0.0');\n next();\n});\n```\n\n#### Protecting a Route\n\n```ts\nclient.get('/admin', (req, res, next) => {\n if (!req.headers['x-admin-token']) {\n return res.status(403).send(\"Forbidden\");\n }\n next();\n}, handler);\n```\n\n### ⚠️\n\n* If your middleware adds an endpoint like `/metrics`, ensure it **doesn't conflict with auto-registered routes** (disable via config).\n* Middleware runs **on every request** unless scoped to a specific route.\n* Always call `next()` in Express-style middleware unless you're sending a response directly.\n\n" }, "docs/microservices-framework/event-sources/event-source-plugins/express-plugin.md": { "content": "\n## Express-as-http Plugin \n**-** [Plugin Source Code](https://github.com/godspeedsystems/gs-plugins/tree/main/plugins/express-as-http)\n\n\nThis plugin is designed to integrate with the Godspeed framework and provides functionality related to event sources using Express.js, a popular Node.js web application framework. It allows you to create event sources that can listen for incoming HTTP requests and trigger actions based on those requests.\n\n## Plugin Features\n\nThe Godspeed Express Plugin provides the following benefits:\n\n1. **Modular Express Integration:** The plugin abstracts away the complexities of setting up an Express.js application, making it effortless to define routes and handlers for incoming HTTP ou can replace Express with Fastify or another HTTP service tomorrow without changing a single line of code in your business logic and http eventsource configurations, giving you flexibility. You can also provide same API using Graphql or Fastify service by just replacing `http` with `http & graphql & fastify` in your event definitions. For ex. `http & graphql & fastify.get.helloworld` \n\n2. **Unified event schema with validations and auth:** Developers can easily subscribe to specific HTTP events by defining routes and handlers using a uniform schema which handles authentication, authorization and validations, not just for Express but also other event sources.\n\n3. **Customizable Configuration:** The plugin allows for easy configuration of Express settings, such as port, request body limits, file size limits, JWT authentication, authorization, swagger generation etc.\n\n4. **Reusable pure function handlers:** The plugin works seamlessly with the Godspeed Core library, enabling the processing of cloud events in pure workflows, decoupled with Express API, making them reusable.\n\n5. **File upload feature:** The Express plugin allows you to upload files as well.\n\n6. **Authenticating users using oauth2:** This will allow your users to sign in to your application using their existing Google/Linkedin/GitHub credentials.\n\n## Plugin Components\n\nThe plugin consists of several key components:\n\n### 1. `EventSource` Class\n\n- This class extends `GSEventSource`, a base class provided by the Godspeed framework for creating event sources.\n\n- It initializes an Express.js server to listen for incoming HTTP requests based on the provided configuration options.\n\n- The `subscribeToEvent` method is used to define how the plugin should respond to specific HTTP routes. It maps incoming HTTP requests to Godspeed Cloud Events, processes them using a provided function, and sends a response.\n\n- The `createGSEvent` static method is used to create a Godspeed Cloud Event from an incoming Express.js request.\n\n### 2. Constants\n\n- `SourceType`: A constant representing the source type of the plugin, which is 'ES' (event source).\n\n- `Type`: A constant representing the loader file of the plugin. The final loader file will be located in the 'types' directory and named `${Type.js}`, where `Type` is 'express' in this case.\n\n- `CONFIG_FILE_NAME`: In the context of an event source, this constant also serves as the event identifier. For a data source, it works as the data source name. In this plugin, it is set to 'http'.\n\n- `DEFAULT_CONFIG`: A default configuration object with options like the port number for the Express.js server.\n\n## Conclusion\n\nOur Express plugin is a powerful addition to the Godspeed framework, allowing you to seamlessly integrate and manage event sources within your applications. With this plugin, you can effortlessly handle HTTP requests, define routes, and trigger actions, making it an essential tool for building robust and responsive applications.\n\nWe value your feedback and contributions. If you have questions, suggestions, or encounter any issues while using the plugin, please don't hesitate to reach out to us. Your insights and ideas can help us improve and enhance this plugin for the entire community.\n\nWe're excited to see what you'll create with the Express plugin, and we look forward to collaborating with you to make your projects even more successful. Happy coding!\n\n" }, "docs/microservices-framework/event-sources/event-types/cron-events.md": { "content": "\n# Cron event\n[Cron](https://github.com/godspeedsystems/gs-plugins/tree/main/plugins/cron-as-eventsource#godspeed-plugin-cron-as-eventsource) jobs are a standard method of scheduling tasks to run on your server. Cron is a service running in the background that will execute commands (jobs) at a specified time, or at a regular interval. Jobs and their schedules are defined in a configuration file called a crontab.\n\n:::tip \n- event key prefix should be the type mensioned in the config yaml file.\n- For cron expressions https://crontab.cronhub.io/\n:::\n\n### Cron Event \n\n( src/events/every_minute_task.yaml )\n```yaml\n# event for Scheduling a task for every minute.\ncron.* * * * *.Asia/Kolkata: # event key\n fn: every_minute\n\n```\n\n### Explanation of Cron Event Configuration:\n\n- `cron.*`: This is the cron syntax for the field representing minutes. The asterisk (*) in this position means \"every minute,\" so the event is scheduled to run every minute.\n\n- `* * * * *`: These asterisks represent the other cron fields, which specify the schedule for hours, days of the month, months, and days of the week, respectively. \n\n- `Asia/Kolkata`: This is a timezone specification. It indicates that the event is scheduled to run in the Asia/Kolkata timezone. Kolkata is a city in India, and this timezone corresponds to the Indian Standard Time (IST).\n\n\n\n### Cron workflow to schedule ( src/functions/every_minute.ts )\n\n```ts\nimport { GSCloudEvent, GSContext, PlainObject } from \"@godspeedsystems/core\";\n/**\n * Prints a message every minute via cron\n */\nexport default function (ctx: GSContext) {\n const { logger, childLogger } = ctx;\n\n // Log using both loggers\n logger.info(\"HELLO from CRON\");\n childLogger.info(\"HELLO from CRON\");\n\n // Return the data\n return new GSStatus(true, 200, \"message\", \"HELLO from CRON\");\n}\n```\n\n\n\n\n" }, "docs/microservices-framework/event-sources/event-types/graphql-events.md": { "content": "# Graphql API\n\nThe GraphQL API configuration in Godspeed allows seamless integration of GraphQL APIs, emphasizing simplicity and efficiency in API development. The configuration file (Apollo.yaml) specifies the GraphQL type and port, ensuring alignment with the event key prefix.\n\n### GraphQL Configuration \nThe Apollo Graphql plugin is currently configured exactly the same as Express and Fastify eventsources. Except that it doesn't have swagger config and doesn't support file upload as of now.\n\n(src/eventsources/apollo.yaml)\n```yaml\n\ntype: graphql\nport: 4000\n# eventsource level default settings (can be overriden at event level)\nauthn:\nauthz:\non_request_validation_error:\non_response_validation_error:\n\n```\n\n:::tip note\nEnsure the event key prefix aligns with the name of the configuration YAML file. In this example, the prefix for the Event key is `apollo` as per the yaml file name (src/eventsources/apollo.yaml). The event schema follows REST standards, resembling HTTP events.\n:::\n\n### GraphQL Event \n\n(src/events/create_category.yaml)\n```yaml\napollo.post./mongo/category: // event key having prefix apollo\n summary: Create a new Category\n description: Create Category from the database\n fn: create\n body:\n content:\n application/json:\n schema:\n type: object\n```\n\n### Graphql Workflow\n\n ```ts\n export default function (ctx: GSContext) {\n const ds = ctx.datasources.mongoose;\n const response = ds.Category.create(ctx.inputs.data.body);\n return {\n code: 201,\n data: response\n }\n }\n ```\n\n\n\n:::tip note\n- use godspeed gen-graphql-schema to auto generate graphql schema.\n- use godspeed serve to start server.\n\n:::\n\nThis configuration emphasizes the simplicity of implementing GraphQL within the Godspeed framework, promoting efficiency and clarity in API development.\n\n" }, "docs/microservices-framework/event-sources/event-types/http-events.md": { "content": "# HTTP events or Sync APIs\n\nIn Godspeed, sync APIs are created as HTTP events. Each API endpoint is essentially an event configured to listen for HTTP requests and respond accordingly. HTTP events accept inputs such as body, headers, path parameters, and query parameters. \n\n### HTTP Event Schema\n\n- The framework provides request and response schema validation out of the box.\n- To switch between events, you'll need to adjust the event schema based on the expected inputs. For instance,\n\nThis event schema is supported by [HTTP express eventsource](https://github.com/godspeedsystems/gs-plugins/tree/main/plugins/express-as-http#godspeed-express-plugin)\n\n```yaml\nhttp.post./mongo/user/search/{id}: #This is the only thing that changes across all the events\n summary: Update a user # as per Swagger spec\n description: Update user from database # as per Swagger spec\n fn: com.biz.mongo.user.update # function to be invoked\n on_validation_error: com.jfs.handle_validation_error\n params: # params as per Swagger spec\n - name: id\n in: path\n required: true\n schema:\n type: string\n - name: name\n in: query\n required: false\n schema:\n type: string\n body: #as per Swagger spec\n content:\n application/json:\n schema:\n $ref: \"#/definitions/mongo/BusinessProfile\" #defined for definition section.\n responses: #as per Swagger spec\n 200:\n content:\n application/json:\n schema:\n type: object\n 500:\n content:\n application/json:\n schema:\n type: string\n```\n\n- The event's first line comprises three key elements: the type of eventsource (e.g., `http`), the method (e.g., `put`), and the URL (`/mongo/user/{id}`). This format is defined by the eventsource plugin, and it is the only line that changes across all events.\n\n#### Accessing Event Properties in Workflows\n\nFor an HTTP event, the headers, query, params and body data are captured in a standard format, and made available in the `inputs` object [for use in the workflows](/docs/microservices-framework/workflows/overview.md).\n\nThe inputs (event) object has following properties:\nIf you have jwt authentication like strategy implemented, then\n\n- **past user object can be accessed as**:`<%inputs.user%>`\n\n- **query can be accessed as**: `<%inputs.query.var_name%>` Present in case of http events\n\n- **params can be accessed as**: `<%inputs.params.path_param%>` Present in case of http events\n\n- **headers can be accessed as**: `<%inputs.headers.some_header_key%>` Present in case of http events\n\n- **body can be accessed as**: `<%inputs.body.key%>` Present for all events except for http events which don't have a body. For ex. http.get\n\n- **files can be accessed as**: `<%inputs.files%>` Any files uploaded via HTTP event. Not present in other kind of events\n\n### Swagger Specs\n\nFramework will give you below folder structure.\n\n```\n .\n ├── src\n ├── datasources\n │ ├── types\n │ | └── axios.ts\n | |\n │ └── api.yaml\n │\n ├── events\n | |\n │ └── helloworld.yaml\n |\n ├── eventsources\n │ ├── types\n │ | └── express.ts\n | |\n │ └── http.yaml\n |\n └── functions\n |\n └── helloworld.yaml\n```\n\n1. Swagger ui is enabled automatically through adding `docs` in **\"./src/eventsources/http.yaml\"**\n\n2. `/` is the default endpoint,if you want to provide your custom swagger endpoint, you can modify the endpoint from **\"./src/eventsources/http.yaml\"**\n\n#### Update http.yaml( src/eventsources/http.yaml )\n\n```yaml\ntype: express\nport: 3000\nbase_url: /api/v1\ndocs:\n endpoint: /\n```\n\n:::tip note\nBoth [express](https://github.com/godspeedsystems/gs-plugins/blob/main/plugins/express-as-http/README.md) and [fastify](https://github.com/godspeedsystems/gs-plugins/tree/main/plugins/fastify-as-http) plugins support this format of configuration to setup swagger\n:::\n\n#### Custom server URL and custom Info:\n\nIn the `http.yaml` file, you have the option to incorporate a custom server URL for API documentation. By including this custom server URL, any autogenerated documentation or Swagger specifications will feature this URL within the `servers` section. Additionally, you can enhance the documentation by specifying a custom `title`, `version`, and `contact information` through the addition of `info`.\n\n```yaml\ntype: express\nport: 3000\nbase_url: /api/v1\ndocs:\n endpoint: /\n info:\n version: 0.0.1\n title: \"Godspeed: Sample Microservice\"\n description: Sample API calls demonstrating the functionality of Godspeed framework\n termsOfService: \"http://swagger.io/terms/\"\n contact:\n name: Mindgrep Technologies Pvt Ltd\n email: talktous@mindgrep.com\n url: \"https://docs.mindgrep.com/docs/microservices/intro\"\n license:\n name: Apache 2.0\n url: \"https://www.apache.org/licenses/LICENSE-2.0.html\"\n servers:\n - url: https://api.example.com:8443/api\n description: staging\n```\n\nFor example,\n\n\"swaggers\"/\n\n\"swagger\"/\n" }, "docs/microservices-framework/event-sources/event-types/kafka-events.md": { "content": "# Kafka event\n> A [Kafka](https://github.com/godspeedsystems/gs-plugins/tree/main/plugins/kafka-as-datasource-as-eventsource#godspeed-plugin-kafka-as-datasource-as-eventsource) event is specified as `{datasourceName}.{topic_name}.{group_id}` in [the Kafka event specification](#example-spec-for-kafka-event).\n\nWithin the Kafka event structure, the content of the message is captured and made accessible as `inputs.body`, facilitating its integration into the handler workflow for processing.\n\n#### Example spec for Kafka event\n\n``` yaml\n # event for consume data from Topic\nKafka.publish-producer1.kafka_proj: # event key\n id: kafka_consumer\n fn: kafka_consume\n body: \n content:\n application/json:\n schema:\n type: object\n properties:\n message: # the content of the message is captured here\n type: string\n required: ['message']\n ```\n\n#### Example workflow consuming a Kafka event\n\n```ts\nimport { GSContext, PlainObject, GSStatus } from \"@godspeedsystems/core\";\n/**\n * Kafka consumer function - returns the consumed Kafka message\n */\nexport default function (ctx: GSContext): GSStatus {\n const { inputs } = ctx;\n\n return new GSStatus(true, 200, 'OK', inputs.data);\n}\n```\n\n\n\n- To know about validation_error, refer [this](/docs/microservices-framework/event-sources/validations/validation-error.md)\n\n:::tip note\n- Response_validation_error, authz, authn are not applicable to this protocol since there is no response (thus no error handling for the same). Similarly, Kafka consumers don't require authentication or authorization, and hence, they are not specified here.\n:::\n\n" }, "docs/microservices-framework/event-sources/event-types/overview.md": { "content": "# Event types\nBased on how processing is handled ,events can be classified into two types: synchronous (sync) and asynchronous (async) events, each suited for various protocols. \n\n\n\"event\n\n\n**Synchronous events** are typically associated with \n- `HTTP/REST protocol:` Express can also handle asynchronous operations, the basic request-response cycle is synchronous.\n- `gRPC:` gRPC is a framework for building remote procedure call (RPC) systems. If you're using the typical request-response pattern, it can be thought of as synchronous.\n- `GraphQL:` Most commonly, GraphQL is used for synchronous communication, but it can handle asynchronous operations when needed.\n\n**Asynchronous events**, on the other hand, are exemplified by\n- `Cron jobs:` Cron jobs are asynchronous because they don't execute immediately when you create or schedule them.\n- `Kafka:` Apache Kafka is an event streaming platform. It's designed for asynchronous, real-time data streaming.\n- `RabbitMQ:` RabbitMQ is a message broker that allows asynchronous communication between distributed systems.\n- `WebSocket communication:` WebSockets provide full-duplex communication channels over a single TCP connection. They are inherently asynchronous, allowing real-time bidirectional communication between clients and servers.\n\nYou can also create **custom events** like \n- `Salesforce:` Salesforce UI or synchronous API calls provide immediate feedback, many of the underlying processes that handle data processing and automation are performed `asynchronously` for scalability and efficiency.\n:::note\nYou can refer [Salesforce plugin](https://docs.godspeed.systems/v1/microservices/events#624-salesforce-event) which we have in V1.\nWe have [bounty](https://forum.godspeed.systems/t/1-million-developer-bounty-program-build-earn-with-godspeed/128) on implementing [plugins for V2](https://github.com/godspeedsystems/gs-plugins#list-of-plugins).\n:::\n- `S3:` Amazon S3 provides real-time access to stored objects and can be used for synchronous operations like serving web content, its internal processes and features are optimized for `asynchronous`.\n- `Google Pub/Sub:` Google Cloud Pub/Sub enables real-time messaging with synchronous operations for immediate communication. Its architecture also supports `asynchronous` messaging, ensuring scalability and efficiency. You can use Google pub/sub to access events like new email on gmail.\n- `Amazon SQS:` :Amazon SQS is a fully managed message queuing service designed for both synchronous and asynchronous operations. It offers immediate message processing while optimizing for scalability through `asynchronous` handling.\n\n\n**To learn more about Schema Driven Development and Events in Godspeed, please watch the video provided below…**\n\n
\n\n
\n" }, "docs/microservices-framework/event-sources/events-overview.md": { "content": "# Introduction To Events\nEvents are the core of creating responsive, real-time applications in godspeed. An event serves as a trigger that initiates specific actions or workflows based on defined conditions or inputs. Whether it’s an API call, a scheduled job, or a message arriving from a queue, events provide a structured way to handle incoming requests and data changes.\n\n\n## What are Events in Godspeed?\nIn the meta-framework world, we call all types of sync events (apis) and async events (ex. Kafka, Socket, Cron) as **Events**. Events are entry points into your Godspeed application, determining how the system should respond to various types of inputs. Each event can be configured with a unique set of properties that define:\n\n- **The Source of the Event:** For example, HTTP requests, Kafka messages, Cron schedules, or custom-defined sources.\n- **The Triggering Conditions:** Conditions or methods (like GET, POST, PUT for HTTP events) that specify when an event should activate.\n- **The Actions or Workflows:** Functions or workflows that are executed when an event is triggered, containing the logic for handling the request or data.\n\n- To start using events, explore the different [Event Source Plugins](/docs/microservices-framework/event-sources/event-source-plugins/Overview) available in Godspeed and choose the ones that suit your application’s needs.\n\n## Defining an Event: Writing an Event Schema\nTo define an event in Godspeed, you need to write an [Event Schema](/docs/microservices-framework/event-sources/event-schema). This schema is a structured YAML configuration that follows the OpenAPI specification, allowing you to define every detail of how the event should behave. \n\n\n## Types of Events\nFor more details on configuring each type of event, check out the dedicated sections on all supported Event types:\n\n**1. Sync Events**\n - [HTTP](/docs/microservices-framework/event-sources/event-types/http-events)\n - [Apollo Graphql](/docs/microservices-framework/event-sources/event-types/graphql-events)\n\n**2. Async Events**\n- [Cron](/docs/microservices-framework/event-sources/event-types/cron-events)\n- [Kafka](/docs/microservices-framework/event-sources/event-types/kafka-events)\n\n\n" }, "docs/microservices-framework/event-sources/overview.md": { "content": "---\ntitle: Eventsources\n---\n# About Eventsources\n\n Eventsources in Godspeed framework captures event and allows you to define entry or trigger points of application. For ex. the `type: express` eventsource will allow you to expose your application through REST API or a `type: cron` eventsource will allow to schedule a recurring call to a workflow. \n \n The eventsources listen on the incoming events. They process incoming event as per the middleware set by you, including [authentication](../authentication/overview.md). Finally, they transform it to Godspeed's standard `GSCloudEvent` object, which is then made available to the event handlers and subsequent child workflows. \n \n To have a look at supported eventsources and understanding their implementation, refer [Godspeed's gs-plugins mono-repo](https://github.com/godspeedsystems/gs-plugins). For ex. **[Kafka](https://github.com/godspeedsystems/gs-plugins/tree/main/plugins/kafka-as-datasource-as-eventsource#godspeed-plugin-kafka-as-datasource-as-eventsource)**\n\n\n## Types of eventsources \nBased on functionality and the nature of the information they provide to the system, eventsources can be divided as below.\n\n 1. Eventsource\n - It can act only as an eventsource\n - has its own client initialization\n - Eg,. Express\n 2. DataSource as an eventsource\n - It can act as eventsource as well as a datasource\n - shares the client with the corresponding datasource\n - eg., Kafka, RabbitMQ\n\n## How to use an eventsource in Godspeed project?\n\nGodspeed framework adopts a pluggable approach that empowers you to define eventsources effortlessly. Our framework provides an interface that caters to diverse eventsource needs. Here's a glimpse into the exceptional eventsource plugins crafted by our core framework team.\n\n\n## How to add plugin in your project?\nYou can use `godspeed plugin add` command to browse and install plugins which are published and maintained by Godspeed.\n\n```bash\n$ godspeed plugin add\n```\n\n```bash\n ,_, ╔════════════════════════════════════╗\n (o,o) ║ Welcome to Godspeed ║\n ({___}) ║ World's First Meta Framework ║\n \" \" ╚════════════════════════════════════╝\n\n? Please select godspeed plugin to install: (Press to select, to move rows)\n┌──────┬────────────────────────────────────────┬────────────────────────────────────────────────────────────────────────────────┐\n│ │ Name │ Description │\n├──────┼────────────────────────────────────────┼────────────────────────────────────────────────────────────────────────────────┤\n│ ❯◯ │ cron-as-eventsource │ Cron as eventsource plugin for Godspeed Framework │\n├──────┼────────────────────────────────────────┼────────────────────────────────────────────────────────────────────────────────┤\n│ ◯ │ aws-as-datasource │ aws as datasource plugin for Godspeed Framework │\n├──────┼────────────────────────────────────────┼────────────────────────────────────────────────────────────────────────────────┤\n│ ◯ │ excel-as-datasource │ excel as datasource plugin for Godspeed Framework │\n├──────┼────────────────────────────────────────┼────────────────────────────────────────────────────────────────────────────────┤\n│ ◯ │ mailer-as-datasource │ mailer as datasource plugin for Godspeed Framework │\n├──────┼────────────────────────────────────────┼────────────────────────────────────────────────────────────────────────────────┤\n│ ◯ │ kafka-as-datasource-as-eventsource │ kafka as datasource-as-eventsource plugin for Godspeed Framework │\n└──────┴────────────────────────────────────────┴────────────────────────────────────────────────────────────────────────────────┘\n\n```\n\nYou can directly install a plugin by specifying its package name:\n\nExample:\n\n```bash\n$ godspeed plugin add @godspeedsystems/plugins-cron-as-eventsource\n```\n\n```bash\n$ godspeed plugin add @godspeedsystems/plugins-kafka-as-datasource-as-eventsource \n```\n\n**To learn more about Event sources in Godspeed, please watch the video provided below…**\n\n\n\n
\n\n
\n" }, "docs/microservices-framework/event-sources/validations/schema-validation.md": { "content": "# JSON Schema validation\n\nThe Framework provides request and response schema validation \n\n### Request schema validation\n\nWe have the ability to define inputs and their types in our request schema, such as path parameters, query parameters, and request body. This allows the framework to validate whether the API has received the specified inputs in the expected types.\n\nWhenever an API is triggered, AJV (Another JSON Schema Validator) verifies the request schema against the provided inputs. If the defined schema matches the inputs, it allows the workflow to execute. Otherwise, it throws an error with a status code of 400 and a descriptive message indicating where the schema validation failed.\n\n### Sample Request Schema\n\n#### Request Schema with body\n\n```yaml\nhttp.post./helloworld:\n fn: helloworld\n body:\n content:\n application/json:\n schema:\n type: object # type of the request body\n required: [name] # Here make the properties mandatory ,can add multiple variable names\n properties: # we can add properties that we get in body\n name: # property name\n type: string # property type\n```\n\nIn the provided example, the `type: object` indicates the type of body received. Following that, the required variables are listed, specifying the properties of the body object that must be present. Similarly, it is also possible to define the types of the property variables present in the body.\n\n#### Failed request schema validation\n\n![req_validation](https://res.cloudinary.com/dzdcjchdc/image/upload/v1704546298/Screenshot_from_2024-01-06_18-31-32_tref8f.png)\n\n#### Request Schema with params\n\n```yaml\nhttp.post./helloworld/:path_params:\n fn: helloworld\n params: # Params begin from here\n - name: path_params # name of the parameter\n in: path # type of parameter path/query\n required: true # mandatory check for path parameter\n schema:\n type: string # type of path parameter\n - name: query_params \n in: query # query parameter\n required: true\n schema:\n type: string\n```\n\n\n#### Sample with pattern\n\n```yaml\n\nhttp.post./helloworld:\n fn: helloworld\n body:\n content:\n application/json:\n schema:\n type: object\n required: [name]\n properties:\n name:\n type: string\n format: date\n pattern: \"[0-9]{4}-[0-9]{2}-[0-9]{2}\" # setting patern in schema\n```\n\n### Response schema validation\n\nJust like request schema validation, there's also response schema validation in place. In this process, the framework checks the response type, validates the properties of the response, and ensures they align with the specified types.\n\nThe process of response schema validation involves storing the response schema, enabling the workflow to execute, and checking the response body along with its properties for validation.\n\nResponse schema validation includes two cases\n\n- Failure in Workflow Execution\n- Successful Workflow Execution but Fails in Response Schema Validation\n\n\n### Sample Response Schema\n\n```yaml\n responses:\n 200:\n content:\n application/json:\n schema:\n type: object #Types of response\n required: [name] # setting mandatory variables\n properties:\n name:\n type: string #Type of property\n\n```\nIf the response schema validation fails api return with `500` `internal server error`\n\n\n#### Failed sample of response schema validation\n\n![case1](https://res.cloudinary.com/dzdcjchdc/image/upload/v1704548714/Screenshot_from_2024-01-06_19-14-52_uwwbek.png)\n\n\n:::tip In the case of failed request schema validation, the APIs respond with a status of `400` and a message indicating a \"bad request.\" Conversely, if the response schema validation encounters an issue, the APIs return a status of `500` along with an \"Internal Server Error\" message.\n:::\n\n### Event with response and request schema validation\n\n```yaml\nhttp.post./helloworld:\n fn: helloworld\n params:\n - name: path_params\n in: path\n required: true\n schema:\n type: string\n - name: query_params\n in: query\n required: true\n schema:\n type: string\n\n body:\n content:\n application/json:\n schema:\n type: object\n required: [name]\n properties:\n name:\n type: string\n responses:\n 200:\n content:\n application/json:\n schema:\n type: object\n required: [name]\n properties:\n name:\n type: string\n```" }, "docs/microservices-framework/event-sources/validations/validation-error.md": { "content": "# Validation Error\n\n- A validation error occurs when data fails to meet the defined criteria or constraints during the validation process.\n\n## Request and Response Validation\n- Verifying that incoming API requests have the required parameters and that those parameters meet specific criteria like data types. If the specified criteria are not met, it results in a request validation error. \n- For more info about Request Validation and its applications, refer [this](/docs/microservices-framework/event-sources/validations/schema-validation.md#request-schema-validation).\n- Similarly, When an API response does not meet the expected criteria, it results in a response validation error.\n:::tip Note\n- We utilize the AJV library for validating both request and response data, and the response format adheres to the standard AJV error format.\n- The `on_request_validation_error` and `on_response_validation_error` handlers are used to override the default errors thrown by the framework (specifically, schema validation errors) and allow developers to customize errors based on their requirements.\n:::\n\n### on_request_validation_error\n\n- To customize the error response in cases where request schema validation fails, you can utilize the `on_request_validation_error` handler. Demonstrating its use in the below example\n\n### Example\n\n```yaml\nhttp.get./validation:\n fn: test_validation\n on_request_validation_error: on_request_validation #can be fn path, or a series of tasks\n params:\n - name: num_1\n in: query\n required: false\n schema:\n type: number\n - name: num_2\n in: query\n required: true\n schema:\n type: number\n\n responses:\n 200:\n content:\n application/json:\n schema:\n type: object\n \n```\n\n\n```ts title=functions/on_request_validation.ts\nimport { GSContext, PlainObject, GSStatus } from \"@godspeedsystems/core\";\n\nexport default function (ctx: GSContext) {\n const {\n inputs: {\n validation_error\n }\n } = ctx;\n\n // Extracting message from validation_error input\n const message = validation_error?.data?.message || \"Unknown validation error\";\n\n return new GSStatus(false, 400, 'Failed', { message });\n}\n \n```\n\n\n\n```ts title=functions/test_validation.ts\nimport { GSContext, PlainObject, GSStatus } from \"@godspeedsystems/core\";\n\nexport default function (ctx: GSContext) {\n const {\n inputs: {\n data: {\n query\n }\n }\n } = ctx;\n\n const responseMessage = `This is number two ${query?.num_2 ?? ''}`;\n return new GSStatus(true, 200, 'OK', responseMessage);\n}\n```\nResponse\n\n- A: Default Error Format\n\"response_error\"\n\n- B: Customized Error\n\"event\n\n\n### on_response_validation_error\n\n:::tip Note\n- The framework deals with **both request and response errors the same way**, except for the error codes and keys.\n- For request validation errors, the associated error code is 400, and the key used is `on_request_validation_error`.\n- In the case of response validation errors, the corresponding error code is 500, and the key employed is `on_response_validation_error`.\n:::" }, "docs/microservices-framework/event-sources/write-apis.md": { "content": "# Writing APIs & Events\n\nIn Godspeed, **every API is defined as an event**, and handled by a corresponding function. This unified event-driven structure works for both REST and non-REST event sources (HTTP, WebSocket, Kafka, etc.).\n\nThis guide explains how to write new APIs in three clear steps:\n\n---\n\n## Step 1: Setup an Event Source\n\nAn **event source** is a mechanism that listens for incoming requests or events. Godspeed supports HTTP (via Express), GraphQL, Fastify, Cron and Kafka.\n\n### 🔹 By Default: HTTP via Express\n\nWhen you create a new project with:\n\n```bash\ngodspeed create my-project\n```\n\nIt comes pre-configured with **Express** as the default event source.\n\nCheck: `src/eventsources/http.yaml`\n\n```yaml\ntype: express\nport: 3000\ndocs:\n endpoint: api-docs\n```\n\nThis allows you to start writing HTTP APIs immediately without additional setup.\n\n---\n\n### 🔹 Optional: Add Another Event Source\n\nIf you want to consume events from other protocols (e.g., Kafka, WebSocket, Cron) use:\n\n```bash\ngodspeed plugin add\n```\n\nSelect the required event source like:\n\n* `kafka-as-datasource-as-eventsource`\n* `cron-as-eventsource`\n* `graphql-as-eventsource`\n\nThen configure it in the `src/eventsources/` folder.\n\n---\n\n## Step 2: Define the Schema for your API\n\n### Event Schema\n\nTo define an api endpoint, first you need to write an Event Schema. This schema is a structured YAML configuration that follows the OpenAPI specification, allowing you to define every detail of how the event should behave. All events adhere to a standard structure, which is one of the core design principles of Godspeed, regardless of their source or protocol.\n\n**It involves specifying:**\n\n- The name/topic/URL of the event\n- The event handler Workflow (fn)\n- Input and Output schema\n- [Validation error handling](/docs/microservices-framework/event-sources/validations/schema-validation)\n- [Authorization checks](/docs/microservices-framework/authorization/overview.md)\n\nBy writing an event schema, you provide a blueprint that defines how an incoming request or message should be handled, making your API endpoints easy to manage and highly configurable.\n\n\n### The generic Event Schema\nGodspeed follows [Schema Driven Development & Single Source of Truth](../introduction/guard-rails.md#1schema-driven-development), [Configure Over Code](../introduction/guard-rails.md#2configure-over-code) and [Modular Architecture](../introduction/guard-rails.md#4-decoupled-architecture) approach in \n- Listening to events from various sources and acting upon them.\n- Generating API documentation (Swagger) and other schemas like Graphql\n\nThe meta-framework follows a generic schema for defining events - their inputs, outputs, swagger specs, with validations, authentication, authorization etc. \n\n```yaml\nhttp.get./greet: #The initial line depicts a fusion of the event, the employed method, and the path associated with the event.\n fn: function_greet #Required. The 'fn' key receives the function name located in 'src/functions' and forwards the accompanying parameters. \n \n #optional configurations\n #Swagger components\n body: #same as requestBody in Swagger world\n params: #same as swagger `parameters` schema\n responses: #same as swagger `responses` schema\n id: # swagger. if not provided, when generating swagger, this is generated from the URI of the event by default\n operationId: # swagger if not provided, check if `id` is set. If that is also not set, use the summary to generate the operationId\n summary:\n tags: # swagger if you give `tags` array in schema of event, framework uses that to add tags to your generated spec. Else it uses the path and name of the file containing the event as tags. For ex. _ \n \n #Other non-swagger components (optional)\n authn: #custom authentication. Currently plugins support JWT. Can be customized\n authz: #your custom authz workflow\n on_request_validation_error: #when validation fails\n on_response_validation_error: #when validation fails\n log: #Open Telemetry compliant log attributes which help debug and search through logs better\n attributes:\n```\nLets understand the first line from the above snippet `http.get./greet`.\n\n`http`: Protocol http eventsource (can be any)\n\n`get` : method (depends on the eventsource used. Can be topic for Kafka)\n\n`/helloworld`: endpoint (In case of http and graphql sources. Can be groupId in case of Kafka for ex.)\n\nWe are exposing an endpoint with a `get` method on `http` protocol. This endpoint is calling an eventhandler called `helloworld` in the second line. Event handlers can be functions written in typescript, javascript or yaml workflows in Godspeed's DSL format. In the above example the helloworld function exists in `src/functions` directory. \n\n### Key Differences between a Sync and Async Event Schema\n\nWhen switching between eventsources, the event schema undergoes significant changes.\n- The first line is changed for each protocol:\n\n In the case of sync events or HTTP events, the start line includes the eventsource name, method, and path. \n\n However, for async events, the start line combines the source name, topic and group ID (for Kafka), or schedule (for Cron).\n\n- Async events like Kafka do not have responses, authentication and authorization fields in schema.\n\n- Cron events do not have any input.\n\n:::tip Note\nYou can apply multiple compatible eventsource instances in a URI for ex. `graphql & http.get./greeting`\n:::\n\n### Example HTTP Event Schema\nExample: `src/events/helloworld.yaml`\n\n```yaml\nhttp.get./helloworld:\n fn: helloworld\n authn: false\n params:\n - name: name\n in: query\n required: true\n schema:\n type: string\n responses:\n 200:\n content:\n application/json:\n schema:\n type: string\n```\n\n### Anatomy of an Event\n\n| Field | Purpose |\n| ---------------------- | --------------------------------------------------------------- |\n| `http.get./helloworld` | Event key – REST method and route |\n| `fn` | The function (event handler) to invoke |\n| `params` | Query/path/header parameters, as per Swagger format |\n| `responses` | API response schema definition for documentation and validation |\n| `authn` | Set to `true` or `false` to require authentication |\n\nThis schema controls how Swagger UI documents the API and how the framework validates inputs.\n\n---\n\n## Step 3: Write the Event Handler Function\n\nThe event handler function processes the request. It can be written in TypeScript or Javascript.\n\n### 🔹 Example: TypeScript Handler Function\n\n`src/functions/helloworld.ts`\n```ts\nimport { GSCloudEvent, GSContext, PlainObject, GSStatus } from \"@godspeedsystems/core\";\nexport default function (ctx: GSContext, args: PlainObject) {\n const {\n inputs: {\n data: {\n query // query parameters from rest api\n // params,//path parameters from endpoint url\n // body, // request body in case of http and graphql apis, event data in case of message bus or socket\n // user, // user payload parsed from jwt token\n // headers //request headers in case of http and graphql apis\n }\n }, \n }= ctx;\n \n return new GSStatus(true, 200, 'OK', 'Hello ' + query.name); \n}\n```\n\n:::tip\nIn Godspeed, your function gets input in a standard JSON format and returns output in a standard JSON format, independent of the eventsource through which this function is triggered. Eventsource could be Express, Fastify, Apollo Graphql or event message bus like Kafka, RabbitMQ or socket message. This means Godspeed has a unified way to deal with all eventsources, giving you modular architecture and re-uasability of your functions.\n:::\n\nClick the link below to explore the complete anatomy of an event handler function written in TypeScript.\nIt illustrates how to access inputs, use the GSContext, interact with datasources, and return responses correctly.\n\n👉 [Full Example: Understanding Godspeed Function Structure](/docs/microservices-framework/workflows/native-language-functions.md#example-typescript-function)\n\n\n### Test Your API\n\nRun the service:\n\n```bash\ngodspeed serve\n```\n\nVisit the Swagger UI at:\n\n```\nhttp://localhost:3000/api-docs\n```\n\nTry the `/helloworld` endpoint. Provide a `name` in the query, and observe the JSON response.\n\n---\n\n## Summary\n\n| Step | Action | File Path |\n| ---- | -------------------------------------- | ------------------------------------- |\n| 1 | Confirm eventsource (Express or other) | `src/eventsources/.yaml` |\n| 2 | Define your event schema | `src/events/.yaml` |\n| 3 | Write the event handler function | `src/functions/.ts` |\n\n---\n\n### Extending Further\n\n* Use `datasources` (Axios, AWS, Prisma) in the handler via `ctx.datasources`\n* Chain multiple tasks using YAML workflows instead of writing code\n* Auto-generate CRUD APIs via `godspeed gen-crud-api`\n---\n\n**To get a quick understanding of Event schema, please watch the video provided below…**\n\n
\n\n
" }, "docs/microservices-framework/guide/advance-guide.md": { "content": "# Advance Guide\n\n### Steps for creating a sample blog project.\n\nLet's create a project which includes a Prisma schema(`mongo.prisma`) for creating a blog app using mongodb.\n\n1. Create a project using Godspeed CLI with below command:\n\n```bash\ngodspeed create blog-app --from-example mongo-as-prisma # blog-app is the name of the app\n```\n### Checkout other examples\n\n:::tip **In this sample project, we have utilized a MongoDB database, but you have the flexibility to choose any other database that Prisma supports and that meets your requirements [MySQL](https://github.com/godspeedsystems/godspeed-codesandbox/tree/gs-projects-examples/Project-Examples/sql), [PostgreSQL](https://github.com/godspeedsystems/godspeed-codesandbox/tree/gs-projects-examples/Project-Examples/postgres) etc**\n:::\n\nFramework will give you below folder structure.\n```\n ├── config\n │ ├── custom-environment-variables.yaml\n │ |\n | ├── default.yaml \n | \n ├── eslintrc.json \n |\n ├── src\n ├── datasources\n │ ├── types\n │ | └── prisma.ts\n | |\n │ └── mongo.prisma\n │\n ├── events\n | |\n │ └── helloworld.yaml\n |\n ├── eventsources\n │ ├── types\n │ | └── express.ts\n | |\n │ └── http.yaml\n |\n └── functions\n │ |\n | └── helloworld.yaml \n |\n └── mappings\n | |\n | └── index.yaml \n |\n └── plugins\n |\n └── sample.js \n```\n\n:::tip Note\n- The framework generates different folders like [config](/docs/microservices-framework/config-and-mappings/config.md),[datasources](/docs/microservices-framework/datasources/overview.md) , [events](/docs/microservices-framework/event-sources/event-schema.md), [eventsources](/docs/microservices-framework/event-sources/overview.md), [functions](/docs/microservices-framework/workflows/overview.md), [mappings](/docs/microservices-framework/config-and-mappings/mappings.md), [plugins](/docs/microservices-framework/inline-scripting/script-plugins.md),etc\n- The `eslintrc.json` file includes a curated list of recommended plugins that can be incorporated into the project.\n:::\n\n\n2. Navigate to you project folder\n\n```bash\ncd blog-app\n```\n\n3. Open the project in vscode using below command:\n\n```bash\ncode .\n```\n\n4. Now to setup your database please follow the steps provided in README.md file of your blog-app project.\n\n\n#### Prisma supports wide range of databases\n* [MySQL](https://dev.mysql.com/doc/)\n* [PostgreSQL](https://www.postgresql.org/docs/)\n* [SQLite](https://www.sqlite.org/docs.html)\n* [CockroachDB](https://www.cockroachlabs.com/docs/cockroachcloud/quickstart)\n* [MariaDB](https://mariadb.org/documentation/)\n* [MongoDB](https://www.mongodb.com/docs/guides/atlas/cluster/)\n\n\nPrisma provides a unified and versatile database experience, enabling you to connect to diverse databases through a single schema definition. This ensures a consistent schema for interacting with various databases and generates a unified API, streamlining development for flexibility and ease. Whether you opt for MySQL, PostgreSQL, SQLite, or other supported databases, Prisma's unified approach simplifies both database management and application development.\n\n5. Open terminal in vscode and enter the below command\n\n```bash\ngodspeed prisma prepare\n```\n**This command will generate the prisma client and will sync the database with prisma schema**\n\n:::tip Note\nWe configure [swagger specs](/docs/microservices-framework/event-sources/event-types/http-events.md#swagger-specs) in src/eventsources/http.yaml\n:::\n\n6. 3000 is the default port number,if you want to provide your custom port number, you can modify the port number from **\"./src/eventsources/http.yaml\"**\n\n```yaml\ntype: express\nport: 3000\n```\n\n7. Now to generate the CRUD API'S enter the below command\n\n```bash\ngodspeed gen-crud-api\n```\n**This command will generate the crud apis based on the sample prisma schema provided at ./src/datasources/mongo.prisma**\n\nCommand generates the below respective folders and files in the Events and Functions.\n\n```bash\n .\n ├── src\n ├── datasources\n │ ├── types\n │ | └── prisma.ts\n | |\n │ └── mongo.prisma\n |\n ├── events\n | ├── post.yaml\n │ └── user.yaml\n |\n ├── eventsources\n │ ├── types\n │ | └── express.ts\n | |\n │ └── http.yaml\n |\n └── functions/com/biz/mongo\n ├── post\n | ├── create.yaml\n | ├── delete.yaml\n | ├── one.yaml\n | ├── search.yaml\n | └── update.yaml\n |\n └── user\n ├── create.yaml\n ├── delete.yaml\n ├── one.yaml\n ├── search.yaml\n └── update.yaml\n```\n\n8. Run `godspeed serve` to start the development server.\n\n```bash\ngodspeed serve\n```\n9. Go to [http://localhost:3000/api-docs](http://localhost:3000/api-docs) to see the API endpoints in the swagger.\n\n\"swagger\n\n\n\n10. Create a user in the Mongodb by giving the proper request body to the post API in the swagger and execute it.\n\"swagger\n\n\n\n11. An appropriate response will be returned on successful execution.\n\"swagger\n\n\n\n**Voila!** Your API backend is up and running. You can use Postman or swagger to test your API's.\n\nHappy building with Godspeed!\n" }, "docs/microservices-framework/guide/first-project.md": { "content": "---\nid: first-project\ntitle: Your first Godspeed Project\ndescription: understanding your first project(or service), running the development server, accessing Swagger UI and testing a basic API.\nkeywords: [Godspeed service, project (service) creation, running project (service) locally, Swagger UI, API testing, helloworld, guide, tutorial]\n---\n\nFollow these steps to set up and launch your new Godspeed project:\n\n1. **Create a new project:**\n ```bash\n godspeed create my-project\n ```\n\n2. **Navigate to the project directory:**\n ```bash\n cd my-project\n ```\n\n3. **Start the development server:**\n ```bash\n godspeed serve\n ```\n\n### Accessing and Testing Your API with Swagger UI\n\nGodspeed automatically generates interactive API documentation using Swagger UI.\n\n1. **Access Swagger UI:**\n The Swagger UI is typically available at:\n ```plaintext\n http://:/`\n ```\n By default, this is:\n ```plaintext\n http://localhost:3000/api-docs\n ```\n\n If you need to customize the default port (`3000`) or the Swagger endpoint (`/api-docs`), you can modify the `./src/eventsources/http.yaml` file.\n\n2. **Test the `/helloworld` API:**\n\n * In the Swagger UI, locate the `/helloworld` endpoint.\n * Click the **`Try it out`** button.\n * You will be prompted to fill in a `name` parameter. Enter a name (e.g., \"John\") and send the request.\n * The server will return the following response:\n ```\n Hello `John`\n ```\n\n### Understanding Your First Godspeed Project\n\nLet's dive into the structure and core concepts of your newly created Godspeed project.\n\n#### Project Scaffolding\n\nGodspeed generates a well-organized project structure:\n\n * The framework creates various folders inside, including:\n * [`config`](/docs/microservices-framework/config-and-mappings/config.md)\n * [`datasources`](/docs/microservices-framework/datasources/overview.md)\n * [`events`](/docs/microservices-framework/event-sources/event-schema.md)\n * [`eventsources`](/docs/microservices-framework/event-sources/overview.md)\n * [`functions`](/docs/microservices-framework/workflows/overview.md)\n * [`mappings`](/docs/microservices-framework/config-and-mappings/mappings.md)\n * The `eslintrc.json` file contains a curated list of recommended ESLint plugins for your project.\n * Swagger specifications for your HTTP endpoints are configured in `src/eventsources/http.yaml`.\n\n:::tip\nFor a deeper understanding of the project's scaffolding structure, refer to the [Walkthrough section](/docs/microservices-framework/guide/walkthrough#moving-forward)\n:::\n\n#### Why the `/helloworld` API Requires a Name\n\nThe `/helloworld` API endpoint asks for a `name` parameter because it's configured to require it as part of the query string.\n\nLet's examine the event schema for this API in `src/events/helloworld.yaml`:\n\n ```yaml\n http.get./helloworld: # `http` server listening via `get` method on `/helloworld` endpoint\n fn: helloworld # the function handler to be called for this endpoint, available in `src/functions`\n params: # JSON-Schema of API parameters like query, headers, path params. Note: This is set as per Swagger standard's `parameters` syntax\n - name: name # This is our name query param\n in: query # Notice the in: query, it can be `path` or `headers` as well\n required: true # true means `name` parameter is required\n schema:\n type: string\n responses: # JSON-Schema of API responses for different status codes. Note: This is set as per Swagger standard's `responses` syntax\n 200:\n content:\n application/json:\n schema:\n type: string\n ```\n\nIn this `helloworld.yaml` file:\n\n * **`params`**: This section describes the input the API expects. Here, it expects a `name` parameter.\n * **`name: name`**: Specifies that the query parameter is named `name`.\n * **`in: query`**: Indicates that the `name` parameter should be included in the URL's query string (e.g., `/helloworld?name=John`). It can also be `path` or `headers`.\n * **`required: true`**: The `name` parameter is mandatory for the API to function.\n * **`schema: { type: string }`**: The `name` parameter must be a string, enforcing text input.\n\n:::tip\nIn the Godspeed meta-framework, every API (REST or GraphQL) and message (websocket, cron, message bus) is referred to as an **`event`**. All events trigger workflows/functions, which act as event handlers (see the `fn:` instruction in the YAML above). Synchronous events return a response, while asynchronous events do not.\n\nThis naming convention might be new to you. While the broader developer community typically refers to only `async events` (like Kafka or websocket messages) as `events`, Godspeed considers both synchronous APIs (REST, GraphQL) and asynchronous events (Message bus, websocket, cron) as `events`.\n:::\n\n#### Testing API Input and Output Validation\n\nGodspeed provides built-in validation for data sent to and from your APIs, ensuring data integrity.\n\n1. **Test missing `name` parameter:**\n Open your browser and navigate to `localhost:3000/helloworld`. Alternatively, run `curl -i localhost:3000/helloworld` from your terminal.\n This will return a `400` error, as you haven't provided the required `name` query parameter:\n\n ```json\n {\n \"validation_error\": {\n \"success\": false,\n \"code\": 400,\n \"message\": \"request validation failed.\",\n \"error\": \"must have required property 'name' in query\",\n \"data\": {\n \"message\": \"The API cannot be executed due to a failure in request params schema validation.\",\n \"error\": {\n \"instancePath\": \"\",\n \"schemaPath\": \"#/required\",\n \"keyword\": \"required\",\n \"params\": {\n \"missingProperty\": \"name\"\n },\n \"message\": \"must have required property 'name' in query\"\n }\n }\n }\n }\n ```\n\n2. **Test with `name` parameter:**\n Now, navigate to `localhost:3000/helloworld?name=mastersilv3r`. This should work correctly:\n\n ```\n Hello mastersilv3r\n ```\n\n### API Collections\n\nGodspeed helps you easily work with API clients.\n\n * **Swagger Collection:**\n Access the automatically generated Swagger collection for your project from the `/docs` folder within your project.\n\n * **Postman Collection:**\n To get a Postman Collection, simply import the Swagger file from `src/docs` into Postman.\n\n\n### Video Tutorial - Short\nThere is a longer and detailed introduction video as well, below on this page.\n\n
\n \n
\n\n\n> If you want some pre-made examples please check the [examples repository](https://github.com/godspeedsystems/gs-node-templates)\n\n### Video Tutorial - Longer and in depth\n\n**Walkthrough the Loan Origination System project made using our meta framework**\n\n
\n\n
\n" }, "docs/microservices-framework/guide/get-started.md": { "content": "---\nid: get-started\ntitle: Getting Started with Godspeed\ndescription: A comprehensive guide to install the Godspeed Meta-Framework, creating your first project(aka service), running the development server, accessing Swagger UI, and testing a basic API.\nkeywords: [Godspeed, Meta-Framework, installation, setup, project (service) creation, CLI, running project (service) locally, Swagger UI, API testing, helloworld, guide, tutorial]\nslug: /get-started\n---\n\nThis guide provides a step-by-step guide to install and get started with Godspeed Meta-Framework. It covers the prerequisites and installation process, both manual and using easy installation scripts. You will learn how to create your first Godspeed project or service and run the development server.\n\n\n\n## Install Godspeed\n\nThe following setup script installs all required prerequisites and the Godspeed runtime components automatically. This simplifies the onboarding process for new users by avoiding manual setup of individual dependencies.\n\n### Windows Users\n\nOpen Powershell as Administrator and run\n```bash\nInvoke-WebRequest -Uri \"https://raw.githubusercontent.com/zero8dotdev/install-godspeed-daemon/main/CompleteInstall.ps1\" -OutFile \"install.ps1\"; Start-Process powershell -ArgumentList \"-File .\\install.ps1\" -Verb RunAs\n```\n\n### Mac/Linux Users\n\n```bash\ncurl -fsSL https://raw.githubusercontent.com/zero8dotdev/install-godspeed-daemon/main/CompleteInstall.sh | bash\n```\n\n### ✅ The Script Will Install:\n\n| Component | Purpose |\n| ----------------- | ------------------------------------------------------------ |\n| `nvm` | Node Version Manager - helps switch between Node.js versions |\n| `node` | JavaScript runtime (installed via nvm) |\n| `npm` | Node package manager (comes with Node.js) |\n| `pnpm` | Efficient alternative to npm/yarn for managing dependencies |\n| `corepack` | Node’s built-in tool for package manager handling |\n| `git` | Version control system - required to clone repositories |\n| `godspeed` CLI | Command-line interface to scaffold, run and manage services |\n| `godspeed-daemon` | Core Godspeed runtime engine that executes workflows |\n---\n\n## Manual Installation\n\n If you prefer manual setup, follow the steps below:\n\n **1. Ensure the Prerequisites are installed:**\n\n - nvm (Node Version Manager)\n - node.js (v18 or higher, installed via nvm)\n - git\n - corepack (comes with Node.js)\n - pnpm (can be enabled via corepack: corepack enable pnpm)\n \n\n **2. Install the Godspeed framework globally:**\n\n ```bash\n npm install -g @godspeedsystems/godspeed\n ```\n\n## Verify Installation\n\nConfirm successful installation:\n\n```bash\ngodspeed --version\n```\n\n## Creating and Running Your First Project\nFollow these steps to set up and launch your new Godspeed project:\n\n1. **Create a new project:**\n ```bash\n godspeed create my-project\n ```\n\n2. **Navigate to the project directory:**\n ```bash\n cd my-project\n ```\n\n3. **Start the development server:**\n ```bash\n godspeed serve\n ```\n\n### Accessing and Testing Your API with Swagger UI\n\nGodspeed automatically generates interactive API documentation using Swagger UI.\n\n1. **Access Swagger UI:**\n The Swagger UI is typically available at:\n ```plaintext\n http://:/`\n ```\n By default, this is:\n ```plaintext\n http://localhost:3000/api-docs\n ```\n\n If you need to customize the default port (`3000`) or the Swagger endpoint (`/api-docs`), you can modify the `./src/eventsources/http.yaml` file.\n\n2. **Test the `/helloworld` API:**\n\n * In the Swagger UI, locate the `/helloworld` endpoint.\n * Click the **`Try it out`** button.\n * You will be prompted to fill in a `name` parameter. Enter a name (e.g., \"John\") and send the request.\n * The server will return the following response:\n ```\n Hello `John`\n ```\n### Troubleshooting Common Issues\n\nHere are solutions to some common issues you might encounter:\n\n1. **\"Running scripts is disabled on this system\"**\n\n * **Solution:** Run PowerShell as Administrator and execute:\n ```powershell\n Set-ExecutionPolicy RemoteSigned\n ```\n\n2. **\"Port Already in Use\"**\n * **Error:**\n ```\n Error: Port 3000 is already in use\n ```\n * **Solution:** Stop any other services using port `3000`, or modify the port in `src/eventsources/http.yaml`.\n\n### Additional Resources\n\n * **Godspeed CLI Help:**\n To see a list of available Godspeed commands, use:\n\n ```bash\n godspeed --help\n ```\n\n Refer to [the full CLI spec](/docs/microservices-framework/CLI.md) for more detailed information, including [how to add plugins for eventsources and datasources](/docs/microservices-framework/CLI#plugin-add-to-install-godspeed-plugin)\n\n\n\n\n\n" }, "docs/microservices-framework/guide/walkthrough.md": { "content": "# Diving Deeper\n\nIn the previous section we got an understanding on how to setup a meta-framework based project for your local development and create a new project via cli commands. This section is dedicated to providing hands-on practice in constructing comprehensive backend services utilizing the meta framework and its associated plugins. Additionally, it aims to facilitate a thorough understanding of all the fundamental concepts underpinning the Godspeed framework.\n\n### Video Playlist for Detailed Walkthrough\n\n> Don't miss! Video Playlist of detailed guide to eventsources, events, functions and datasources: [Watch here](https://www.youtube.com/watch?v=GdJ0ghpQ7oA&list=PLRuRJ3PaaJ7ti9bfStNTXqsxxW9wvwA_H)\n\n### As we move forward\n\nDo ckeck the [tenets](../introduction/tenets.md), [design principles](../introduction/design-principles.md) and [guardrails](../introduction/guard-rails.md) baked into the meta-framework as part of its design - helping teams develop microservices based applications with best practices. \n\n#### The three pillars\nHave you seen the [three pillars of abstraction](../introduction/design-principles.md#three-fundamental-abstractions) in the Meta-Framework?\n\n- [Eventsources and event schemas](../event-sources/overview.md)\n- [Event handler functions (Pure functions)](../workflows/overview.md) \n- [Datasources](../datasources/overview.md)\n\nYou may want to check why you should use the [4th generation](https://godspeed.systems/blog/godspeed-fourth-generation-programming-framework) Meta-Framework versus creating apps using pure 3rd generation frameworks like Nodejs or Django stack from scratch.\n\n![img The fourth generation approach of Meta Framework](../../../static/img/four_generations_proramming.png)\n\n> Note: The Meta-Framework currently supports Nodejs based ecosystem\n\n### Moving forward\n:::tip\nAs mentioned in the [getting started](./get-started.md) section all APIs and async events are together referred to as `events`. And hence any source like Express, Fastify, Kafka, Apollo Graphql are called [EventSources](../event-sources/overview.md).\n:::\nIn [getting started](./get-started.md) section we created a project that includes a simple 'hello world' example which uses an http Express [eventsource](../event-sources/overview.md), [events](/docs/microservices-framework/event-sources/event-schema.md) and [event handler functions](/docs/microservices-framework/workflows/overview.md). \n\n\nNow we can move into further details.\n\n:::tip\nIn order to see detailed examples and documentation of eventsources, events and event handlers please visit [Express HTTP Eventsource Plugin documentation](../event-sources/event-source-plugins/Express%20Http%20Eventsource.md). \n:::\n\n### Event Configuration\n\n> All event definitions are stored in `src/events` folders in YAML files. Each YAML file can have multiple events or a single event defined. The folder structure within `src/events` determines the categories (grouping of your API) that you will see in the generated Swagger and Postman collection. You can group events in any files and nested folders as per your need.\n\nNow lets understand how our `helloworld` api endpoint is working behind the scene\n\n**Open `./src/events/helloworld.yaml`**\n\n```yaml\n\"http.get./helloworld\":\n fn: helloworld # event handler function - the logic resides here\n authn: false\n params: #same as Swagger params.\n - name: name\n in: query\n schema:\n type: string\n```\n\nLets understand the first line from the above snippet `http.get./helloworld`. \n\n#### `http`: Protocol http eventsource\n\n#### `get` : method\n\n#### `/helloworld`: endpoint\n\nWe are exposing an endpoint with a `get` method on `http` protocol. This endpoint is calling a function `fn`: `helloworld` in the second line of the above code snippet.\n\n> For the sake of brevity, in the above example we have omitted further details of event configuration like the JSON schema of `body, query, headers, responses` and as well `authn, authz` settings among more. \n\n:::tip\nYou can find in-depth introduction to event schema [here](/docs/microservices-framework/event-sources/event-schema).\n:::\n\nThe meta-framework follows Schema Driven Development and Single Source of Truth as its first guardrail. [Check the blog here](https://godspeed.systems/blog/schema-driven-development-and-single-source-of-truth) on reasons why this is an essential practice for any 10x engineering team. It allows you to use the Swagger (aka OAS3) spec which includes JSON schema spec, to define your event schemas as single source of truth. The schema format remains same independently of the eventsource which listens on an event. \n\nFrom your event schemas the meta-framework generates the Swagger specs, Graphql schema (if applicable) and also enable input validation, output validation, authentication, authorization without explicitly writing boilerplate code.\n\nThe [event configuration](/docs/microservices-framework/event-sources/event-schema) uses Swagger and JSON Schema for defining shape of `requestBody, params, query, headers & responses`. The overall event schema has more configurations as well, like `authn, authz`. Further the event schemas in the meta-framework are universally same across all event sources *with only two variations per eventsource* which are\n- First line of the event URI may change based on the type of eventsource. \n ```yaml\n http.{method}.{url}: # http event URI - same for Express, Fastify and Graphql plugins\n kafka.{topic}.{group_id}: # kafka event URI\n cron.* * * * *.Asia/Kolkata: # cron event\n ```\n- Async events don't return response and hence don't need `responses` in the shema. But REST and Graphql events should have a `responses` section, defined in Swagger format. \n\nFor http service like Express, Fastify and Graphql, all events are defined in the same format. This way you learn once and reuse everywhere, independent of the event source which is capturing the event. This is the fourth guardrail of the Meta Framework - Decoupled or Modular Architecture.\n\n\n### Event Handler Functions\n\n[Event handlers](../workflows/overview.md) are **pure functions** which take input as JSON and return output as JSON, independent of the eventsource from which the event originated. This is again another adoption of decoupled architecture approach in the meta framework.\n\n**Sample typescript event handler function**\nHere is how a function handler looks like.\n\n```typescript\nimport { GSContext, PlainObject } from \"@godspeedsystems/core\";\n\nexport default function (ctx: GSContext) {\n const {body, headers, params, query, user} = ctx.inputs.data;\n const name: string = query.name; //name expected in request query\n return {\n data: 'Its working! ' + name, //the data key gets set as the body in API response\n code: 200, //Response status code\n success: true, //success can be true or false\n headers: { //custom headers to be attached from this function call\n custom_response_header: 'something'\n }\n }\n //Note: if you wish to use static typing in returning the response instead of using JSON, you can return GSStatus. More on that below.\n}\n```\n\nWe are importing [GSContext](/docs/microservices-framework/workflows/native-language-functions#gscontext) & [GSStatus](/docs/microservices-framework/workflows/native-language-functions#gsstatus) from core package of meta-framework. Go to their respective section to more about them.\n\n\n\n\nSo far we have seen how can we use Express plugin and also we created an endpoint which returns a response with some code and headers. \n\n### Swagger generation\n\nThe meta-framework gives you an autogenerated Swagger collection and UI to test your API endpoints.\n\nIf you are new to Swagger or in other words Open API Spec 3, along with JSON Schema standard, then we strongly suggest to study that. These are standards for defining schema of HTTP APIs. A Swagger schema can also be imported in a Postman collection, or generated from Postman. \n\nYour Swagger docs are automatically generated and stored in `/docs` folder when the project starts or you make a change to your eventsource or event schemas. The documentation is generated from a combination of settings at Eventsource and events levels.\n\n- **Eventsource level** (refer the `docs` section in http.yaml). This is applicable for Express and Fastify eventsources.\n- **Event level**\n - When you enable authentication on an event, its security scheme is set accordingly in generated swagger. By default authentication is enabled on all events when enabled on eventsource instance level itself.\n - `requestBody`, `params`, `responses` ,`operationId`, `id`, `summary`, `description`, `tags` are also picked up or auto-calculated from event spec.\n\n> See more details on Swagger related configurations in http eventsource and event level both, in [Express Plugin documentation](../event-sources/event-source-plugins/Express%20Http%20Eventsource.md)\n\n### CRUD API generation\nCheckout how to generate CRUD APIs in a [step by step guide](../CRUD_API.md).\n\n### Graphql setup with schema generation\nCheck out the [Apollo Graphql plugin documentation](../event-sources/event-source-plugins/Apollo%20GraphQl%20Eventsource.md) for more details.\n\n### Datasources\n[Datasources](../datasources/overview.md) are any locations which allow to send or retrieve data. These could be another API service, message bus, file system, database, text search engine etc. \n\n### Using Datasource and Eventsource Plugins\nYou can import any datasource or eventsource in your project without writing code, with the available plugins for [eventsources](../event-sources/event-source-plugins/Overview.md) and [datasources](../datasources/list-of-plugins.md). \n\n#### **Eventsource Plugins**\n\nCurrently officially supported eventsources plugins are [Express Server](../event-sources/event-source-plugins/Express%20Http%20Eventsource.md), [Fastify Server](../event-sources/event-source-plugins/Fastify%20Eventsource.md), [cron](../event-sources/event-source-plugins/Cron%20Eventsource.md), [Kafka](../event-sources/event-source-plugins/Kafka%20Eventsource.md) & [Apollo Graphql Server](../event-sources/event-source-plugins/Apollo%20GraphQl%20Eventsource.md)\n\n#### Datasource Plugins\nCurrently officially supported datasource plugins are [Prisma](../datasources/datasource-plugins/Prisma%20Datasource.md), [Mongoose](../datasources/datasource-plugins/Mongoose%20Datasource.md), [Axios](../datasources/datasource-plugins/Axios%20Datasource.md), [Memcached](../datasources/datasource-plugins/Memcached%20Datasource.md), [Kafka](../datasources/datasource-plugins/Kafka%20Datasource.md), [Redis](../datasources/datasource-plugins/Redis%20Datasource.md), [AWS](../datasources/datasource-plugins/AWS%20Datasource.md)\n\n#### Creating Own Plugins\n\nYou can modify our existing plugins or create new plugins from scratch and reuse across your projects. A detailed overview of how to [create eventsource plugins](../event-sources/create-eventsource-plugin.md) and [datasource plugins](../datasources/create-datasource-plugin.md) exists.\n\n### JWT authentication\n\n> The Meta Framework currently supports standardised JWT authentication implementation across [Express](../event-sources/event-source-plugins/Express%20Http%20Eventsource.md), [Fastify](../event-sources/event-source-plugins/Fastify%20Eventsource.md) and [Apollo Graphql plugins](../event-sources/event-source-plugins/Apollo%20GraphQl%20Eventsource.md).\n\nConfigure the eventsource to enable jwt authentication. More detailed information about authentication available [here](../authentication/overview.md)\n\n**Configuring JWT in Express \"./src/eventsources/http.yaml\"**\n> Express, Fastify and Graphql plugins support standard way of handling JWT based authenticatin.\n```\ntype: express\njwt:\n issuer: <%config.issuer%> \n audience: <%config.audience%> \n secretOrKey: <%config.secret%>\n```\n*This enables JWT authentication on all your endpoints for the given eventsource. *\n\n**Disabling authentication on a given endpoint**\nYou can disable authn on any endpoint by setting `authn: false`\n```yaml\n\"http.get./helloworld\":\n fn: helloworld\n authn: false\n```\n\n### Authorization (RBAC, ABAC, PBAC)\nThe meta-framework gives you full freedom to handle authorization [(more details here)](../authorization/overview.md) based on RBAC, ABAC or PBAC, in a generic way, independent of the event source. You can integrate any IAM or policy engine to use with the meta-framework. \n\n**Authorization at eventsource level**\nAuthorization can be a list of YAML instructions or just one function which could be in turn TS, JS or YAML function in Godspeed's DSL.\n\n```yaml\nauthz: \n - fn: com.gs.is_allowed\n\n# OR\nauthz: com.gs.is_allowed\n```\n\n**Disabling authorization at event level**\nIf authorization is enabled at eventsource level for all its events, but you wish to disable authorization for any particular events, you can disable authorization at an event level by saying `authz: false`.\n\n```yaml\n\"http.get./helloworld\":\n fn: helloworld\n authz: false\n```\n\n**Customizing authorization at event level**\nIf authorization is enabled at eventsource level for all its events, but you wish to customise authorization for any particular events, you can disable authorization at the event level, by setting an authorization function or inline yaml workflow there itself.\n```yaml\n\"http.get./helloworld\":\n fn: helloworld\n authz: com.biz.user_is_allowed_for_this_event \n```\n\n### Referencing pre-made project templates\nPre-made projects are a great place to start learning about the meta-framework. You can refer these pre-made examples to learn about different features of meta-framework, and reuse the code from there. Feel free to clone, refer and re-use the repos given below.\n\n#### Basic Project\n\nRepository - [Hello World](https://github.com/godspeedsystems/gs-node-templates/tree/master/hello_world)\n\n#### Full Stack App\nA full stack app with Godspeed based backend and an embedded React project for frontend.\n\nRepository - [With Godspeed and React](https://github.com/godspeedsystems/gs-node-templates/tree/master/FullStack_App_With_React).\n\n#### Loan Origination System\n\nA more complex fintech project with diverse use cases for issuing loans via multiple lenders, policies, loan offers, KYC, loan account creation etc in a microservice based design.\n\nRepository - [Loan Origination System](https://github.com/godspeedsystems/gs-node-templates/tree/master/LOS). \n\n_Check the Readme.md and Setup.md files in this repo as it requires a docker container of Postgres and Kafka to be running. Dockerfile is provided in the project._\n\n### Detailed Walkthrough & Playlist\n\n#### Walkthrough\nA walkthough on a Godspeed project with Loan Origination System example\n\n\n
\n\n
\n\n#### Detailed Explainer Playlist\n:::tip\n[Watch here](https://www.youtube.com/watch?v=GdJ0ghpQ7oA&list=PLRuRJ3PaaJ7ti9bfStNTXqsxxW9wvwA_H)\n:::\n\n### There is more\n\nFor the sake of brevity this section does not include every feature of Godspeed. But feel free to browse the documentatin for more insights and gems!\n" }, "docs/microservices-framework/guides.md": { "content": "---\nsidebar_position: 1\ntitle: Guides and FAQs\ndescription: A collection of guides and frequently asked questions about using the Godspeed Meta-Framework, covering topics such as setting up APIs, writing workflows and functions, and accessing datasources.\nkeywords: [Godspeed, guides, FAQs, API setup, workflows, functions, datasources, REST API, CRUD API, Swagger UI, Postman, authentication, JWT, OAuth2, callbacks, webhooks, TypeScript, inline scripting, databases, datastores, plugins, environment variables, secrets]\n---\n\nThis section provides answers to common questions and step-by-step guides for using the Godspeed Meta-Framework. It covers essential topics including setting up and securing APIs (REST, CRUD, Swagger, Postman, JWT, OAuth2), writing business logic using workflows and functions (TypeScript and Javascript), and accessing various datasources like databases, APIs, and caches.\n\n### Setting up the APIs of your service\n\n- **[How to create Rest APIs in Godspeed framework?](/docs/microservices-framework/how-to/create-api)**\n\n- **[How to generate CRUD APIs?](/docs/microservices-framework/CRUD_API)**\n\n- **[How to open Swagger UI for your APIs?](/docs/microservices-framework/guide/get-started#step-3-access-swagger-ui)**\n\n- **[How to generate postman collection for your APIs?](/docs/microservices-framework/guide/get-started#postman-collection)** \n\n- **[How to add JWT authentication in your Rest APIs?](/docs/microservices-framework/authentication/jwt-authentication)**\n\n- **[How to use OAuth2 for user login and protection of your endpoints?](/docs/microservices-framework/authentication/oauth2-authentication)**\n\n- **[How to disable authentication for a particular endpoint? ](/docs/microservices-framework/authentication/jwt-authentication#disabling-jwt-authentication-at-event-level)**\n\n- **[How to call REST APIs using Axios?](/docs/microservices-framework/how-to/axios-apis)**\n\n- **[How to call APIs using client SDK?](/docs/microservices-framework/datasources/create-custom-datasource)**\n\n- **[How to handle API callbacks and webhooks?](/docs/microservices-framework/how-to/callbacks)**\n\n\n\n### Writing your Workflows or Functions\nIn Godspeed, the words Workflows and Functions mean one and the same thing, i.e. your business logic.\n\n- **[How to write typescript workflows in godspeed?](/docs/microservices-framework/workflows/native-language-functions)**\n\n- **[How to use inline scripting in Godspeed YAML configurations and workflows?](/docs/microservices-framework/inline-scripting/overview)**\n\n\n\n\n\n### Accessing other APIs, databases and datastores from workflows\nIn Godspeed, datasource can mean any place where you send or retrieve data from. It could mean:\n- External APIs\n- Datastores\n - Databases (Types - SQL, NoSQL, graph, key value, columnar, OLAP, OLTP, document)\n - Caches\n - Search engine\n - Vector stores (specially useful for AI and recommendation engines)\n - File system or files\n - Memory\n \n- **[How to Send Emails with Nodemailer in Godspeed?](/docs/microservices-framework/datasources/datasource-plugins/Nodemailer%20Datasource.md)**\n\n- **[How to access databases using Prisma?](/docs/microservices-framework/databases/Overview)**\n\n- **[How to access databases using Mongoose?](/docs/microservices-framework/databases/MongoDB#mongoose-as-datasource-plugin)**\n\n- **[How to access any datastore by creating a custom datasource?](/docs/microservices-framework/datasources/create-custom-datasource)**\n\n- **[How to invoke datasource clients from typescript workflows?](/docs/microservices-framework/how-to/call-datasource)**\n\n- **[How to access the environment variables from the typescript workflows?](/docs/microservices-framework/how-to/short-faqs)**\n\n- **[How to update and run the project after renaming any file?](/docs/microservices-framework/how-to/short-faqs#how-to-update-and-run-the-project-after-renaming-any-file)**\n\n- **[How to handle secrets, api keys, connection_urls etc.?](/docs/microservices-framework/config-and-mappings/config#step-1-define-environment-variables-in-yaml-configuration)**\n\n- **[How to export a variable in environment ?](/docs/microservices-framework/config-and-mappings/config#step-2-set-environment-variable-values)**\n\n" }, "docs/microservices-framework/how-to/axios-apis.md": { "content": "# Level up your data-fetching game with Axios\n\nIn Godspeed, Axios is used as a datasource plugin for making HTTP requests to external APIs or services. \nThis allows your application to communicate with third-party services or other microservices by sending requests\nand receiving responses in a structured way. Axios simplifies the process of making requests, \nhandling authentication, managing headers, handling errors, \nand retrying requests, which is especially useful for complex workflows or external integrations.\n\n## Step-by-Step Guide:\n\n### **Step 1: Configure Axios as a Datasource**\n\n1. Create a godspeed project from the CLI and by default the axios plugin is integrated into your project if not, add the plugin from the CLI and select the **\"axios-as-datasource\" plugin** to integrate the plugin.\n\n ```bash\n godspeed plugin add\n ```\n\n2. **Open the Axios Configuration file**: In your project, go to `src/datasources/`. This file will define the base configuration for Axios.\n\nIn this file, you can set the base URL and any custom headers or authentication needed for interacting with the third-party API.\n\n**Example api.yaml:**\n\n ```yaml\n type: axios\n base_url: https://httpbin.org # Base URL of the third-party API. You can use `https://httpbin.org` for testing.\n\n # Following fields are optional, you can use them as per your requirement\n curlifiedLogs: true # to print all api calls in curl format\n\n authn: # to do Authentication of API calls with token refresh logic\n fn: my_bank.authn\n refreshOn:\n statusCode: [401]\n\n headers: # to set Common headers in all API calls\n Content-Type: application/json\n Cookie: <%mappings.my_bank.auth_workflow_cookie%>\n\n # Retry logic for failed API calls for ex on Internal server errors or request timeouts\n retry:\n when: #the condition\n status: [500, 503] # an array or single value of codes (optional). Default 500\n message: my custom expected message for retry #And (optionally) when response has this message\n max_attempts: 5\n type: constant # or random, exponential\n interval: PT15s\n # type: exponential\n # min_interval: PT5s\n # max_internal: PT15s\n # type: random\n # min_interval: PT5s\n # max_internal: PT15s\n \n ```\n\n### Step 2: Set Up an Event\n\n1. **Create an Event to Trigger the Workflow**: Define an HTTP event to trigger the workflow and call the third-party API.\n Example `fetchData.yaml`:\n\n```yaml\nhttp.get./fetch-data:\n fn: fetchDataWorkflow\n authn: false\n responses:\n 200:\n content:\n application/json:\n schema:\n type: object\n```\n ### Explanation:\n - **http.get./fetch-data:**: The first line of any http event defines the (protocol.Method./end-point)\n Here, /fetch-data is the end point which will send a get request to third party API by calling a workflow.\n - **fn**: fn defines the workflow function to be called. here it is, fetchDataWorkflow.\n\n\n### Step 3: Set Up Workflow to Use the Axios Datasource\n\n Go to `src/functions/` and create a file (e.g., `fetchDataWorkflow.ts`).Use the Axios configuration to make the API call. You can specify the endpoint, HTTP method, and any parameters needed for the request.\n\n```ts\nimport { GSContext, GSDataSource, logger, PlainObject } from \"@godspeedsystems/core\";\n\nexport default async function (ctx: GSContext) {\n const client: GSDataSource = ctx.datasources.api;\n\n const res = await client.execute(ctx, {\n meta: {\n method: 'get',\n url: '/api/items',\n },\n });\n return res;\n};\n```\n\n\n\n ### Explanation:\n\n - **fn: datasource.api.get./anything** - References a predefined Axios function `axiosRequest` to make the API call which contains:-\n\n - **api**: it will be the name of confguration file of the axios datasource. User can give it any name like api1, api2. Same name will be used here.\n - **get**: HTTP method for example, Get/Post/Put\n - **/anything**: The endpoint of the API, relative to the base URL. Here, /anything is the endpoint given by user relative to the base url defined in api.yaml above. So it will send request to this url https://httpbin.org/anything\n - **params**: Optional query parameters.\n\n - **on_error**: Defines error handling if the API call fails.\n\n\n### Step 4: Start the Server and Test the API:\n ```bash\n godspeed serve\n ```\n\n - Go to `http://localhost:3000/api-docs` to access Swagger UI.\n - Test the `/fetch-data` endpoint to ensure the third-party API call is working and data is being returned.\n\n---\n\nFollowing these steps, you’ll be able to interact with third-party APIs in Godspeed using Axios while keeping the configuration modular and manageable." }, "docs/microservices-framework/how-to/call-datasource.md": { "content": "\n# Invoking Datasource Clients from TypeScript Workflows\n\nIn Godspeed, all configured datasources can be accessed through `ctx.datasources`. \n\nThere are two main ways to invoke datasource methods depending on your needs.\n\n---\n\n## Option 1: Direct Access via `client` Key\n\nEach datasource has a `client` key to directly access its client instance. This might be:\n - A **single client instance** (e.g., Axios)\n - **Multiple service instances** (e.g., AWS services like S3, DynamoDB)\n - **Database models** (e.g., MongoDB models via Mongoose)\n\n### Example\nTo access AWS S3 services directly:\n\n```typescript\nconst res = await ctx.datasources.aws.client.s3.listBuckets(args);\n```\n---\n\n## Option 2: Using the `execute` Method\n\nThe `execute` method is available for all datasources and is often more flexible because it leverages the plugin’s full capabilities, including:\n - **Error handling checks** and **response codes**\n - **Retries** and **caching** mechanisms (if supported by the plugin)\n\n### **Usage**\nPass arguments directly to the `execute` method, along with a `meta` object that specifies the `entityType` and `method`:\n```typescript\nconst res = await ctx.datasources.aws.execute(ctx, {\n ...args,\n meta: { entityType: 's3', method: 'listBuckets' }\n});\n```\n---\n\n### Example TypeScript Function Using Datasources\n\nBelow is a TypeScript function that demonstrates invoking both datasource clients and project functions:\n\n```typescript\nimport { GSContext, GSStatus } from \"@godspeedsystems/core\";\n\nexport default async function (ctx: GSContext, args: any) {\n // Example of calling another function (YAML, JS, TS) within a project:\n const helloRes = await ctx.functions['com.gs.helloworld2'](ctx, args);\n\n // **Option 1**: Access AWS S3 directly via the `client` key\n const s3Buckets = await ctx.datasources.aws.client.s3.listBuckets(args);\n\n // **Option 2**: Access AWS S3 via the `execute` method, which uses plugin capabilities\n const s3Res = await ctx.datasources.aws.execute(ctx, {\n ...args,\n meta: { entityType: 's3', method: 'listBuckets' }\n });\n\n // Handling response and status checks\n if (!s3Res.success) {\n return new GSStatus(false, s3Res.code || 500, 'FAiled', {\n message: \"Internal Server Error\",\n info: s3Res.message\n });\n }\n // Returning data in Godspeed:\n // If only data is returned (without `success` or `code` keys), the framework defaults `success` to `true` and response code to `200`.\n return s3Res;\n}\n```\n---\n\n\n\n" }, "docs/microservices-framework/how-to/callbacks.md": { "content": "# Handle Callbacks in Godspeed\nHandling callbacks in your application just requires setting up an **HTTP endpoint**.\n\nThis endpoint will act as a listener for responses from third-party services, which send data or notifications to your server when specific events occur.\n\n### Steps to Handle Callbacks in Godspeed\n\n### 1. Define the Callback URL Endpoint\n\nTo receive a callback, define an **HTTP endpoint** in Godspeed that the third-party service can use as the callback URL.\n\n1. **Create a New Event** in the `src/events` directory. This event will listen for incoming callback requests.\n\nExample **`callback-event.yaml`**:\n\n ```yaml\n http.post./callback:\n summary: \"Handle callbacks from external services\"\n description: \"Endpoint to handle callback responses from external services.\"\n fn: handleCallback\n authn: false\n body:\n content:\n application/json:\n schema:\n type: object\n properties:\n status:\n type: string\n data:\n type: object\n required: [status]\n responses:\n 200:\n description: Callback handled successfully\n content:\n application/json:\n schema:\n type: object\n properties:\n message:\n type: string\n example: Callback received and processed\n 400:\n description: Invalid callback request\n ```\n\n **Explanation**:\n - **Path**: The callback endpoint listens at `POST /callback`.\n - **Body**: Defines the structure of the callback payload (e.g., `status` and `data` fields).\n - **Responses**: Specifies possible responses, such as `200` for successful processing and `400` for invalid requests.\n\n---\n\n### 2. Create the Workflow to Process the Callback\n\nDefine a workflow to process the callback data, handling any necessary business logic.\n\n1. **Create a Workflow File** in the `src/functions` directory to define how the callback data is processed.\n\nExample **`handleCallback.yaml`**:\n\n ```yaml\n id: handleCallback\n summary: Process callback data from external service\n tasks:\n - id: validate_callback\n fn: validateCallbackData\n args:\n data: <% inputs.body.data %>\n status: <% inputs.body.status %>\n\n - id: process_data\n fn: processData\n args:\n data: <% validate_callback.result %>\n\n - id: respond_success\n args:\n statusCode: 200\n body:\n message: Callback received and processed successfully\n ```\n\n **Explanation**:\n - **validate_callback**: A task to validate incoming callback data.\n - **process_data**: Processes the data after validation.\n - **respond_success**: Responds with a `200` status code if all tasks are completed successfully.\n\n---\n\n### 3. Set the Callback URL with the Third-Party Service\n\n1. **Register the Callback URL** with the external service.\n - Provide the callback URL in the following format: `http://:/callback`\n - Replace `` and `` with your actual server's URL and port.\n\n For example:\n ```plaintext\n http://localhost:3000/callback\n ```\n\n2. **Test the Callback**:\n - Use tools like **Postman** or a similar HTTP client to simulate a POST request to the `/callback` endpoint.\n - Ensure the payload matches the expected schema to confirm the callback flow.\n\n---\n\n### Example Scenario: Handling Payment Gateway Callback\n\nLet’s say you’re integrating with a payment gateway, and you want to handle a payment status update callback:\n\n1. **Set Up Event and Workflow**: Follow the same steps as above to create the event (`payment_callback.yaml`) and workflow (`handlePaymentCallback`) files.\n2. **Register the Callback URL**: Provide the payment gateway with your callback URL.\n3. **Process the Callback**: In the `handlePaymentCallback` workflow, define specific tasks to validate payment status, log transactions, or update your database based on the callback data.\n\n---\n\nThis setup gives you full control over handling incoming callbacks, allowing you to process and store data as required. Each component—from the event to the functions—can be customized to suit the specific needs of the callback use case." }, "docs/microservices-framework/how-to/create-api.md": { "content": "# Creating APIs in Godspeed\n\n## Step-by-Step Guide to create APIs in Godspeed framework\n\nCreating a REST API in Godspeed involves defining an **HTTP event** to handle incoming requests and configuring a workflow/function to process the data and return a response. \n \n Here’s a step-by-step guide to setting up a REST API in Godspeed.\n\n### Step 1: Confirm HTTP EventSource Configuration\nOpen src/eventsources/http.yaml to confirm the Express plugin (the HTTP eventsource) is installed and configured correctly.\n[Click here](/docs/microservices-framework/event-sources/event-source-plugins/Express%20Http%20Eventsource#instance-file) and check how the default http.yaml file look like\n\n### Step 2: Define the API or HTTP Event\n In Godspeed, each API endpoint is represented by an HTTP event. \n Here’s how to define an event using to create a simple API that takes a name as input \n\n **Create an Event:**\n \n Create a YAML file in the src/events/ directory (e.g., greet-user.yaml).\n\n Define the HTTP Event to handle POST requests at the /greet endpoint and expect a name in the request body and returns a greeting.\n\n ```yaml\n http.post./greet:\n fn: greet-user\n summary: Greet a user\n description: Accepts a name in req body and returns a greeting\n authn: false\n body:\n content:\n application/json:\n schema:\n type: object\n properties:\n name:\n type: string\n description: Name of the user to greet\n responses: # JSON-Schema of API responses set as per Swagger's standard responses syntax\n 200:\n description: Greeting message\n content:\n application/json:\n schema:\n type: object\n properties:\n message:\n type: string\n example: 'Hello, John!'\n 400:\n description: Bad request if name is missing\n content:\n application/json:\n schema:\n type: object\n properties:\n error:\n type: string\n example: Invalid request. 'name' is required.\n \n ```\n\n### Step 3: Set Up the Workflow\n Now, create a workflow/function to process the request in the src/functions/ directory. Workflow logic can be written either in typescript/javascript or in yaml.\n\n ### 3.1 Typescript function\n Hereby sharing a typescript function which shows all that you get in your event handler workflow when an event is triggered. The generic input structure is constant whether for Express, Fastify, Kafka, Salesforce, Socket etc.\n\n **All that you get in your typescript workflow:**\n\n ```typescript\n import { GSCloudEvent, GSContext, PlainObject } from \"@godspeedsystems/core\";\n import Pino from 'pino';\n\n export default function (ctx: GSContext, args: any) {\n const {\n inputs: {\n data: {\n params, //path parameters from endpoint url\n body, // request body in case of http and graphql apis, event data in case of message bus or socket\n query, // query parameters from rest api\n user, // user payload parsed from jwt token\n headers //request headers in case of http and graphql apis\n }\n }, \n childLogger, // context specific logger. Read pino childLogger for more information\n logger, // Basic logger of the project, generally prefer childLogger for logging \n outputs, // outputs of previously executed tasks of yaml workflows (if any)\n functions, // all loaded workflows/functions from the src/functions/ folder\n datasources, //all configured datasources from src/datasources\n mappings //mappings from src/mappings folder. this is useful for loading key value configurations for business logic of your project\n }: {\n inputs: GSCloudEvent, \n childLogger: Pino.Logger, // you can also add custom attributes to childLogger\n logger: Pino.Logger,\n outputs: PlainObject, \n functions: PlainObject, \n datasources: PlainObject,\n mappings: PlainObject\n } = ctx;\n\n // Will print with workflow_name and task_id attributes. \n childLogger.info('Server is running healthy');\n // Will print without workflow_name and task_id attributes\n logger.info('Arguments passed %o', args);\n logger.info('Inputs object \\n user %o query %o body %o headers %o params %o', user, query, body, headers, params);\n logger.info('Outputs object has outputs from previous tasks with given ids %o', Object.keys(outputs));\n logger.info('Datasources object has following datasource clients %o', Object.keys(datasources));\n logger.info('Total functions found in the project %s', Object.keys(functions).length);\n\n // Returning only data\n return 'Its working! ' + body.name;\n\n //SAME AS\n return {\n data: 'Its working! ' + body.name,\n code: 200,\n // success: true,\n // headers: undefined // or u can set response headers as key: value pairs, \n //for example headers:{custom-header1:\"xyz\" }\n }\n //SAME AS\n return {\n data: 'Its working! ' + body.name,\n code: 200,\n success: true,\n headers: undefined // or u can set response headers as key: value pairs, \n //for example headers:{custom-header1:\"xyz\" }\n }\n }\n ```\n### Example typescript workflow to return greetings with user's name\nThis workflow will take the name from the request and return a personalized greeting. \n\n`greet-user-workflow.ts`\n```typescript\n import { GSCloudEvent, GSContext, PlainObject, GSStatus } from \"@godspeedsystems/core\";\n export default function (ctx: GSContext) {\n const {\n inputs: {\n data: {\n body\n }\n }, \n }= ctx;\n return new GSStatus(true, 200, 'OK', 'Hello ' + body.name, {}); \n }\n\n```\n\n### Step 5: Test the API\n\n- **Start the Godspeed Server:**\n ```bash\n godspeed serve\n ```\n- **Access Swagger UI:** \n\nIn Godspeed, the **Swagger UI** is generated automatically for all defined events for documenting and testing the APIs. \nSwagger UI is typically accessed at the `/api-docs` endpoint, appended to the `base URL` and `port` where the server is running. \nHere’s the general format for accessing Swagger UI:\n\n```plaintext\nhttp://:/` deafult example, http://localhost:3000/api-docs\n```\n\n- If your server is hosted on `example.com` and running on port **8080**:\n ```plaintext\n http://example.com:8080/api-docs\n ```\nHere is a Screenshot of sample Swagger UI :\n\n ![img](../../../static/img/swagger_helloworld.png)\n\n- Test the /greet Endpoint:\n\n Select the POST /greet endpoint in Swagger.\n Provide a JSON body with a name:\n ```json\n {\n \"name\": \"John\"\n }\n ```\n Execute the request and verify the response.\n \n" }, "docs/microservices-framework/how-to/sample-ts-functions.md": { "content": "---\ntitle: Examples of ts functions to be used in prisma datasource interactions\ndescription: Examples of ts functions to be used in prisma datasource interactions\nkeywords: [Godspeed, prisma function, find, get, create, update, ts functions, database interactions, interagte prisma, prisma APIs, call prisma, import prisma client]\n---\n\n```ts title=create.ts\nimport { GSContext, GSStatus, PlainObject } from \"@godspeedsystems/core\";\nimport { PrismaClient } from \"@prisma/client\";\n\nmodule.exports = async (ctx: GSContext, args: PlainObject) => {\n const { inputs: { data: { user, inputs, params, headers } }, logger, datasources } = ctx;\n const userId= user.userId ;\n\n const client: PrismaClient = datasources.schmaName.client;\n try {\n const getUser = await client.User.findUnique({\n where: { id: userId } });\n if(getUser)\n {\n return new GSStatus(true, 200, 'OK', getUser );\n }\n else\n {\n return new GSStatus(true, 200, 'OK', {} );\n }\n }\n catch (error: any) {\n const errorData = error.stack || error;\n logger.info(errorData);\n }\n }\n ```\n\n\n```ts title=update.ts\nimport { GSContext, GSStatus, PlainObject } from \"@godspeedsystems/core\";\nimport { PrismaClient } from \"@prisma/client\";\nmodule.exports = async (ctx: GSContext, args: PlainObject) => {\n const { inputs: { data: { user, body } }, logger, datasources } = ctx;\n try { \n const client: PrismaClient = datasources.schemaName.client;\n const updatedUser = await client.user.update({ \n where: { id: user.userId },\n data: { ...body } }); \n const contactEmail = await client.contacts.update({ \n data: { email: updatedUser.email, \n userId: user.userId } });\n \n return new GSStatus(true, 200, 'OK', updatedUser, {});\n }\n catch (error: any) {\n const errorData = error.stack || error;\n return new GSStatus(false, 400, 'Failed', errorData, {});\n }\n };\n```\n\n```ts title getOne.ts\nimport { GSContext, GSStatus, PlainObject } from \"@godspeedsystems/core\";\nimport { PrismaClient } from \"@prisma/client\";\n\nmodule.exports = async (ctx: GSContext, args: PlainObject) => {\n const { inputs: { data: { user, inputs, params, headers } }, logger, datasources } = ctx;\n const userId= user.userId ;\n\n const client: PrismaClient = datasources.schemaName.client;\n try {\n const getUser = await client.User.findUnique({\n where: { id: userId } });\n if(getUser)\n {\n return new GSStatus(true, 200, 'OK', getUser, );\n }\n else\n {\n return new GSStatus(true, 200, 'OK', {}, );\n }\n }\n catch (error: any) {\n const errorData = error.stack || error;\n logger.info(errorData);\n }\n }\n```" }, "docs/microservices-framework/how-to/short-faqs.md": { "content": "# FAQs\n\n### How to update and run the project after renaming any file?\n \n With `godspeed clean` command you can remove all pre-compiled files from your `/dist` folder. It is useful to clean up the dist folder to remove old references to deleted/renamed files in `src`\n\n\n\n\n\n\n" }, "docs/microservices-framework/inline-scripting/overview.md": { "content": "# Scripting\n\nInline scripting involves embedding and executing code directly within Godspeed YAML code (e.g. workflows, datasources, eventsource, events, etc.,) enhancing seamless integration of script-based logic or functionality. \nIt is used directly by embedding the code/variables in `<% %>` tags. This code is evaluated whenever the yaml is needed to be evaluated e.g. at loadtime or runtime. \n \n**Loadtime evaluation** There are some variables/code which should be evaluated at loadtime. For example, using [configs and mappings](../config-and-mappings/config.md) in datasources, events, or eventsources as given below:\n```yaml title=src/datasources/api.yaml\ntype: axios\nbase_url: <% config.api.base_url %>\n```\n```yaml title=src/events/helloworld.yaml\n\"http.post./helloworld\":\n fn: helloworld\n body:\n content:\n application/json:\n schema:\n type: object\n properties:\n name:\n type: string\n gender:\n type: string\n enum: <% mappings.gender %>\n```\n\n**Runtime evaluation** There are some variables/code which should be evaluated at runtime. For example, using [GSContext properties](../workflows/native-language-functions.md/#gscontext) in workflows as given below:\n```yaml title=src/workflows/helloworld.yaml\nid: helloworld\ntasks:\n - id: first_task\n fn: com.gs.return\n args: <% \"Hello, The gender of \" + inputs.body.name + \" is \" + inputs.body.gender %>\n```\n\n## Default language at global level\nThe default language is `js`. You can change the default language globally in `defaults.lang` key in `config/default.yaml`. It will be applicable everywhere unless overridden explicitly. \n```yaml title=config/default.yaml\ndefaults:\n lang: js #coffee\n```\n\n## Override the default language\nYou can override the default language by specifying the language inside the starting tag like `\nport: \n```\n\n\n## Scripting in datasources\nWithin datasources, [config or mappings](../config-and-mappings/config.md), can be accessed at loadtime.\n```yaml title=src/datasources/api.yaml\ntype: axios\nbase_url: <% config.api.base_url %>\n```\n\n## Scripting in eventsources and events\nWithin datasources, you can use scripting as given in the below examples: \n- accessing [config and mappings](../config-and-mappings/config.md).\n- [authz instruction](../authorization/authz-usecases.md/#a-authorization-at-event-source-level)\n```yaml title=eventsources/http.yaml\ntype: express\nport: <% config.http.port %>\ndocs:\n endpoint: /api-docs\njwt:\n issuer: <% config.jwt.iss %>\n audience: <% config.jwt.aud %>\n secretOrKey: <% config.jwt.sec %>\nauthz:\n - fn: com.gs.transform \n id: authz_task\n args: | # if this condition fails, the else gets executed\n \n```\n\n```yaml title=src/events/helloworld.yaml\n\"http.post./helloworld\":\n fn: helloworld\n body:\n content:\n application/json:\n schema:\n type: object\n properties:\n name:\n type: string\n gender:\n type: string\n enum: <% mappings.gender %>\n authz:\n - fn: com.gs.transform \n id: authz_task\n args: | # if this condition fails, the else gets executed\n \n```\n" }, "docs/microservices-framework/inline-scripting/script-plugins.md": { "content": "---\nsidebar_position: 3\ntitle: Plugins\n---\n\n# Script Plugins\n\nPlugins are utility JS/TS synchronous functions designed to enhance inline scripting capabilities. You can write any piece of code in a plugin and access it inside your inline scripts at any time.\n\nThey can be use like this \n```\ndate: <% time_epoch_convertEpochToDate(inputs.body.datetimestamp) %>\n```\n\n:::tip\nNote: These are not to be confused with eventsource and datasource plugins. \n:::\n\n## Project structure\nPlugins are present in `src/plugins` directory. The default format is js/ts and you can store plugins in the nested directories also.\n```\n.\n├── config\n└── src\n └── plugins\n └── index.ts\n └── time\n └── epoch.ts\n └── epoch\n └── convertEpoch.ts\n```\n## How they are loaded?\n:::tip Note\n1. If the file name is index.ts then its content is available directly at global level i.e. you don't need to write index explicitly while accessing the plugin e.g. `randomInt`. \n2. For other file names you need to mention the file name using underscore notation while accessing the plugins function inside your workflow e.g. `time_epoch_convertEpochToDate`\n3. If it's a default import then you don't need to mention the plugin function name e.g. `epoch_convertEpoch`\n:::\n\n## Sample plugins\nThese are the sample plugins file which export plugin functions named `randomInt` and `convertEpochToDate`.\n```ts title=\"plugins/index.ts\"\nexport function randomInt(min: number, max: number) {\n return Math.floor(Math.random() * (max - min + 1)) + min;\n}\n```\n\n```ts title=\"plugins/time/epoch.ts\"\nimport format from 'date-fns/format';\n\nexport function convertEpochToDate(inputTimestamp: string){\n const newDateTime = new Date(inputTimestamp);\n return format(newDateTime, 'yyyy-MM-dd HH:mm:ss');\n}\n```\n\n```ts title=\"plugins/epoch/convertEpoch.ts\"\nimport format from 'date-fns/format';\n\nexport default function convertEpoch(inputTimestamp: string){\n const newDateTime = new Date(inputTimestamp);\n return format(newDateTime, 'yyyy-MM-dd HH:mm:ss');\n}\n```\n\n## Sample workflow using plugins\nYou can use these plugins in your workflows as given below:\n```\n - id: httpbinCof_step1\n description: Hit http bin with some dummy data. It will send back same as response\n fn: datasource.api.post./anything\n args:\n data:\n personal_email_id: 'ala.eforwich@email.com'\n id: <% 'UID-' + randomInt(1,9) %>\n date: <% time_epoch_convertEpochToDate(inputs.body.datetimestamp) %>\n default_date: <% epoch_convertEpoch(inputs.body.datetimestamp) %>\n\n```" }, "docs/microservices-framework/introduction/benefits.md": { "content": "# Why Godspeed?\n\nGodspeed is more than just a meta-framework—it’s a philosophy for building modular, scalable, and error-resilient systems with minimal boilerplate. It embodies key software design principles, turbocharged with plugins, schema-based automation, and observability—all with a configure-over-code approach.\n\n## Core Benefits\n\n### Single Source of Truth\nGodspeed follows a Single Source of Truth architecture, where the database schema becomes the authoritative source for generating every other aspect of the application’s behavior—API structure, validations, documentation, and workflow code.\n\n### One-to-One Mapping Between DB Schema and API\nAll API definitions and types are automatically generated from the Prisma-based DB schema.\n\nThis ensures a direct and consistent mapping—any update to the DB schema is instantly reflected across APIs, validations, and documentation, with no manual sync effort.\n\n### Database Agnostic, Yet Workflow Stable\nWorkflows use Prisma ORM, which supports multiple databases (Postgres, MySQL, SQLite, MongoDB, etc.) using the same schema and query syntax.\n\nThis means you can switch your database without rewriting your APIs or workflows.\n\n### Auto-Generated Validations\nRequest and response validations are derived from the schema automatically.\n\nNo need to handwrite Joi/Zod/JSON Schema logic—validation is enforced from the single source of truth.\n\n### Universal Schema for All Consumers\nThe same API schema is used to generate:\n- Swagger/OpenAPI specs\n- Postman collections\n- GraphQL schemas\n\nThis guarantees that all API consumers (UI, QA, external integrations) share a consistent and up-to-date contract.\n\n### Schema-Driven Development\n\n* Types and validations are auto-generated from the DB schema using Prisma ORM.\n* Eliminates duplication and reduces chances of mistakes in type definitions or API specifications.\n\n### Built-in Validation and Authentication\n\n* Request and response validations are handled automatically from the schema.\n* No need to define separate validation logic or schemas.\n* AuthN/AuthZ is declarative and decoupled from the core workflows.\n\n### Modular and Decoupled Architecture\n\n* Functions are independent of:\n\n * The HTTP server implementation.\n * The authentication mechanism.\n * The protocol (HTTP, gRPC, GraphQL, etc.)\n* Enables:\n\n * Writing **unit tests without running the server**.\n * Easily swapping or upgrading protocols/servers (e.g., from Express to Fastify or to gRPC).\n * Uniform testing strategies in CI pipelines or pre-commit hooks.\n\n### Configure Over Code\n\n* Replace large codebases with small, declarative configuration files.\n* OAuth2 (Google, GitHub, LinkedIn) works through configuration, not hand-written integrations.\n* Protocol changes or API additions don’t require framework changes—just update your event/config files.\n\n### Fine-Grained API and Workflow Definitions\n\n* APIs are **automatically generated and strongly typed**, minimizing incorrect inputs.\n* Swagger/OpenAPI spec is generated and versioned with the codebase, offering:\n\n * Centralized, up-to-date API documentation.\n * Seamless collaboration across frontend/backend/microservices.\n\n### Observability and Tracing\n\n* Uses **OpenTelemetry format** and **Pino logging**.\n* Emits logs, traces, and metrics with complete context:\n\n * API route, function ID, user info, request ID, span ID, etc.\n* Integrates with platforms like **Grafana, Datadog, Sentry**, and more.\n\n### Performance and Developer Experience\n\n* Pino logger ensures better performance than traditional `console.log`.\n* Project scaffolding, CLI tooling, and plugin integrations reduce time-to-production drastically.\n* Minimal boilerplate = faster onboarding, easier debugging, and fewer errors.\n\n## Guardrails: The Four Pillars of Godspeed\n\n1. **Schema-Driven Everything**\n\n * All components—events, workflows, validations—are driven by schemas.\n * Ensures consistency and predictability across the stack.\n\n2. **Configure Over Code**\n\n * Empower devs to define behavior declaratively via YAML/JSON.\n * Reduces cognitive load and maintenance overhead.\n\n3. **Security First**\n\n * Built-in support for OAuth2, JWT, and custom AuthN/AuthZ.\n * Encourages secure-by-default workflows, with minimal developer overhead.\n\n4. **Decoupled Components**\n\n * Everything—from event sources to workflows—is **loosely coupled** and **interchangeable**.\n * Promotes clean separation of concerns and long-term maintainability.\n\n\n" }, "docs/microservices-framework/introduction/design-principles.md": { "content": "# Design Principles\n\n### Three fundamental abstractions\nEvery microservice in the Godspeed framework has three fundamental abstractions which encompass the functionality of any microservice in general.\n\n- **[Event-sources](/microservices-framework/event-sources/overview):** In the Godspeed framework, Event Sources serve as a mechanism for defining the primary entry or initiation points of an application. Developers can use [pre-built](/docs/microservices-framework/event-sources/event-source-plugins/Overview.md) event source types or create their [own plugins](/docs/microservices-framework/event-sources/create-eventsource-plugin.md). For instance, you can opt for an ['Express'](https://github.com/godspeedsystems/gs-plugins/tree/main/plugins/express-as-http#godspeed-express-plugin) type of Event Source to expose your microservice through REST APIs, ['Kafka'](https://github.com/godspeedsystems/gs-plugins/tree/main/plugins/kafka-as-datasource-as-eventsource#godspeed-plugin-kafka-as-datasource-as-eventsource) message bus event source, a 'Salesforce' Event Source, or you may choose a ['cron'](https://github.com/godspeedsystems/gs-plugins/tree/main/plugins/cron-as-eventsource#godspeed-plugin-cron-as-eventsource) type Event Source to schedule recurring calls to specific workflows. This level of configurability empowers developers to finely tune the behavior of their application according to their requirements.\n\n- **[Events](/docs/microservices-framework/event-sources/event-types/overview.md):** Events trigger workflows. Events are of two types: sync (http: REST, gRpc, Graphql) or async(message bus, socket, cron) Events are generated by event sources like REST endpoints, gRPC, message bus, webhooks, websockets, S3, and more...\n\n\n- **[Functions or Workflows](/docs/microservices-framework/workflows/overview.md):** Workflows are triggered by events. They not only perform business logic but also provide orchestration and federation over datasources. They will use datasources to store or retrieve data, join across various datasources, transform data, emit events and send responses. The framework provides two mechanisms for writing workflows - YAML DSL or native language like JS, TS or Java. For simple workflows, the framework provides a YAML DSL with some inbuilt functions. This saves boilerplate and lines of code where we mainly query other datasources with authz, caching, retries, token refresh, custom telemetry etc. YAML DSL is useful when business logic is simple. If YAML does not suffice for any particular case, developers can author [JS/TS functions or workflows](/docs/microservices-framework/workflows/native-language-functions.md).\n\n\n- **[Datasources](/docs/microservices-framework/datasources/overview.md):** Datasources are locations where data can be stored or read from. Developers can use [pre-built](/docs/microservices-framework/datasources/list-of-plugins.md) datasource types or create their [custom plugin](/docs/microservices-framework/datasources/create-datasource-plugin.md). For example API datasource (another microservice or third party), datastores (RDBMS, document, key-value), file system, S3 storage, message bus, cache etc. A microservice can use multiple datasources. \n\n### Schema driven development\n\nThe event schema should serve as single source of truth and binding contract between teams integrating microservices or frontend and backend. This allows them to work in parallel, in a decoupled manner, and enhances team efficiency. At Godspeed, this serves as a fundamental pillar of our design. The database schema written in Prisma format generates the CRUD API schema and also the CRUD workflows. Further, the schema of all other kind of events whether Sync or Async, follow Swagger Spec to define the input and output shape. From this event spec one can generate a [Graphql](https://github.com/godspeedsystems/gs-plugins/blob/880d8516c17b19dc012535d88c651b60d8454809/plugins/graphql-as-eventsource/README.md) or gRpc schema as well (Planned). Next, for calling an external API, the default API client integration (ex. [Axios](https://github.com/godspeedsystems/gs-plugins/tree/main/plugins/axios-as-datasource#godspeed-plugin-axios-as-datasource) for Nodejs), validates the input and output of the API based on the API's Swagger spec defined in the API integration. To initiate development, the developer needs to first and foremost define the API and data schema.\n\n### Standardized YAML based DSL and configurations\nAt Godspeed we want projects written across languages like JS, Java, Golang to be atleast 70% same via higher level abstractions coded as a YAML DSL. This allows consistency of implementation, helps new developers onboard existing projects easily, prevents technical debt and also eases migration from one language or integration to another.\n\nEvents, workflows, datasources, configurations and environment variables are defined with high level cross-framework, cross-language, cross-integration abstractions, using YAML-based DSL which allows inline scripting in JS, CoffeeScript or Groovy (coming soon with Java port). However, when necessary, workflows can also be scripted in pure JavaScript, Typescript or Java files.\n\nGodspeed's YAML-based DSL simplifies the expression of what is to be done. In many cases, it results in shorter, boilerplate free, comprehensible code compared to traditional programming. Plugin developers define \n- The custom URI of their event types\n- The DSL of the functions exposed by their [plugin](/docs/microservices-framework/datasources/list-of-plugins.md)\n\n\n### Unified datastore model and API\n\nThe unified model configuration and CRUD API, which includes popular SQL, NoSQL stores including [Elasticgraph](/docs/microservices-framework/datasources/datasource-plugins/elasticgraph/elasticgraph.md) (a unique ORM over Elasticsearch), offer standardized interfaces to various types of datastores, whether SQL or NoSQL. Each integration adapts to the nature of the data store. The Prisma and Elasticgraph plugins provided by Godspeed expose the native functions of the client used, giving developer the freedom to use the universal syntax or native queries.\n\n\n### Schema driven data validation\nWe follow Swagger spec as a standard to validate the schema of the event, whether incoming or outgoing events (HTTP), without developer having to write any code. In case of database API calls, the datastore plugin validates the arguments based on the DB model specified in Prisma format.\nThe plugins for HTTP APIs or datastores offer validation for third-party API requests and responses, datastore queries, and incoming events based on Swagger spec or DB schema. For more intricate validation scenarios, such as conditional validation based on attributes like subject, object, environment, or payload, developers can incorporate these rules into the application logic as part of middleware or workflows.\n\n### Authentication\nAuthentication helps to identify who is the user, and generate their access tokens or JWT token for authorized access to the resources of the application.\nThe framework gives developers full freedom to setup any kind of authentication. For ex. they can setup simple auth using the microservice's internal datastore. Or they can invoke an IAM service like ORY Kratos, AWS Okta, or an inhouse service. They can also add OAUTH2 authentication using different providers like Google, Microsoft, Apple, Github etc. using pre-built plugins, or import and customize an existing HTTP plugin like Express, by adding PassportJS middleware.\n\n### Authorization\n[Authorization](/docs/microservices-framework/authorization/overview.md) is key to security, for multi-tenant or variety of other use cases. The framework allows neat, clean and low code syntax to have a fine grained authorization in place, at the event level or workflow's task level, when querying a database or another API.\nDevelopers define authorization rules for each event or workflow task using straightforward configurations for JWT validation or RBAC/ABAC. For more complex use cases, for ex. where they query a policy engine and dynamically compute the permissions, they can write workflows or native functions to access the datasources, compute the rules on the fly, and patch the outcome of that function into a task's `authz` parameter. \nThese rules encompass not only access to API endpoints but also provide fine-grained data access within datastores, for table, row and column level access. The framework allows seamless integration with third-party authorization services or ACL databases via the datasource abstraction.\n\n### Autogenerated Swagger spec\nFollowing the principles of Schema Driven Development, the event spec of the microservice can be used to auto-generate the Swagger spec for HTTP APIs exposed by this serice. The framework provides autogenerated Swagger documentation using CLI.\n\n### Autogenerated CRUD API\nThe framework provides autogenereated CRUD APIs from database model written in Prisma format. Generated API's can be extended by the developers as per their needs. We are planning auto generate or Graphql and gRpc APIs, and may release a developer bountry for the same soon.\n\n### Environment variables and configurations\nThe framework promites setting up of environment variables in a pre-defined YAML file. Though the developer can also allow access by other means via a .env file or setting them up manually. Further [configurations](/docs/microservices-framework/config-and-mappings/config.md) are to be written in `/config` folder. These variables are accessible in \n- Other configuration files\n- Datasource, event source and event definitions\n- Workflows\n\n### Log redaction\nThe framework allows developer to specify the keys that may have sensitive information and should never get published in logs by mistake. There is a centralized check for such keys before a log is about to be printed. \n\n### Telemetry autoinstrumentation using OTEL\nGodspeed allows a developer to add auto-instrumentation which publishes logs, trace and APM information in OTEL standard format, supported by all major observability backends. The APM export captures not just the RAM, CPU information per node/pod/service, but also the latency information of the incoming API calls, with broken down spans giving breakup of latency across the calls to datastores or external APIs. This helps to find out exact bottlenecks. Further the logs and trace/spans are correlated to find out exactly where the error happened in a request spanning multiple microservices with each calling multiple datasources and doing internal computation. \nDeveloper can also add custom logging, span creation and BPM metrics at task level. For ex. new user registration, failed login attempt etc." }, "docs/microservices-framework/introduction/guard-rails.md": { "content": "\n# GuardRails \n\nGuardrails in the context of software development refer to a set of predefined rules, best practices, and constraints that guide developers in creating code that is secure, efficient, and maintainable. \n\nAt Godspeed, our development philosophy revolves around four core guardrails, shaping a robust and innovative approach to software creation:\n\n### 1.Schema driven development:\n\nEmbracing the power of structured data, Godspeed follows a schema-driven development approach. This guardrail ensures that the events, workflows within the software are defined by clear and well-documented schemas. \n\nDive deeper into the fascinating world of schema-driven development by watching the video below...\n\n
\n\n
\n\n\n### 2.Configure over code:\n\nAt Godspeed, we believe in empowerment over complexity. That's why we champion configuration over endless coding. This guardrail encourages the use of high-level abstractions and configuration settings, giving developers the power to tailor the software's behavior without getting lost in the weeds of complex code changes.\n\nReady to unravel the magic of configure over code? Take a closer look in the video below, where we break it down in simple terms!\"\n\n\n
\n\n
\n\n\n\n### 3.Security:\n\nAt Godspeed, security is non-negotiable. We've fortified our software with robust security standards and practices, including secure coding, strong authentication, and proactive vulnerability mitigation. \n\nWatch the video below to see how we keep your digital fortress impregnable.\"\n\n
\n\n
\n\n\n\n### 4. Decoupled Architecture:\n\nGodspeed's guardrail of decoupled architecture advocates for the separation of concerns within the software. By modularizing components and minimizing dependencies, this approach enhances scalability, maintainability, and collaboration among development teams. \n\nWatch the below video for more understanding.\n\n
\n\n
\n\n" }, "docs/microservices-framework/introduction/overview.md": { "content": "# Godspeed - Meta Framework\n\nThis document is meant for technology leaders, architects, and developers. Its purpose is to present the platform's overarching objectives, guiding principles, design elements, components, and functionalities to the target audience.\n\n## Introduction\n\n> Our mission at Godspeed is to bring greater standardization, best practices and ease of engineering to the tech teams of the world. We wish to achieve this as a collaboration by and for the community. We wish to develop a meta-framework along with tools, integrations, learning content, licensing and marketplace, based on a value system which fosters develop-rights, equity, fairness and wellbeing for all.\n\nWe want to democratize tech for teams to effortlessly create, maintain, and monitor complex applications with reduced effort, agility, scalability, quality, performance, cost effectiveness and minimization of technical debt and talent hurdle. Our goal is to liberate teams from the burdensome, repetitive tasks, boilerplate, wiring, so that instead of focusing on the how, they retain a razor sharp focus on the what.\n\nTo achieve this, we're committed to providing teams with the meta-framework, tools, integrations and learning that they need to develop and maintain products, guided by first principles and best practices like\n\n* Standardized implementation across languages, frameworks & tools\n* Decoupled architecture with plug and play integrations\n* Zero boilerplate with pre-built abstractions\n* Schema and standards driven development\n* Shift left\n* Cloud agnostic setup\n* No vendor lock-in\n\n---\n\n## Aims\n\n### Boosting tech team's efficiency\n\nThe meta-framework with developer guardrails, generative features and pre-built integrations provides a ready-made feature set, a YAML DSL for important abstractions, simplified project setup, OTEL based [telemetry](/docs/microservices-framework/telemetry/overview.md) and devops tooling, streamlining the lives of developers and reducing their work and hence the chance of mistakes. This enables them to concentrate on and achieve their primary tasks with minimal effort, time, and cost.\n\nThe framework includes the essential functionalities of a 'modern microservice' by default, allowing developers to concentrate solely on business logic, resulting in significant reduction in workload.\n![productivity](/img/productivity.png)\n\n### Lower learning curve.\n\nDevelopers can deliver a simple microservice on their first day itself. The standardized guardrails enforce their adoption and learning of best practices and engineering concepts, upskilling on the way as the projects proceed. In most cases, only a few in the organization need in-depth expertise, while others can quickly adapt to the framework through training and collaborate effectively, with support from internal team or from Godspeed.\n\n### Easy brownfield adoption\n\nTeams can adopt Godspeed by simply including the NPM module or jar in their existing project and installing the [CLI](/docs/microservices-framework/CLI.md). Godspeed ships as a node module. Hence it can be imported in any nodejs , bunjs or springboot project easily or developers can even reuse the existing event sources and data sources without any hassle.\n\n### Security\n\nThe framework can read the environmental variables from a secure source like K8s Vault. It supports JWT, RBAC and ABAC based fine grained [authorization](/docs/microservices-framework/authorization/overview.md). For data at rest, developers can use encryption mechanisms over datastores. Log redaction allows to hide sensitive information from logs.\n\n### Flexibility\n\nDeveloper should be able to implement anything they need, or replace existing [eventsources](/docs/microservices-framework/event-sources/overview.md) or [datasources](/docs/microservices-framework/datasources/overview.md) with ease. They should also be able to migrate a project from one language or framework to another with least effort.\n\n### Scale, performance and monitoring\n\nFor scale, we encourage the adoption of horizontal scaling approach based on Kubernetes. Developers can deploy a service on a Kubernetes cluster on any cloud.\n\nFor performance, we believe the datasources (APIs and DBs) are the bottlenecks most of the time, and hence the framework allows an easy integration of a cache of choice, over the calls to the [datasources](/docs/microservices-framework/datasources/overview.md). An easier way to setup Graphql like subscriptions and dual writes is planned.\n\nFor monitoring, the framework microservices allows export of APM and BPM signals in OTEL format which is supported by all major observability backend solutions. We provide a pre-configured Grafana dashboard, with correlated logs and traces, and detailed APM out of the box. Using the devops plugin of Godspeed CLI, teams can install the full Grafana stack with Loki, Mimir, Tempo and Minio, on a Kubernetes cluster for scalable telemetry ingestion.\n\n### Maintainability\n\nThe standardized guardrails with clearly defined developer's boundaries, ensure neat, simple & systematic implementations across projects and individual developers with diverse experiences. This avoids technical debt from creeping into the project over time. Further, the decoupled architecture and a neat and modular implementation allows agility in bringing rapid changes as per the ever evolving needs and scenarios.\n\n---\n\n## Design Principles\n\nIn order to serve the [Goals](#aims) and [Tenets](/docs/microservices-framework/introduction/tenets.md) of the framework, we have followed certain [design principles](/docs/microservices-framework/introduction/design-principles.md).\n\n## Framework architecture\n\nThe three main pillars of Godspeed framework: [eventsources](/docs/microservices-framework/event-sources/overview.md), [datasources](/docs/microservices-framework/datasources/overview.md), and [functions or workflows](/docs/microservices-framework/workflows/overview.md).\n\n> Do read more about them in the [design principles](/docs/microservices-framework/introduction/design-principles.md) section.\n\n![framework-architecture](/img/framework-architecture.png)\n\n---\n\n### Building blocks of framework:\n\n1. [**EventSources:**](/docs/microservices-framework/event-sources/overview.md) Pluggable event sources of different kinds like, **HTTP with Express or Fastify, gRpc or Graphql server, cron, web socket, Message bus with Kafka or RabbitMQ, an event from Salesforce**.\n\n 1.1. [**Events:**](/docs/microservices-framework/event-sources/overview.md) Events of async and sync kind are defined in standardized YAML DSL with endpoint, authorization rules and Swagger spec of input and output (as applicable).\n2. [**DataSources:**](/docs/microservices-framework/datasources/overview.md) Pluggable datastores or API clients, to send or retrieve data. For ex. **MongoDB, Redis, AWS API etc**\n3. [**Pure Workflows or Functions:**](/docs/microservices-framework/workflows/overview.md) The events invoke pure functions or workflows which contain the business logic. They take JSON as input and return JSON based output.\n4. [**Config:**](/docs/microservices-framework/config-and-mappings/config.md) The configuration variables as well as their values are defined in yaml files under `config/` directory. Some variables are specific to the framework and rest variables can be created as per the business use cases.\n5. [**ENV:**](/docs/microservices-framework/config-and-mappings/config.md#environment-variables) Sensitive data, like database URLs, that require concealment are specified in .env files and made available in the rest of the project via GSContext object.\n\n## Features\n\n![features](/img/features.png)\n" }, "docs/microservices-framework/introduction/tenets.md": { "content": "# Tenets\n\n\n### Focus on the what and not the how\n\nThanks to Godspeed's out of box integrations, abstractions and command line assistance, developers can focus on implementing features rather than scaffolding, wiring up integrations, writing boilerplate code, doing repetitive tasks, setting up CI/CD, [telemetry](/docs/microservices-framework/telemetry/overview.md) etc. \n\n### Decoupled Architecture\nThe framework emphasizes and facilitates the adoption of [decoupled architecture](https://youtu.be/tVWDbVPsLFQ?si=tSILBF1LSoDmCn4Q), a practice that brings notable advantages.\n\nFor example, decoupled [event sources](/docs/microservices-framework/event-sources/overview.md) and [datasources](/docs/microservices-framework/datasources/overview.md) mean that replacing HTTP server, message bus, databases, cache systems, HTTP client etc. should require minimum to no changes in the project. Even switching between languages, like from Nodejs to Java, should require change in business logic only, leaving the abstracted event definitions, database models, API calls and business logic untouched.\n\n### Standards driven\nAll projects of an organization should follow standardized and systematic implementation for ensuring maintainability and easy adoption by new developers.\nGodspeed is called a meta-framework, or a framework of frameworks, because it unifies the way in which microservices are developed across the organization, even with varying languages and frameworks like Nodejs, BunJS, Java Springboot (coming soon), Golang/Python (coming in 2024) etc., via its standardized abstractions & scaffolding. \n\nFurther the standardization includes established industry standards into system design, such as [OpenTelemetry](/docs/microservices-framework/telemetry/overview.md) for observability, Swagger specifications for API and event schema, and [Prisma](https://www.prisma.io/) for database model definition.\n\n" }, "docs/microservices-framework/swagger-specs.md": { "content": "---\ntitle: Swagger Specs\ndescription: Learn how to configure and generate Swagger specifications and Swagger UI for your Godspeed project's HTTP events, including setting custom information and server URLs.\nkeywords: [Godspeed, Swagger, OpenAPI, API documentation, HTTP events, configuration, http.yaml, Swagger UI, swagger-specs, guide, documentation]\n---\n\nThis document explains how to configure and utilize Swagger (OpenAPI) specifications and the interactive Swagger UI within your Godspeed project. It covers enabling Swagger for HTTP events, customizing the endpoint, adding custom information and server URLs to the specs, and how tags and operationIds are generated.\n\nFramework will give you below folder structure.\n\n```\n .\n ├── src\n ├── docs\n │ └── http-swagger.json \n ├── datasources\n │ ├── types\n │ | └── axios.ts\n | |\n │ └── api.yaml\n │\n ├── events\n | |\n │ └── helloworld.yaml\n |\n ├── eventsources\n │ ├── types\n │ | └── express.ts\n | |\n │ └── http.yaml\n |\n └── functions\n |\n └── helloworld.ts\n```\n\nWhen a service boots up, framework store the swagger docs for http events in `src/docs/`\n\n1. To enable swagger ui add `docs` in **\"./src/eventsources/http.yaml\"**\n\n2. `/api-docs` is the default endpoint,if you want to provide your custom swagger endpoint, you can modify the endpoint from **\"./src/eventsources/http.yaml\"**\n\n### update http.yaml( src/eventsources/http.yaml )\n```yaml\ntype: express\nport: 3000\ndocs:\n endpoint: /api-docs\n```\n\n## Custom info and server URLs info\nYou can add custom server URL for API documentation in http.yaml.\nBy adding the custom server url, your autogenerated documentation or swagger specs will have this url set in the `servers`.\n\n```yaml\ntype: express\nport: 3000\ndocs:\n endpoint: /\n info: # info object as per swagger 3.0 spec\n title: Sample Godspeed App\n version: 1.1.0\n summary: some http calls\n description: lets play with Godspeed\n contact:\n name: API Support\n url: 'http://www.myfintech.com/support'\n email: support@myfintech.com\n license:\n name: Apache 2.0\n url: 'https://www.apache.org/licenses/LICENSE-2.0.html'\n servers:\n - url: 'http://localhost:3001'\n description: Public API server\n - url: 'http://localhost:3001'\n description: Internal API server\n```\n### Generate Swagger UI with tags and operationId\n\nFor tags: Either developer gives `tags` array in schema of event or else the framwork generates the tags for a given event from [`${folderPath}_${filePath}`]\n\nFor operationId: developer can give `operationId`, or `id` in the event shema. If none of this is present, then the summary of the event used to generate the operationId.\n\nFor example,\n![Swagger specs](https://docs.godspeed.systems/assets/images/swagger_spec-5218946d179677ac711303f8d406b4ee.png)" }, "docs/microservices-framework/telemetry/architecture.md": { "content": "---\nsidebar_position: 1\ntitle: Observability Architecture\ntoc_min_heading_level: 2\ntoc_max_heading_level: 4\n---\n\n\nThe below architecture diagram explains the implementation of observability in Godspeed. \n\n![arch](/img/opentelemetry.jpg)\n\n**(a)** Traces are sent to OTEL Collector directly by the application (push based mechanism) using OTLP/gRPC [protocol](https://opentelemetry.io/docs/specs/otlp/). [Tempo](https://grafana.com/docs/tempo/latest/) is used as tracing backend for traces. \n**(b)** Metrics are scraped by the OTEL Collector from the application's `/metrics` endpoint (pull based mechanism). [Prometheus](https://prometheus.io/docs/introduction/overview/) is used as a monitoring tool for metrics with Mimir as its backend. \n**(c)** Logs are collected by a fluent bit service. Then it sends the logs to [Loki](https://grafana.com/docs/loki/latest/). \n**(d)** [Grafana Open Source](https://grafana.com/docs/grafana/latest/fundamentals/#grafana-open-source) is used for the visualization of all the telemetry data.\n\n" }, "docs/microservices-framework/telemetry/configuration.md": { "content": "---\nsidebar_position: 1\ntitle: Generic OTel Configuration\ntoc_min_heading_level: 2\ntoc_max_heading_level: 3\n---\n\nThe generic configuration is applicable for all traces, logs and metrics. It must be applied to use observability in Godspeed. It includes setting up various [OTel environment variables](https://opentelemetry.io/docs/specs/otel/configuration/sdk-environment-variables/#general-sdk-configuration) and the framework environment variables.\n\n### Enable OTel\n[Enable OTel](../CLI.md/#otel) in your service using [Godspeed CLI](../CLI.md) by setting `OTEL_ENABLED` to true. You can alternatively set it in the `.env` file of your service. \n```\nexport OTEL_ENABLED=true\n```\n\n### Specify the service name\nSpecify the service name by which you want to setup observability. Set it as env variable or in the `.env` file of your service. \n```\nexport OTEL_SERVICE_NAME=sample_proj1\n```\n\n\n" }, "docs/microservices-framework/telemetry/custom-metrics-logs-traces.md": { "content": "# Custom metrics, traces and logs (BPM)\nYou can add custom metrics, traces and logs in the workflow DSL at each task level. Once you add these BPM (Business performance metrics) then these will be available out of the box along with APM (Application performance metrics).\n\n### DSL spec for custom metrics\nYou can specify custom metrics in a task by follwing the below DSL spec. Check [Prometheis metrics types](https://prometheus.io/docs/concepts/metric_types/#metric-types) to find the types of available metrics.\n```yaml\nmetrics:\n- name: metric_name\n type: counter|gauge|histogram|summary\n labels: \n label1: val1\n label2: val2\n \n # following functions depending on the metric type and all of them could be scripts, can use inputs/outputs\n inc: 10\n dec: 10\n set: 100\n observe: 2000\n timer: true|false(boolean) starts at the beginning of workflow/task and ends at the end of workflow/task\n```\n\n| metric type | function |\n| ------------- | ------------- |\n| counter | inc |\n| gauge | inc, dec, set |\n| histogram | observe, timer |\n| summary | observe |\n\nRefer [prom-client](https://www.npmjs.com/package/prom-client) to know more about the functions to be used for each metrics type.\n\n#### Example\nIn the following example, we are using two custom metrics: \n**1. ** httpbin_calls_total: counter type metric, counter is incremented by 1. \n**2. ** httpbin_calls_duration: histogram type metric, timer is set to true to record duration. \n\n```yaml\nsummary: Call an API and transform the \ntasks:\n - id: httpbin_step1 # the response of this will be accessible within the parent step key, under the step1 sub key\n name: http bin step\n description: Hit http bin with some dummy data. It will send back same as response\n fn: datasource.api.post./anything \n metrics:\n - name: httpbin_calls_total\n help: 'httpbin_calls_total counter of httpbin requests labeled with: method, status_code'\n type: counter\n labels:\n method: httpbin\n status_code: <% outputs.httpbin_step1.code %> \n inc: 1\n - name: httpbin_calls_duration\n help: 'httpbin_calls_duration duration histogram of httpbin responses labeled with: method, status_code'\n type: histogram\n labels:\n method: httpbin\n status_code: <% outputs.httpbin_step1.code %> \n timer: true \n args:\n params: <% inputs.query %>\n data: <% inputs.body %>\n```\n\n### DSL spec for custom trace\nYou can add custom trace in the workflow tasks. Then these custom traces will be traced along with the default [traces](./tracing.md). Check [OTel traces](https://opentelemetry.io/docs/concepts/signals/traces/) to understand more about trace, span, attributes, etc.,\n```yaml\ntrace:\n name: span_name\n attributes:\n attribute1: value1\n attribute2: value2\n```\n\n#### Example\nIn the following example, we are creating a new span named `httpbin_trace` with span attributes `request` and `param`. This span gets created when the task starts and ended when the task completes its execution.\n\n```yaml\nsummary: Call an API and transform the \ntasks:\n - id: httpbin_step1 # the response of this will be accessible within the parent step key, under the step1 sub key\n name: http bin step\n description: Hit http bin with some dummy data. It will send back same as response\n fn: datasource.api.post./anything\n trace:\n name: httpbin_trace\n attributes:\n request: <%inputs.body%>\n param: <%inputs.query%>\n args:\n params: <% inputs.query %>\n data: <% inputs.body %>\n```\n\n### DSL spec for custom logs\nYou can add custom log before and after a task in a workflow.\n```yaml\nlogs:\n before:\n level: fatal|error|warn|info|debug|trace # refer pino for levels\n message: 'Sample log before'\n params: \n param1: val1\n param2: val2\n attributes:\n request:\n query: <%inputs.query%>\n after:\n level: info\n message: 'Sample log after'\n params:\n attributes: \n```\n\nPlease refer to [OTEL Logging Data model](https://opentelemetry.io/docs/reference/specification/logs/data-model/) for understanding of fields dumped in the logs.\n> `message` and `params` are part of `Body` field and `attributes` are part of `Attributes` field in the log.\n\n#### Example\nIn the following example, we are adding two additional logs before and after the task execution. \n\n```yaml\nsummary: Call an API and transform the \ntasks:\n - id: httpbin_step1 # the response of this will be accessible within the parent step key, under the step1 sub key\n name: http bin step\n description: Hit http bin with some dummy data. It will send back same as response\n fn: datasource.api.post./anything\n logs:\n before:\n level: error\n message: 'Hello'\n params: \n - key1: v1\n key2: v2\n - v1\n attributes: \n request: <%inputs.query%>\n after:\n level: error\n message: 'World'\n params: \n key1: v1\n key2: v2\n attributes: \n customer_name: <% outputs.httpbin_step1.data.json.customer_name %> \n args:\n params: <% inputs.query %>\n data: <% inputs.body %>\n```\n\n** Sample Logs ** \n```json\n{\"Body\":\"Hello [{\\\"key1\\\":\\\"v1\\\",\\\"key2\\\":\\\"v2\\\"},\\\"v1\\\"]\",\"Timestamp\":\"2024-04-10T09:40:45.122Z000000\",\"SeverityNumber\":9,\"SeverityText\":\"INFO\",\"TraceId\":\"afde0bf5bb3533d932c1c04c30d91172\",\"SpanId\":\"ad477b2cf81ca711\",\"TraceFlags\":\"01\",\"Resource\":{\"service.name\":\"unknown_service:node\",\"host.hostname\":\"9ce06d358ba7\",\"process.pid\":67228},\"Attributes\":{\"request\":{\"status\":\"Hello\"},\"task_id\":\"if\",\"workflow_name\":\"if_else\"}}\n. . . . . . . . . . .\n{\"Body\":\"World {\\\"key1\\\":\\\"v1\\\",\\\"key2\\\":\\\"v2\\\"}\",\"Timestamp\":\"2024-04-10T09:40:45.127Z000000\",\"SeverityNumber\":17,\"SeverityText\":\"ERROR\",\"TraceId\":\"afde0bf5bb3533d932c1c04c30d91172\",\"SpanId\":\"ad477b2cf81ca711\",\"TraceFlags\":\"01\",\"Resource\":{\"service.name\":\"unknown_service:node\",\"host.hostname\":\"9ce06d358ba7\",\"process.pid\":67228},\"Attributes\":{\"customer_name\":\"Hell!\",\"task_id\":\"if\",\"workflow_name\":\"if_else\"}}\n``` \n" }, "docs/microservices-framework/telemetry/logging.md": { "content": "---\nsidebar_position: 1\ntitle: Logs\ntoc_min_heading_level: 2\ntoc_max_heading_level: 4\n---\n\nLogs are the application/service logs that are displayed on console. Sample logs: \n```\n[14:46:00.881] DEBUG: Datasources found in src/datasources [\"api\",\"mem-cache\"]\n section: \"loading_functions\"\n[14:46:00.882] DEBUG: Starting to parse and load GSFunction id: first_task name: helloworld\n parent: {\n \"workflow_name\": \"helloworld\",\n \"task_id\": \"helloworld\"\n }\n workflow_name: \"helloworld\"\n task_id: \"first_task\" \n```\nGodspeed adds some attributes (extra key/value pairs) to the logs to enhance logging and provide more information about the location. For example, in the above logs, \n**`section`**: which represents section the service is in while getting loading. \n**`parent`**: an object with information about the parent workflow and task. \n**`workflow_name`**: which represents the current workflow. \n**`task_id`**: which represents the current task. \n\n### Log formats\n\nThere are two types of log formats in Godspeed:\n1. **[For Development environment - pino pretty format](#pino-pretty-format)**\n\n2. **[For Production environment - OTel format](#otel-format)**\n\n### pino pretty format\n\n[pino pretty format](https://www.npmjs.com/package/pino-pretty) is more readable and user friendly. It is mostly used in development environments on user's local machine. Logs are dumped in this format when any of the below conditions is satisfied: \n\n**(a)** observability is [disabled](/docs/microservices-framework/CLI#otel-disable) i.e. OTEL_ENABLED env variable is not set to true. \n**(b)** NODE_ENV is set to 'dev'.\n \nSample Logs:\n```\n[14:46:00.813] INFO: [START] Load definitions from /home/gurjot/data/cli-test/v2_test2/dist/definitions\n[14:46:00.816] DEBUG: Definitions loaded and registered to ajvInstance\n[14:46:00.817] INFO: [END] Load definitions\n[14:46:00.818] INFO: [START] Load mappings from /home/gurjot/data/cli-test/v2_test2/dist/mappings\n[14:46:00.819] INFO: [END] Load mappings\n[14:46:00.819] INFO: [START] Load data sources from /home/gurjot/data/cli-test/v2_test2/dist/datasources\n. . . . . . . . .\n[14:46:00.881] DEBUG: JS/TS functions found in src/functions my_bank_api.auth_workflow,validations.request.standardResponse\n section: \"loading_functions\"\n. . . . . . . .\n[14:46:00.882] DEBUG: Starting to parse and load GSFunction id: helloworld name: helloworld\n workflow_name: \"helloworld\"\n task_id: \"helloworld\"\n```\n\n### OTel format\n[OTel Logging format](https://opentelemetry.io/docs/reference/specification/logs/data-model/) is used in Production/UAT and other environments where the user needs to enable observability. \n:::tip \n OTel Logging format provides co-relation between logs and traces by specifying tracing information (like TraceId, SpanId or TraceFlags) in the logs. With this co-relation information, one can navigate from logs to trace to get more information about any scenario. \n:::\n\nLogs are dumped in this format when all of the below conditions are satisfied: \n**(a)** observability is [enabled](/docs/microservices-framework/CLI.md/#otel) i.e. OTEL_ENABLED env variable is set to true. \n**(b)** NODE_ENV is not set to 'dev'.\n\n```json\n{\"Body\":\"[START] Load definitions from /home/gurjot/data/cli-test/v2_test2/dist/definitions\",\"Timestamp\":\"2024-04-10T09:40:45.122Z000000\",\"SeverityNumber\":9,\"SeverityText\":\"INFO\",\"Resource\":{\"service.name\":\"sample_app\",\"env\":\"production\",\"host.hostname\":\"ThinkPadT480s\"},\"Attributes\":{}}\n{\"Body\":\"Definitions loaded and registered to ajvInstance\",\"Timestamp\":\"2024-04-10T09:40:45.124Z000000\",\"SeverityNumber\":5,\"SeverityText\":\"DEBUG\",\"Resource\":{\"service.name\":\"sample_app\",\"env\":\"production\",\"host.hostname\":\"ThinkPadT480s\"},\"Attributes\":{}}\n{\"Body\":\"[END] Load definitions\",\"Timestamp\":\"2024-04-10T09:40:45.125Z000000\",\"SeverityNumber\":9,\"SeverityText\":\"INFO\",\"Resource\":{\"service.name\":\"sample_app\",\"env\":\"production\",\"host.hostname\":\"ThinkPadT480s\"},\"Attributes\":{}}\n{\"Body\":\"[START] Load mappings from /home/gurjot/data/cli-test/v2_test2/dist/mappings\",\"Timestamp\":\"2024-04-10T09:40:45.125Z000000\",\"SeverityNumber\":9,\"SeverityText\":\"INFO\",\"Resource\":{\"service.name\":\"sample_app\",\"env\":\"production\",\"host.hostname\":\"ThinkPadT480s\"},\"Attributes\":{}}\n{\"Body\":\"[END] Load mappings\",\"Timestamp\":\"2024-04-10T09:40:45.126Z000000\",\"SeverityNumber\":9,\"SeverityText\":\"INFO\",\"Resource\":{\"service.name\":\"sample_app\",\"env\":\"production\",\"host.hostname\":\"ThinkPadT480s\"},\"Attributes\":{}}\n{\"Body\":\"[START] Load data sources from /home/gurjot/data/cli-test/v2_test2/dist/datasources\",\"Timestamp\":\"2024-04-10T09:40:45.126Z000000\",\"SeverityNumber\":9,\"SeverityText\":\"INFO\",\"Resource\":{\"service.name\":\"sample_app\",\"env\":\"production\",\"host.hostname\":\"ThinkPadT480s\"},\"Attributes\":{}}\n. . . . . . . . .\n{\"Body\":\"JS/TS functions found in src/functions my_bank_api.auth_workflow,validations.request.standardResponse\",\"Timestamp\":\"2024-04-10T09:40:45.210Z000000\",\"SeverityNumber\":5,\"SeverityText\":\"DEBUG\",\"Resource\":{\"service.name\":\"sample_app\",\"env\":\"production\",\"host.hostname\":\"ThinkPadT480s\"},\"Attributes\":{\"section\":\"loading_functions\"}}\n. . . . . . . . .\n{\"Body\":\"Starting to parse and load GSFunction id: helloworld name: helloworld\",\"Timestamp\":\"2024-04-10T09:40:45.210Z000000\",\"SeverityNumber\":5,\"SeverityText\":\"DEBUG\",\"Resource\":{\"service.name\":\"sample_app\",\"env\":\"production\",\"host.hostname\":\"ThinkPadT480s\"},\"Attributes\":{\"workflow_name\":\"helloworld\",\"task_id\":\"helloworld\"}}\n``` \n\n### Log level\nThe minimum level set to log above this level. Please refer [Pino log levels](https://github.com/pinojs/pino/blob/master/docs/api.md#options) for more information. Set `log.level` in [config](/docs/microservices-framework/config-and-mappings/config.md#static-variables).\n\n```yaml title=config/default.yaml\nlog:\n redact: # ['a.b.c', 'a.b.*', 'req.headers', 'mobileNumber'] #pino redact rules. Default null.\n level: debug #by default info\n sync: true #By default sync is false. For debugging, keep it true. For performance keep it false.\n timestamp: stdTimeFunctions.isoTime #Pino date formats\n bindings: # should pid and hostname be enabled in pino log bindings.\n pid: false\n hostname: true \n```\n\nSample Logs:\n```\n[14:46:00.813] INFO: [START] Load definitions from /home/gurjot/data/cli-test/v2_test2/dist/definitions\n[14:46:00.816] DEBUG: Definitions loaded and registered to ajvInstance\n[14:46:00.817] INFO: [END] Load definitions\n```\n\n### Log fields masking\nIf you want to hide sensitive information in logs then define the fields which need to be hidden in `redact` feature in [Static variables](/docs/microservices-framework/config-and-mappings/config.md#static-variables). Redaction path syntax is standard JSON object lookup. \nFor example, \n```yaml title=\"config/default.yaml\"\nlog:\n redact: ['a.b.c', 'a.b.*', 'req.headers', 'mobileNumber'] #pino redact rules. Default null.\n level: debug #by default info\n sync: true #By default sync is false. For debugging, keep it true. For performance keep it false.\n timestamp: stdTimeFunctions.isoTime #Pino date formats\n bindings: # should pid and hostname be enabled in pino log bindings.\n pid: false\n hostname: true \n```\nBy specifying the above redaction paths, the objects which have these properties will be masked in the logs.\n\n:::note\nPlease refer [Pino redaction paths](https://github.com/pinojs/pino/blob/master/docs/redaction.md#paths) for more information.\n:::\n\n**Generic convention** \nIf you want to mask any field in the objects in all deep nesting levels then you can use `**.` convention instead of specifying each path explicitly.\n\nFor example, \n```yaml title=\"config/default.yaml\"\nlog:\n redact: ['**.mobileNumber'] \n```\nBy specifying the above redaction path, `mobileNumber` field will be redacted in logs in all nesting levels. \n \nSample masked logs:\n```json\n{\"Body\":\"args after evaluation: step1 {\\\"name\\\":\\\"ABC\\\",\\\"gender\\\":\\\"M\\\",\\\"age\\\":25,\\\"mobileNumber\\\":\\\"*****\\\"}\",\"Timestamp\":\"2024-04-10T09:40:45.124Z000000\",\"SeverityNumber\":9,\"SeverityText\":\"INFO\",\"Resource\":{\"service.name\":\"sample_app\",\"host.hostname\":\"4030f41a75cb\",\"process.pid\":3593},\"Attributes\":{\"event\":\"/helloworld.http.get\",\"workflow_name\":\"helloworld\",\"task_id\":\"step1\"}}\n{\"Body\":\"Executing handler step1 {\\\"name\\\":\\\"ABC\\\",\\\"gender\\\":\\\"M\\\",\\\"age\\\":25,\\\"mobileNumber\\\":\\\"*****\\\"}\",\"Timestamp\":\"2024-04-10T09:40:45.124Z000000\",\"SeverityNumber\":9,\"SeverityText\":\"INFO\",\"Resource\":{\"service.name\":\"sample_app\",\"host.hostname\":\"4030f41a75cb\",\"process.pid\":3593},\"Attributes\":{\"event\":\"/helloworld.http.get\",\"workflow_name\":\"helloworld\",\"task_id\":\"step1\"}}\n{\"Body\":\"Result of _executeFn step1 {\\\"name\\\":\\\"ABC\\\",\\\"gender\\\":\\\"M\\\",\\\"age\\\":25,\\\"mobileNumber\\\":\\\"*****\\\"}\",\"Timestamp\":\"2024-04-10T09:40:45.130Z000000\",\"SeverityNumber\":9,\"SeverityText\":\"INFO\",\"Resource\":{\"service.name\":\"sample_app\",\"host.hostname\":\"4030f41a75cb\",\"process.pid\":3593},\"Attributes\":{\"event\":\"/helloworld.http.get\",\"workflow_name\":\"helloworld\",\"task_id\":\"step1\"}}\n{\"Body\":\"Result of _executeFn add_mobileNumber_transformation_step2 {\\\"request_data\\\":{\\\"payload\\\":{\\\"data\\\":{\\\"body\\\":{\\\"mobileNumber\\\":\\\"*****\\\"}}}}}\",\"Timestamp\":\"2024-04-10T09:40:45.190Z000000\",\"SeverityNumber\":9,\"SeverityText\":\"INFO\",\"Resource\":{\"service.name\":\"sample_app\",\"host.hostname\":\"4030f41a75cb\",\"process.pid\":3593},\"Attributes\":{\"event\":\"/helloworld.http.get\",\"workflow_name\":\"helloworld\",\"task_id\":\"add_mobileNumber_transformation_step2\"}}\n{\"Body\":\"this.id: hello_world, output: {\\\"request_data\\\":{\\\"payload\\\":{\\\"data\\\":{\\\"body\\\":{\\\"mobileNumber\\\":\\\"*****\\\"}}}}}\",\"Timestamp\":\"2024-04-10T09:40:45.191Z000000\",\"SeverityNumber\":5,\"SeverityText\":\"DEBUG\",\"Resource\":{\"service.name\":\"sample_app\",\"host.hostname\":\"4030f41a75cb\",\"process.pid\":3593},\"Attributes\":{\"event\":\"/helloworld.http.get\",\"workflow_name\":\"helloworld\",\"task_id\":\"hello_world\"}}\n```\n\n### Log timestamp\nDifferent timestamp formats. Please refer [pino.stdTimeFunctions](https://github.com/pinojs/pino/blob/master/docs/api.md#pinostdtimefunctions-object) for more information. Set `log.timestamp` in [config](/docs/microservices-framework/config-and-mappings/config.md#static-variables).\n```yaml title=config/default.yaml\nlog:\n redact: ['a.b.c', 'a.b.*', 'req.headers', 'mobileNumber'] #pino redact rules. Default null.\n level: debug #by default info\n sync: true #By default sync is false. For debugging, keep it true. For performance keep it false.\n timestamp: stdTimeFunctions.isoTime #Pino date formats\n bindings: # should pid and hostname be enabled in pino log bindings.\n pid: false\n hostname: false \n```\nSample Logs:\n```\n[14:46:00.813] INFO: [START] Load definitions from /home/gurjot/data/cli-test/v2_test2/dist/definitions\n[14:46:00.816] DEBUG: Definitions loaded and registered to ajvInstance\n[14:46:00.817] INFO: [END] Load definitions\n```\n\n### Log bindings\nBinds two extra properties to each log entry by default: the program's process ID (pid), and the current machine's hostname. Set `log.bindings` in [config](/docs/microservices-framework/config-and-mappings/config.md#static-variables).\n```yaml title=config/default.yaml\nlog:\n redact: ['a.b.c', 'a.b.*', 'req.headers', 'mobileNumber'] #pino redact rules. Default null.\n level: debug #by default info\n sync: true #By default sync is false. For debugging, keep it true. For performance keep it false.\n timestamp: stdTimeFunctions.isoTime #Pino date formats\n bindings: # should pid and hostname be enabled in pino log bindings.\n pid: true\n hostname: true \n```\nSample Logs:\n```\n[14:46:00.813] INFO (16542 on HP-EliteBook-840-G3): [START] Load definitions from /home/gurjot/data/cli-test/v2_test2/dist/definitions\n[14:46:00.816] DEBUG (16542 on HP-EliteBook-840-G3): Definitions loaded and registered to ajvInstance\n[14:46:00.817] INFO (16542 on HP-EliteBook-840-G3): [END] Load definitions\n```\n\n### Asynchronous/Synchronous logs\nSpecifies whether to do synchronous or asynchronous logging. Please refer [asynchronous logging](https://github.com/pinojs/pino/blob/master/docs/asynchronous.md) for more information. Set `log.sync` in [config](/docs/microservices-framework/config-and-mappings/config.md#static-variables).\n:::tip\nIt is recommented to set it to true for debugging purposes. For performance keep it false.\n:::\n\n```yaml title=config/default.yaml\nlog:\n redact: ['a.b.c', 'a.b.*', 'req.headers', 'mobileNumber'] #pino redact rules. Default null.\n level: debug #by default info\n sync: false #By default sync is false. For debugging, keep it true. For performance keep it false.\n timestamp: stdTimeFunctions.isoTime #Pino date formats\n bindings: # should pid and hostname be enabled in pino log bindings.\n pid: true\n hostname: true \n```\nSample Logs:\n```\n[14:46:00.813] INFO (16542 on HP-EliteBook-840-G3): [START] Load definitions from /home/gurjot/data/cli-test/v2_test2/dist/definitions\n[14:46:00.816] DEBUG (16542 on HP-EliteBook-840-G3): Definitions loaded and registered to ajvInstance\n[14:46:00.817] INFO (16542 on HP-EliteBook-840-G3): [END] Load definitions\n```\n\n### Custom log attributes\n\n### For all events\nYou can add any custom attribute in the logs whenever any event is triggered on your service. The value for the custom identifier can be picked up from event body, params, query, or headers. \n\n**To enable this feature for common logging attributes across all events,you need to specify two things:**\n\n**i** `log.attributes` variable as in [config](/docs/microservices-framework/config-and-mappings/config.md#static-variables) which contains custom identifiers.\n\nFor example, this is the sample static configuration:\n```yaml\nlog:\n attributes: \n mobileNumber: <% query.mobileNumber %>\n id: <% params.id %>\n lan: <% body.data.lan %>\n name: <% headers.name %>\n gender: <% mappings.Gender %>\n```\n\n**ii** location of the identifier in the request payload. As specified in the above example, \n**-** if mobileNumber is present in query params then specify `query.mobileNumber`. \n**-** if id is present in path params then specify `params.id`. \n**-** if lan is present in data field inside body then specify `body.data.lan`. \n**-** if name is present in headers then specify `headers.name`. \n**-** if gender is present in data field inside mappings then specify `<% mappings.Gender %>`. \n\n#### Sample Logs \n```json\n{\"Body\":\"Processing event /test/:id.http.post\",\"Timestamp\":\"2024-04-10T09:40:45.191Z000000\",\"SeverityNumber\":9,\"SeverityText\":\"INFO\",\"TraceId\":\"3b66e6f8ec6624f6467af1226503a39e\",\"SpanId\":\"eb6e7d89ac381e9f\",\"TraceFlags\":\"01\",\"Resource\":{\"service.name\":\"sample_app\",\"host.hostname\":\"5252603e08be\",\"process.pid\":828},\"Attributes\":{\"event\":\"/test/:id.http.post\",\"workflow_name\":\"com.jfs.test\",\"mobileNumber\":\"9878987898\",\"id\":\"12\",\"lan\":\"12345\"}}\n{\"Body\":\"event inputs {\\\"baseUrl\\\":\\\"\\\",\\\"body\\\":{\\\"data\\\":{\\\"lan\\\":\\\"12345\\\"}},\\\"fresh\\\":false,\\\"hostname\\\":\\\"localhost\\\",\\\"ip\\\":\\\"::ffff:172.22.0.1\\\",\\\"ips\\\":[],\\\"method\\\":\\\"POST\\\",\\\"originalUrl\\\":\\\"/test/12?mobileNumber=9878987898\\\",\\\"params\\\":{\\\"id\\\":\\\"12\\\"},\\\"path\\\":\\\"/test/12\\\",\\\"protocol\\\":\\\"http\\\",\\\"query\\\":{\\\"mobileNumber\\\":\\\"9878987898\\\"},\\\"route\\\":{\\\"path\\\":\\\"/test/:id\\\",\\\"stack\\\":[{\\\"name\\\":\\\"\\\",\\\"keys\\\":[],\\\"regexp\\\":{\\\"fast_star\\\":false,\\\"fast_slash\\\":false},\\\"method\\\":\\\"post\\\"},{\\\"name\\\":\\\"\\\",\\\"keys\\\":[],\\\"regexp\\\":{\\\"fast_star\\\":false,\\\"fast_slash\\\":false},\\\"method\\\":\\\"post\\\"}],\\\"methods\\\":{\\\"post\\\":true}},\\\"secure\\\":false,\\\"stale\\\":true,\\\"subdomains\\\":[],\\\"xhr\\\":false,\\\"headers\\\":{\\\"content-type\\\":\\\"application/json\\\",\\\"user-agent\\\":\\\"PostmanRuntime/7.29.2\\\",\\\"accept\\\":\\\"*/*\\\",\\\"postman-token\\\":\\\"9e57df7d-0a75-48b6-bc52-921bd5c045b7\\\",\\\"host\\\":\\\"localhost:4000\\\",\\\"accept-encoding\\\":\\\"gzip, deflate, br\\\",\\\"connection\\\":\\\"keep-alive\\\",\\\"content-length\\\":\\\"46\\\"},\\\"files\\\":[]}\",\"Timestamp\":\"2024-04-10T09:40:45.196Z000000\",\"SeverityNumber\":9,\"SeverityText\":\"INFO\",\"TraceId\":\"3b66e6f8ec6624f6467af1226503a39e\",\"SpanId\":\"eb6e7d89ac381e9f\",\"TraceFlags\":\"01\",\"Resource\":{\"service.name\":\"sample_app\",\"host.hostname\":\"5252603e08be\",\"process.pid\":828},\"Attributes\":{\"event\":\"/test/:id.http.post\",\"workflow_name\":\"com.jfs.test\",\"mobileNumber\":\"9878987898\",\"id\":\"12\",\"lan\":\"12345\"}}\n{\"Body\":\"event body and eventSpec exist\",\"Timestamp\":\"2024-04-10T09:40:45.197Z000000\",\"SeverityNumber\":9,\"SeverityText\":\"INFO\",\"TraceId\":\"3b66e6f8ec6624f6467af1226503a39e\",\"SpanId\":\"eb6e7d89ac381e9f\",\"TraceFlags\":\"01\",\"Resource\":{\"service.name\":\"sample_app\",\"host.hostname\":\"5252603e08be\",\"process.pid\":828},\"Attributes\":{\"event\":\"/test/:id.http.post\",\"workflow_name\":\"com.jfs.test\",\"mobileNumber\":\"9878987898\",\"id\":\"12\",\"lan\":\"12345\"}}\n```\n\n### 2. At eventsource level\n\nYou can override log attributes at eventsource level also. You can specify customized log attributes for specific eventsource. This will override default custom attributes as defined in the [previous section](../telemetry/logging.md/#1-for-all-events).\n\nTo enable this feature, you need to specify `log.attributes` variable in the eventsource config.\n```yaml\ntype: express\nport: 3000\nbase_url: /api/v1 #the base url of the http service\n\n#Basic swagger setup\ndocs:\n endpoint: /api-docs # the url on which the service will start\n\nlog:\n attributes:\n event_type: myevent \n```\n\n##### Sample Logs\n```json\n{ Body: \"return value [] 200 %o\"\n Timestamp: \"2024-04-10T09:40:45.197Z000000\"\n SeverityNumber: 9\n SeverityText: \"INFO\"\n TraceId: \"3fba9b9bd5d10d00b1b730b74c8eba51\"\n SpanId: \"985e8a8d6a18568b\"\n TraceFlags: \"01\"\n Resource: {\n \"service.name\": \"sample_app\",\n \"host.hostname\": \"6295f63d9181\",\n \"process.pid\": 13956\n }\n Attributes: {\n \"event\": \"/helloworld\",\n \"workflow_name\": \"helloworld\",\n \"file_name\": \"helloworld\",\n \"event_type\": \"myevent\"\n }}\n```\n\n### 3. At event level\n\nYou can override log attributes at event level also. You can specify customized log attributes for specific event. This will override default custom attributes as defined in the [previous section](../telemetry/logging.md/#2-at-eventsource-level).\n\nTo enable this feature ,you need to specify `log.attributes` on event level which contains custom identifiers.\n```yaml\n\"http.get./helloworld\":\n fn: helloworld\n authn: false\n log:\n attributes:\n msgparameter:\n fruit: apple\n identifier: 1\n params:\n - name: name\n in: query\n required: true\n schema:\n type: string\n responses:\n 200:\n content:\n application/json:\n schema:\n type: string\n```\n\n##### Sample Logs\n```json\n{ Body: \"return value [] 200 %o\"\n Timestamp: \"1688565778237000000\"\n SeverityNumber: 9\n SeverityText: \"INFO\"\n TraceId: \"3fba9b9bd5d10d00b1b730b74c8eba51\"\n SpanId: \"985e8a8d6a18568b\"\n TraceFlags: \"01\"\n Resource: {\n \"service.name\": \"sample_app\",\n \"host.hostname\": \"6295f63d9181\",\n \"process.pid\": 13956\n }\n Attributes: {\n \"event\": \"/helloworld\",\n \"workflow_name\": \"helloworld\",\n \"file_name\": \"helloworld\",\n \"msgparameter\": {\n \"fruit\": \"apple\"\n },\n \"identifier\": 1,\n \"task_id\": \"\"\n }}\n```\n### 4. Custom on_error logging in workflow/tasks\n\nIn case you want to log specific attributes when an error happens in a task, set those values in `on_error.log_attributes` of that task.\n\nFor ex.\n\n```yaml\nsummary: add custom error logs on workflow\nid: validation_error\ntasks:\n - id: error_transform\n fn: com.biz.error_log\non_error:\n log_attributes:\n error_type: \"enter your custom error type here\"\n error_message: \"xyz value is required\"\n```\n\n##### Sample logs\n\n```json\n {Timestamp: \"1688563866502000000\"\n SeverityNumber: 17\n SeverityText: \"ERROR\"\n TraceId: \"7563f0bd1e6c6508e58a4d1de1464635\"\n SpanId: \"c4c65132ef79982f\"\n TraceFlags: \"01\"\n Resource: {\n \"service.name\": \"sample_app\",\n \"host.hostname\": \"6295f63d9181\",\n \"process.pid\": 8455\n }\n Attributes: {\n \"event\": \"/helloworld\",\n \"workflow_name\": \"helloworld\",\n \"file_name\": \"helloworld\",\n \"msgparameter\": {\n \"fruit\": \"apple\"\n },\n \"task_id\": \"response_validation_error_handler\",\n \"error\": {\n \"error_type\": \"enter your custom error type here\",\n \"error_message\": \"xyz value is required\"\n }\n }}\n```\n\n## Collector configuration\n:::tipTo be coming soon\nFollow this [Github issue](https://github.com/godspeedsystems/gs-node-service/issues/1018) for more updates.\n:::\n" }, "docs/microservices-framework/telemetry/metrics.md": { "content": "---\nsidebar_position: 1\ntitle: Metrics\ntoc_min_heading_level: 2\ntoc_max_heading_level: 3\n---\n\nGodspeed supports **Prometheus-based metrics** out of the box. These metrics help you monitor the performance and behavior of your services, including request rates, latencies, and Prisma database query statistics.\n\n### When Are Metrics Available?\n\nMetrics are exposed automatically at the `/metrics` endpoint **once OpenTelemetry (OTEL) is enabled**.\n\nTo enable metrics:\n\n```env\nOTEL_ENABLED=true\n```\n---\n\n### Default Behavior\n\nWhen OTEL is enabled:\n\n* The `/metrics` endpoint is registered by default.\n* It exposes:\n * **HTTP request metrics** (via the Express plugin)\n * **Prisma datasource metrics** (if configured)\n\n---\n\n### How to Customize Metrics Behavior\n\nYou can customize metric collection in your HTTP event source (`http.yaml`) like so:\n\n```yaml\ntype: express\nport: 3000\nmetrics:\n\n metricsPath: '/metrics' \n requestDurationBuckets: [0.2, 0.5, 1, 1.5, 2, 2.5] # Opttional: Custom latency buckets (in seconds)\n requestLengthBuckets: [128, 256, 512, 1024] # Optional: Track incoming request sizes\n responseLengthBuckets: [256, 512, 1024, 2048] # Optional: Track outgoing response sizes\n```\n\n### Overriding `/metrics` Endpoint (Optional)\n\nIf you want full control (e.g., to combine multiple metrics like HTTP + multiple Prisma clients), you can disable the default metrics route and define your own:\n\n#### 1. In `http.yaml`:\n\n```yaml\nmetrics:\n metricsPath: '/custom-metrics'\n\n```\n\n#### 2. In your `express.ts` event source:\n\n```ts\nclient.get('/metrics', async (req, res) => {\n const expressMetrics = await promClient.register.metrics();\n const prismaMetrics = await prismaClient.$metrics.prometheus();\n res.set('Content-Type', promClient.register.contentType);\n res.end(`${expressMetrics}\\n${prismaMetrics}`);\n});\n```\n\n### What Metrics Are Available?\n\n#### HTTP Metrics (via Express)\n\nAutomatically exposed when OTEL is enabled:\n\n* `http_requests_total`\n* `http_request_duration_seconds`\n* `http_request_size_bytes`\n* `http_response_size_bytes`\n\nThese metrics include labels like `method`, `route`, and `status_code`.\n\n#### Prisma Metrics (via `$metrics.prometheus()`)\n\nIf you have registered prisma metrics manually by setting upa middleware, then, you get:\n\n* `prisma_client_queries_total`\n* `prisma_client_queries_active`\n* `prisma_client_queries_wait`\n* ...and more depending on your query workload\n\n\n### Testing Metrics\n\nOnce your app is running with OTEL enabled:\n\n```bash\ncurl http://localhost:3000/metrics\n```\n\nYou should see all registered Prometheus metrics.\n\n> Use tools like **Prometheus**, **Grafana**, or **Datadog** to scrape and visualize this data.\n\n---\n\n### Example Output Snippet\n\n```\n# HELP http_request_duration_seconds Duration of HTTP requests\n# TYPE http_request_duration_seconds histogram\nhttp_request_duration_seconds_bucket{method=\"GET\",route=\"/helloworld\",status_code=\"200\",le=\"1\"} 10\n\n# HELP prisma_client_queries_total Total number of Prisma client queries executed\n# TYPE prisma_client_queries_total counter\nprisma_client_queries_total{datasource=\"main\"} 1023\n```\n\n---\n\n## Summary\n\nWith a few lines of configuration, Godspeed enables robust Prometheus-based observability:\n\n* **Plug-and-play setup with OTEL**\n* **Customizable bucket metrics for request profiling**\n* **Flexible override of the default `/metrics` endpoint**\n\n> Start observing, debugging, and scaling your services with real-time insights today!\n\n---\n\n:::info\n\nMetrics for custom plugins\n\nFollow this [Github issue](https://github.com/godspeedsystems/gs-node-service/issues/1016) to know how prometheus based metrics can be exposed for the other custom [eventsource](../event-sources/event-source-plugins/) and [datasource](../datasources/datasource-plugins/Overview.md) plugins.\n:::\n\n\n" }, "docs/microservices-framework/telemetry/overview.md": { "content": "---\nsidebar_position: 1\ntitle: Observability\ntoc_min_heading_level: 2\ntoc_max_heading_level: 4\n---\n\nObservability means to understand and analyze an application’s internal state at any given time based only on the knowledge of its external outputs. \n\n## Why you need observability?\nIt allows developers to have access to more accurate information about system faults in distributed environments, without additional testing and coding. The more observable a system, the more quickly and accurately you can reach out to the root cause of a problem.\n\nAn observability implementation includes a practice of collecting telemetry data such as: \n**1. Metrics**: Collecting values about known performance measurements from your applications and infrastructure you can put on dashboards or use for alerting. It helps you find out when there’s a problem. \n**2. Traces**: Collecting the trace of user requests throughout the various components of your applications and infrastructure. It helps you find out where a problem happens. \n**3. Logs**: Collecting errors, warnings, and other information about events happening within applications and infrastructure. It helps you find out the cause of a problem. \n\n## Using OpenTelemetry\n[OpenTelemetry](https://opentelemetry.io/docs/), also known as OTel, is a CNCF incubating project and a vendor-neutral open source observability framework for instrumenting, generating, collecting, and exporting telemetry data i.e. traces, metrics, and logs.\n\nGodspeed supports Application Performance Monitoring(APM) and Business Performance Monitoring(BPM) out of the box by leveraging the OpenTelemetry standard and its supporting tech ecosystem. \n\n> Not even a single request must go untracked!\n\n:::tip\n\nCheck out the video talks on observability with Godspeed\n\n1. [TEMPLE](https://www.youtube.com/watch?v=2BtCi72LPlM)\n2. [Grafana and OTEL based observability](https://www.youtube.com/watch?v=hOKwwYdofcA)\n:::\n\n## Goals\n\n### Auto application performance monitoring\n\nNo code APM across microservices, integrable with standard APM tools and logging backends, without any dev effort.\n\n### Backend agnostic\n\nNumerous open source and commercial softwares for Observability support OpenTelemetry out of the box, allowing one to switch between them if needed.\n\n### Complete debuggability\n\nCollect, correlate and debug signals across logs (events), traces and metrics, based on the request id and the attributes defined for the organization. For example, app version, function, DB query, K8s pod, domain, microservice etc.\n\n### Enable/disable observability\nYou can [enable/disable OTEL](../CLI.md/#otel-enabling-and-disabling-open-telemetry) in Godspeed.\n\n\n" }, "docs/microservices-framework/telemetry/tracing.md": { "content": "---\nsidebar_position: 1\ntitle: Traces\ntoc_min_heading_level: 2\ntoc_max_heading_level: 3\n---\n\nTraces are sent to OTEL Collector directly by the application (push based mechanism) using OTLP/gRPC [protocol](https://opentelemetry.io/docs/specs/otlp/) as explained in the [architecture](./overview.md/#architecture).\n\n## Application configuration\n\n### OTEL_EXPORTER_OTLP_ENDPOINT\nSpecify the IP address of your OTEL collector as `OTEL_EXPORTER_OTLP_ENDPOINT` env variable. Refer [OTEL Exporter](https://opentelemetry.io/docs/reference/specification/protocol/exporter/#endpoint-urls-for-otlphttp) for more information.\n```\nexport OTEL_EXPORTER_OTLP_ENDPOINT=:4317\n```\nFor example,\n```\nexport OTEL_EXPORTER_OTLP_ENDPOINT=http://172.17.0.1:4317\n```\n\n\n" }, "docs/microservices-framework/vscode-extension/language-tools.md": { "content": "---\nsidebar_position: 1\ntitle: VS Code Extension.\ntoc_min_heading_level: 2\ntoc_max_heading_level: 4\n---\n\n## Introduction\n\nHello developers Thank You for using Godspeed Microservice Framework. \n\n### Pre-requisites:\n\n1. VS Code editor\n\n2. Godspeed Framework Version V2\n\n\n### Steps for Using the Language Tool Extension.\n\n- Open your [VS Code](https://code.visualstudio.com/) app.\n\n- Go to the Extension section and search Godspeed Language Tools. \n\n- Install the Godspeed Language Tool Extension in your VS Code. \n\n![languagetools](/img/godspeedlanguagetools.png)\n\n\n#### Introduction to Snippets \n\n- We have introduced few features in the extension and one of them is,[Snippets](https://code.visualstudio.com/docs/editor/userdefinedsnippets) now there are Godspeed's snippets for event and workflow.\n\n- If you type `Godspeed Event` in the src/event (path) file then it will give you some sample data to make an event.\n\n- In the Event Snippets contains the event string , authn , summary, description, fn, on_validation_error, sample params, sample body, responses. After creating an Event you can configure the code with your requirements For Ex. Changing the body, params , responses. You can change the values by pressing up and down arrow and tab or Enter to move next.\n\n- Sample of eventsnippet\n\n![preview](/img/video-gif/eventsnippet.gif \"Godspeed Event Sample\")\n\n\n```\nhttp.get./event-endpoint:\n authn: false\n summary: 'summary of the event'\n description: 'description of the event'\n fn: 'full path of the function, seprated with dots, remove .yaml, like: com.biz.helloworld'\n on_validation_error: com.jfs.handle_validation_error\n params:\n - name: world\n in: query\n required: true\n body:\n content:\n application/json:\n schema:\n type: object\n properties:\n name:\n type: string\n email:\n type: string\n required: [name , email]\n responses:\n '200':\n description: OK\n content:\n application/json:\n schema:\n type: object\n```\n\n\n- If you type `Godspeed Workflow` in the src/workflow (path) then it will give a snippet of sample workflow which includes summary, description, id, tasks, fn and args.\n\n\n![preview](/img/video-gif/workflowsnippet.gif \"Godspeed Workflow Sample\")\n\n```\nsummary: 'the title'\ndescription: 'more details'\nid: 'unique_ID'\ntasks:\n - id: 'unique_id_of_the_task'\n fn: com.gs.return\n args: <% %>\n```\n- You can also add Built-in Tasks in the workflows. There are snippets for that task. \n\n- For Examples -\n\n![preview](/img/video-gif/workflowsnippet.gif \"Godspeed Workflow Sample\")\n\n\n#### Rules to Write an Event\n\n##### Mandatory Fiels\n\n- There are some key-value pairs in the event and workflows, which needs to be filled to get a proper output, below are the examples of it.\n\n\n#### In Event\n\n- There are few key-value pairs which are important to the workflow, those are mandatory in the event.\n\n - For params - \n\n - names , in \n\n ```\n params:\n - name: world\n in: query\n required: true\n ```\n\n ![preview](/img/video-gif/paramsSnippet.gif \"Godspeed params Sample\")\n\n - For Body - \n\n - content , application/json, schema, type\n\n ```\n body:\n description: This is the Description of body\n required: false\n content:\n application/json:\n schema:\n type: string\n ```\n![preview](/img/video-gif/bodysnippet.gif \"Godspeed body Sample\")\n\n - For responses - \n\n - 200, content, application/json schema, type\n\n ```\n responses:\n 200:\n description: OK\n content:\n application/json:\n schema:\n type: object\n ```\n![preview](/img/video-gif/resopnseSnippet.gif \"Godspeed response Sample\")\n\n\n###### In Workflow \n\n - in the Workflow there are some key-value pairs that are important to make good workflows.\n\n - tasks , id, fn , args \n\n \n### Rules\n\n\n#### Event \n\n - Minimun length of the description and summary is 50 letters and minimum length is 2.\n\n\n\n#### Workflow \n - To define an Id in the Workflow id should be all in small letters , without spaces, without numbers and minimum charactors is 2 and maximum length is 45\n\n ```\n String does not match the pattern of \"^[^\\s]+$\"\n ```\n\n - Minimum length of the summary and description are 2 and maximum length is 50\n ```\n String is longer than the maximum length of 50.\n ```\n\n - fn should be one of inbuilt functions or from the plugins (it starts with datasources)\n\n### For more information you can watch our full Deatiled video on language tools and snippets of it on our [Youtube](https://www.youtube.com/embed/Yir19zd492I) channel. \n\n
\n \n
\n\n\n### Thank You !" }, "docs/microservices-framework/workflows/native-language-functions.md": { "content": "# Writing functions in Godspeed\n\nSince the framework currently supports Node.js, Deno and Bun.js ecosystems, the native languages currently supported are TypeScript and JavaScript. This allows users to create custom functions. A native language workflow enables us to incorporate additional features using JavaScript or TypeScript, where we have the capability to implement intricate business logic.\n\n:::tip\nIn Godspeed, your function gets input in a standard JSON format and returns output in a standard JSON format, independent of the eventsource through which this function is triggered. Eventsource could be Express, Fastify, Apollo Graphql or event message bus like Kafka, RabbitMQ or socket message. This means Godspeed has a unified way to deal with all eventsources, providing a modular architecture and re-uasability of your functions.\n:::\n\n### Example Typescript function\n```typescript\nimport { GSCloudEvent, GSContext } from \"@godspeedsystems/core\";\nimport Pino from 'pino';\n\nexport default function (ctx: GSContext) {\n const {\n inputs: {\n data: {\n params, //path parameters from endpoint url\n body, // request body in case of http and graphql apis, event data in case of message bus or socket\n query, // query parameters from rest api\n user, // user payload parsed from jwt token\n headers //request headers in case of http and graphql apis\n }\n }, \n childLogger, // context specific logger. Read pino childLogger for more information\n logger, // Basic logger of the project, generally prefer childLogger for logging \n outputs, // outputs of previously executed tasks of yaml workflows (if any)\n functions, // all loaded workflows/functions from the src/functions/ folder\n datasources, //all configured datasources from src/datasources\n mappings //mappings from src/mappings folder. this is useful for loading key value configurations for business logic of your project\n }: {\n inputs: GSCloudEvent, \n childLogger: Pino.Logger, // you can also add custom attributes to childLogger\n logger: Pino.Logger,\n outputs: PlainObject, \n functions: PlainObject, \n datasources: PlainObject,\n mappings: PlainObject\n } = ctx;\n\n // Will print with workflow_name and task_id attributes. \n childLogger.info('Server is running healthy');\n // Will print without workflow_name and task_id attributes\n logger.info('Inputs object \\n user %o query %o body %o headers %o params %o', user, query, body, headers, params);\n logger.info('Outputs object has outputs from previous tasks with given ids %o', Object.keys(outputs));\n logger.info('Datasources object has following datasource clients %o', Object.keys(datasources));\n logger.info('Total functions found in the project %s', Object.keys(functions).length);\n\n // Returning only data\n return 'Its working! ' + body.name;\n\n //SAME AS\n return {\n data: 'Its working! ' + body.name,\n code: 200,\n // success: true,\n // headers: undefined\n }\n //SAME AS\n return {\n data: 'Its working! ' + body.name,\n code: 200,\n success: true,\n headers: undefined // or u can set response headers as key: value pairs, \n //for example headers:{custom-header1:\"xyz\" }\n }\n}\n```\n\n:::tip\n\nFor seeing how framework handles data returned from a function, including calculation of `code`, `success` and `data`, [read this section](./native-language-functions#handling-return-from-godspeed-functions) at the bottom of the page. \n:::\n\n### GSContext\n\nGSContext carries the loaded components of this project and as well the inputs of the current event. Every information you need to know or store about the event and the workflow executed so far, and as well the loaded `functions`, `datasources`, `logger`, `childLogger`, `config`, `mappings` etc, is available in the `GSContext` object.\n\n:::tip note\n Every function/workflow has access to the ctx object, which is passed as an argument, and furthermore, you can access its properties by destructuring it.\n:::\n\n### More about GSContext\n\nCheck the code of GSContext interface [here](https://github.com/godspeedsystems/gs-node-service/blob/v2/src/core/interfaces.ts). GSContext has the contextual information of your current workflow and is available to the event handlers (`functions`). It is passed to any sub workflows subsequently called by the event handler. \nIt includes all the context specific information like tracing information, actor, environment, headers, payload etc.\n\n\n\n\n\n\n\n\n\n\n\n#### Inputs\n\nInputs Provide you all the Information you passed to event like `headers`, `params`, `query`, `params` (path params), `body` & `user` (parsed JWT information).\n\n\n```javascript\n const {inputs} = ctx;\n const body = inputs.data.body;\n```\n#### Outputs\n\nTo access outputs of tasks executed before the current task, developer can destruct ctx object just like how inputs and datasources.If we have more then one task, we can access first task outputs in second task with Outputs object. we should access first task output by useing it's id.\n\n```javascript\n const {outputs} = ctx;\n const firstTaskOutput = outputs[firstTaskId]\n```\n\n### Accessing Datasource Clients\n \nWith `datasources` we can access all [datasources](../datasources/overview.md), their clients and methods.\n\n```javascript\nconst { datasources} = ctx;\nconst responseData = await datasources.prismaSchema.client.User.create({\n data: inputs.body\n})\n\n```\n### ChildLogger\n\nWith childLogger you have accessibility to Pino logger instance with context information set - for example the `log.attributes` set in eventsource or event level.\n\n```javascript\n const { childLogger} = ctx;\n childLogger.info('inputs: %o', inputs.body);\n\n```\n\n### Returning from a function\n\n### GSStatus\n\nDevelopers can return pure JSON object in response, or return GSStatus if they use Typescript.\nThe GSStatus is a built-in class in Godspeed. We invoke it when we're prepared to define an API response manually and dispatch it.\n\nGSStatus has the below properties.\n```yaml\n success: boolean;\n code?: number;\n message?: string;\n data?: any;\n headers?: {\n [key: string]: any;\n };\n```\n\nWe return with GSStatus as below\n\n```typescript\n return new GSStatus(true, 200, 'OK', responseData, headers);\n```\n\n#### Different ways to return from a function\n```typescript\n // Returning only data\n return 'Its working! ' + body.name;\n\n //SAME AS\n return {\n data: 'Its working! ' + body.name,\n code: 200,\n message: 'OK', //HTTP protocol message to be returned from service\n // success: true,\n // headers: undefined\n }\n //SAME AS\n return {\n data: 'Its working! ' + body.name,\n code: 200,\n message: 'OK', //HTTP protocol message to be returned from service\n success: true,\n // headers: undefined\n }\n //SAME AS\n return {\n data: 'Its working! ' + body.name,\n code: 200,\n message: 'OK', //HTTP protocol message to be returned from service\n success: true,\n headers: undefined\n }\n // SAME AS returning GSStatus like this\n return new GSStatus(true, 200, 'OK', 'Its working! ' + body.name, headers); \n```\n\n:::tip\nCheck [event handler response](#handling-return-from-godspeed-functions) to know how framework handles GSStatus.\n:::\n\n### Invoking other functions\n- All functions within a Godspeed project, including those written in YAML, JavaScript (JS), or TypeScript (TS), are accessible through the `ctx.functions` object.\n- Ofcourse you can also `import` them in the standard Typescript or Javascript way\n\n ```ts\n export default async function (ctx: GSContext) {\n\n //Calling functions (yaml, js, ts) from within a ts/js function, \n // all functions written inside src/functions folder are available under ctx.functions.\n const res = await ctx.functions['com.gs.helloworld2'](ctx);\n //Same As\n const res = await require('com/gs/helloworld2')(ctx);\n return res\n // works same as return new GSStatus(true, 200, 'OK', res );\n}\n\n ```\n\n### Invoking datasource clients from functions\nSimilarly, all datasource clients initialised in a Godspeed project are conveniently available under the `ctx.datasources` object.\n\nThere are two options for invoking datasource clients from functions:\n\n**Option 1: Using the client key**\n\nEvery datasource exposes a client key. The client may be a single instance like in the case of Axios, or multiple datasource client instances like in the case of AWS, or database models like in the case of Mongoose (depending on the plugin being used).\n\n**Option 2: Using the execute method**\n\nAll datasources have an execute method. This may be preferable in case you want to utilise the full capabilities of the plugin wrapped over the native clients, like error handling checks and response codes, retries, caching etc.\n\n```ts\n export default async function (ctx: GSContext, args: any) {\n // Calling datasource functions from ctx.datasources\n\n // OPTION 1 Using the client key\n const res = await ctx.datasources.aws.client.s3.listBuckets(args);\n\n // OPTION 2 Using the execute method\n const res = await ctx.datasources.aws.execute(ctx, {\n \n //Pass exactly same args as this aws service's method takes\n ...args, //Along with args, pass meta object\n meta: { // meta can contain {entityType, method}\n entityType: 's3', \n method: 'listBuckets'\n }\n });\n if (!res.success) {\n return new GSStatus(false, res.code || 500, 'my response message', {message: \"Internal Server Error\", info: res.message})\n }\n //If a developer only returns data without setting keys like \"success\" or \"code\" in the response,\n // the framework assumes it is just the data.\n //In such cases, the response code defaults to 200, and success is assumed to be true.\n \n return res\n // works same as return new GSStatus(true, 200, 'OK', res );\n}\n\n ```\n### Handling return from godspeed functions\n\nBy default, all the framework defined functions or developer written functions, have to return either [GSStatus](#gsstatus) or data. \nNow lets see how the framework qualifies your return as GSStatus or simple data.\nThe framework sees that your returned data has one of `code` or `success` meta-keys. \n \n**i.** If both are present, it looks for the other GSStatus keys and set them. \n**ii.** If only code is present and lies between 200-399, then success is assumed to be true else false. It looks for the other GSStatus keys and set them. \n**iii.** If only success is present, then code is assumed to be 200. It looks for the other GSStatus keys and set them. \n**iv.** If it doesn't find any of these keys, it assumes all that you have returned is intended to be GSStatus.data then it adds `code: 200` and `success: true` internally to your response and create a `GSStatus` out of it to pass on to next tasks or workflows. \n\n\n\n:::info\nYou can study the code [here](https://github.com/godspeedsystems/gs-node-service/blob/v2/src/functions/com/gs/transform.ts) to understand the above scenario better.\n:::\n" }, "docs/microservices-framework/workflows/overview.md": { "content": "---\ntitle: Functions (Event Handler Workflows)\n---\n\nThe Meta Framework allows you to write business logic in [Typescript and Javascript functions](/docs/microservices-framework/workflows/native-language-functions.md)\nYou may use any language depending on your choice. We suggest to prefer Typescript over Javascript for type check reasons. \n\n\n### Video explanation of functions\n\n\n\n
\n\n
\n\n### Writing functions\n\n#### Typescript\n```typescript\nmodule.exports = function greet(GSContext: ctx){\n return `Hello ${ctx.inputs.data.query.name}!`;\n}\n```\n#### Javascript\n```javascript\nmodule.exports = function greet(ctx){\n return `Hello ${ctx.inputs.data.query.name}!`;\n}\n```\n\n" }, "docs/saarthi/advanced-usage/available-tools/access-mcp-resource.md": { "content": "# access_mcp_resource\n\nThe `access_mcp_resource` tool retrieves data from resources exposed by connected Model Context Protocol (MCP) servers. It allows Saarthi to access files, API responses, documentation, or system information that provides additional context for tasks.\n\n## Parameters\n\nThe tool accepts these parameters:\n\n- `server_name` (required): The name of the MCP server providing the resource\n- `uri` (required): The URI identifying the specific resource to access\n\n## What It Does\n\nThis tool connects to MCP servers and fetches data from their exposed resources. Unlike `use_mcp_tool` which executes actions, this tool specifically retrieves information that serves as context for tasks.\n\n## When is it used?\n\n- When Saarthi needs additional context from external systems\n- When Saarthi needs to access domain-specific data from specialized MCP servers\n- When Saarthi needs to retrieve reference documentation hosted by MCP servers\n- When Saarthi needs to integrate real-time data from external APIs via MCP\n\n## Key Features\n\n- Retrieves both text and image data from MCP resources\n- Requires user approval before executing resource access\n- Uses URI-based addressing to precisely identify resources\n- Integrates with the Model Context Protocol SDK\n- Displays resource content appropriately based on content type\n- Supports timeouts for reliable network operations\n- Handles server connection states (connected, connecting, disconnected)\n- Discovers available resources from connected servers\n- Processes structured response data with metadata\n- Handles image content special rendering\n\n## Limitations\n\n- Depends on external MCP servers being available and connected\n- Limited to the resources provided by connected servers\n- Cannot access resources from disabled servers\n- Network issues can affect reliability and performance\n- Resource access subject to configured timeouts\n- URI formats are determined by the specific MCP server implementation\n- No offline or cached resource access capabilities\n\n## How It Works\n\nWhen the `access_mcp_resource` tool is invoked, it follows this process:\n\n1. **Connection Validation**:\n - Verifies that an MCP hub is available and initialized\n - Confirms the specified server exists in the connection list\n - Checks if the server is disabled (returns an error if it is)\n\n2. **User Approval**:\n - Presents the resource access request to the user for approval\n - Provides server name and resource URI for user verification\n - Proceeds only if the user approves the resource access\n\n3. **Resource Request**:\n - Uses the Model Context Protocol SDK to communicate with servers\n - Makes a `resources/read` request to the server through the MCP hub\n - Applies configured timeouts to prevent hanging on unresponsive servers\n\n4. **Response Processing**:\n - Receives a structured response with metadata and content arrays\n - Processes text content for display to the user\n - Handles image data specially for appropriate display\n - Returns the processed resource data to Saarthi for use in the current task\n\n## Resource Types\n\nMCP servers can provide two main types of resources:\n\n1. **Standard Resources**:\n - Fixed resources with specific URIs\n - Defined name, description, and MIME type\n - Direct access without parameters\n - Typically represent static data or real-time information\n\n2. **Resource Templates**:\n - Parameterized resources with placeholder values in URIs\n - Allow dynamic resource generation based on provided parameters\n - Can represent queries or filtered views of data\n - More flexible but require additional URI formatting\n\n## Examples When Used\n\n- When helping with API development, Saarthi retrieves endpoint specifications from MCP resources to ensure correct implementation.\n- When assisting with data visualization, Saarthi accesses current data samples from connected MCP servers.\n- When working in specialized domains, Saarthi retrieves technical documentation to provide accurate guidance.\n- When generating industry-specific code, Saarthi references compliance requirements from documentation resources.\n\n## Usage Examples\n\nAccessing current weather data:\n```\n\nweather-server\nweather://san-francisco/current\n\n```\n\nRetrieving API documentation:\n```\n\napi-docs\ndocs://payment-service/endpoints\n\n```\n\nAccessing domain-specific knowledge:\n```\n\nknowledge-base\nkb://medical/terminology/common\n\n```\n\nFetching system configuration:\n```\n\ninfra-monitor\nconfig://production/database\n\n```\n" }, "docs/saarthi/advanced-usage/available-tools/apply-diff.md": { "content": "# apply_diff\n\nThe `apply_diff` tool makes precise, surgical changes to files by specifying exactly what content to replace. It uses a sophisticated strategy for finding and applying changes while maintaining proper code formatting and structure.\n\n## Parameters\n\nThe tool accepts these parameters:\n\n- `path` (required): The path of the file to modify relative to the current working directory.\n- `diff` (required): The search/replace block defining the changes using a format specific to the active diff strategy.\n- `start_line` (optional): A hint for where the search content begins. _Note: This top-level parameter appears unused by the current main strategy, which relies on `:start_line:` within the diff content._\n- `end_line` (optional): A hint for where the search content ends. _Note: This top-level parameter appears unused by the current main strategy._\n\n## What It Does\n\nThis tool applies targeted changes to existing files using fuzzy matching guided by line number hints to locate and replace content precisely. Unlike simple search and replace, it identifies the exact block for replacement based on the provided content and location hints.\n\n## When is it used?\n\n- When Saarthi needs to make precise changes to existing code without rewriting entire files.\n- When refactoring specific sections of code while maintaining surrounding context.\n- When fixing bugs in existing code with surgical precision.\n- When implementing feature enhancements that modify only certain parts of a file.\n\n## Key Features\n\n- Uses fuzzy matching (Levenshtein distance on normalized strings) guided by a `:start_line:` hint, with configurable confidence thresholds (typically 0.8-1.0).\n- Provides context around matches using `BUFFER_LINES` (default 40).\n- Performs a middle-out search within a configurable context window (`bufferLines`) around the hinted start line.\n- Preserves code formatting and indentation passively by replacing exact blocks.\n- Shows changes in a diff view for user review and editing before applying.\n- Tracks consecutive errors per file (`consecutiveMistakeCountForApplyDiff`) to prevent repeated failures.\n- Validates file access against `.saarthiignore` rules.\n- Handles multi-line edits effectively.\n\n## Limitations\n\n- Works best with unique, distinctive code sections for reliable identification.\n- Performance can vary with very large files or highly repetitive code patterns.\n- Fuzzy matching might occasionally select incorrect locations if content is ambiguous.\n- Each diff strategy has specific format requirements.\n- Complex edits might require careful strategy selection or manual review.\n\n## How It Works\n\nWhen the `apply_diff` tool is invoked, it follows this process:\n\n1. **Parameter Validation**: Validates required `path` and `diff` parameters.\n2. **RooIgnore Check**: Validates if the target file path is allowed by `.saarthiignore` rules.\n3. **File Analysis**: Loads the target file content.\n4. **Match Finding**: Uses a fuzzy matching algorithm (Levenshtein on normalized strings) guided by the `:start_line:` hint within a context window (`BUFFER_LINES`), searching middle-out to locate the target content based on the confidence threshold.\n5. **Change Preparation**: Generates the proposed changes by replacing the identified block.\n6. **User Interaction**:\n * Displays the changes in a diff view.\n * Allows the user to review and potentially edit the proposed changes.\n * Waits for user approval or rejection.\n7. **Change Application**: If approved, applies the changes (potentially including user edits) to the file.\n8. **Error Handling**: If errors occur (e.g., match failure, partial application), increments the `consecutiveMistakeCountForApplyDiff` for the file and reports the failure type.\n9. **Feedback**: Returns the result, including any user feedback or error details.\n\n## Diff Format Requirements\n\nThe `` parameter requires a specific format supporting one or more changes in a single request. Each change block requires a line number hint for the original content.\n\n* **Requires**: Exact match for the `SEARCH` block content (within the fuzzy threshold), including whitespace and indentation. The `:start_line:` number hint is mandatory within each block. The `:end_line:` hint is optional (but supported by the parser). Markers like `<<<<<<<` within the file's content must be escaped (`\\\\`) in the SEARCH block.\n\nExample format for the `` block:\n\n```diff\n<<<<<<< SEARCH\n:start_line:10\n:end_line:12\n-------\n // Old calculation logic\n const result = value * 0.9;\n return result;\n=======\n // Updated calculation logic with logging\n console.log(`Calculating for value: ${value}`);\n const result = value * 0.95; // Adjusted factor\n return result;\n>>>>>>> REPLACE\n\n<<<<<<< SEARCH\n:start_line:25\n:end_line:25\n-------\n const defaultTimeout = 5000;\n=======\n const defaultTimeout = 10000; // Increased timeout\n>>>>>>> REPLACE" }, "docs/saarthi/advanced-usage/available-tools/ask-followup-question.md": { "content": "# ask_followup_question\n\nThe `ask_followup_question` tool enables interactive communication by asking specific questions to gather additional information needed to complete tasks effectively.\n\n## Parameters\n\nThe tool accepts these parameters:\n\n- `question` (required): The specific question to ask the user\n- `follow_up` (optional): A list of 2-4 suggested answers that help guide user responses, each within `` tags\n\n## What It Does\n\nThis tool creates a conversational interface between Saarthi and the user, allowing for gathering clarification, additional details, or user preferences when facing ambiguities or decision points. Each question can include suggested responses to streamline the interaction.\n\n## When is it used?\n\n- When critical information is missing from the original request\n- When Saarthi needs to choose between multiple valid implementation approaches\n- When technical details or preferences are required to proceed\n- When Saarthi encounters ambiguities that need resolution\n- When additional context would significantly improve the solution quality\n\n## Key Features\n\n- Provides a structured way to gather specific information without breaking workflow\n- Includes suggested answers to reduce user typing and guide responses\n- Maintains conversation history and context across interactions\n- Supports responses containing images and code snippets\n- Available in all modes as part of the \"always available\" tool set\n- Enables direct user guidance on implementation decisions\n- Formats responses with `` tags to distinguish them from regular conversation\n- Resets consecutive error counter when used successfully\n\n## Limitations\n\n- Limited to asking one specific question per tool use\n- Presents suggestions as selectable options in the UI\n- Cannot force structured responses – users can still respond freely\n- Excessive use can slow down task completion and create a fragmented experience\n- Suggested answers must be complete, with no placeholders requiring user edits\n- No built-in validation for user responses\n- Contains no mechanism to enforce specific answer formats\n\n## How It Works\n\nWhen the `ask_followup_question` tool is invoked, it follows this process:\n\n1. **Parameter Validation**: Validates the required `question` parameter and checks for optional suggestions\n - Ensures question text is provided\n - Parses any suggested answers from the `follow_up` parameter using the `fast-xml-parser` library\n - Normalizes suggestions into an array format even if there's only one suggestion\n\n2. **JSON Transformation**: Converts the XML structure into a standardized JSON format for UI display\n ```typescript\n {\n question: \"User's question here\",\n suggest: [\n { answer: \"Suggestion 1\" },\n { answer: \"Suggestion 2\" }\n ]\n }\n ```\n\n3. **UI Integration**:\n - Passes the JSON structure to the UI layer via the `ask(\"followup\", ...)` method\n - Displays selectable suggestion buttons to the user in the interface\n - Creates an interactive experience for selecting or typing a response\n\n4. **Response Collection and Processing**:\n - Captures user text input and any images included in the response\n - Wraps user responses in `` tags when returning to the assistant\n - Preserves any images included in the user's response\n - Maintains the conversational context by adding the response to the history\n - Resets the consecutive error counter when the tool is used successfully\n\n5. **Error Handling**:\n - Tracks consecutive mistakes using a counter\n - Resets the counter when the tool is used successfully\n - Provides specific error messages:\n - For missing parameters: \"Missing required parameter 'question'\"\n - For XML parsing: \"Failed to parse operations: [error message]\"\n - For invalid format: \"Invalid operations xml format\"\n - Contains safeguards to prevent tool execution when required parameters are missing\n - Increments consecutive mistake count when errors occur\n\n## Workflow Sequence\n\nThe question-answer cycle follows this sequence:\n\n1. **Information Gap Recognition**: Saarthi identifies missing information needed to proceed\n2. **Specific Question Creation**: Saarthi formulates a clear, targeted question\n3. **Suggestion Development**: Saarthi creates relevant suggested answers (optional but recommended)\n4. **Tool Invocation**: Assistant invokes the tool with question and optional suggestions\n5. **UI Presentation**: Question and suggestions are displayed to the user as interactive elements\n6. **User Response**: The user selects a suggestion or provides a custom answer\n7. **Message Handling**: System handles both partial and complete messages\n - For streaming responses, processes chunks as they arrive\n - For complete messages, processes the entire response at once\n - Maintains state consistency regardless of message chunking\n8. **Response Processing**: System wraps the response in `` tags and preserves images\n9. **Context Integration**: Response is added to the conversation history\n10. **Task Continuation**: Saarthi proceeds with the task using the new information\n\n## Examples When Used\n\n- When developing a web application, Saarthi might ask about preferred styling frameworks (Bootstrap, Tailwind, custom CSS)\n- When creating an API, Saarthi might ask about authentication methods (JWT, OAuth, API keys)\n- When refactoring code, Saarthi might ask about prioritizing performance vs. readability\n- When setting up a database, Saarthi might ask about specific schema design preferences\n- When creating a custom feature, Saarthi might ask about specific behavior expectations\n- When troubleshooting errors, Saarthi might ask about specific environment details\n\n## Response Format\n\nWhen a user responds to a question, the response is formatted with `` tags for clarity and consistency. This format helps clearly separate the user's answer from other conversation elements.\n\nExample response format:\n```\n\nUse Tailwind CSS for utility-first styling with maximum flexibility\n\n```\n\nKey characteristics of the response format:\n- Surrounded by `` tags\n- Preserves formatting of the original response\n- Maintains images if included in the response\n- Returns the complete text, even for long responses\n\n## Usage Examples\n\nAsking about implementation preferences:\n```\n\nWhich styling approach would you prefer for this web application?\n\nUse Bootstrap for rapid development with consistent components\nUse Tailwind CSS for utility-first styling with maximum flexibility\nUse vanilla CSS with custom styling for complete control and minimal dependencies\n\n\n```\n\n\nRequesting technical clarification:\n```\n\nWhat database should this application use for storing user data?\n\nMongoDB for flexible schema and document-based storage\nPostgreSQL for relational data with strong consistency guarantees\nFirebase for real-time updates and simplified backend management\nSQLite for lightweight local storage without external dependencies\n\n\n```\n\nAddressing ambiguity in requirements:\n```\n\nHow should the application handle user authentication?\n\nImplement email/password authentication with account verification\nUse social login providers (Google, GitHub, etc.) for quick signup\nImplement both email/password and social login options\n\n\n```\n" }, "docs/saarthi/advanced-usage/available-tools/attempt-completion.md": { "content": "# attempt_completion\n\nThe `attempt_completion` tool signals that Saarthi believes a task is complete and presents results to the user. It provides a summary of what was accomplished, optionally includes a command to demonstrate the result, and supports continued refinement through user feedback.\n\n## Parameters\n\nThe tool accepts these parameters:\n\n- `result` (required): The final result description summarizing what was accomplished\n- `command` (optional): A CLI command to execute to demonstrate the result\n\n## What It Does\n\nThis tool marks the end of a task by presenting a final summary of what was accomplished. It signals to both the user and the system that the current task is complete. When a command is provided, it can also demonstrate the result through a live execution.\n\n## When is it used?\n\n- When Saarthi believes the user's task has been completed\n- When summarizing what changes or improvements were made\n- When demonstrating the current result with a command execution\n- When providing a checkpoint for user feedback on the solution\n- When transitioning from one phase of work to potential refinements\n\n## Key Features\n\n- Provides a clear signal that Saarthi believes the task is complete\n- Summarizes accomplishments in a concise message\n- Optionally demonstrates results through command execution\n- Enables user feedback for further refinements\n- Displays results in a special UI format distinct from regular messages\n- Captures task completion telemetry for system analytics\n- Maintains a structured conversation flow by providing checkpoints\n- Supports subtask completion within larger workflows\n- Ensures users receive a clear summary of what was done\n- Available in all modes as part of the \"always available\" tool group\n\n## Limitations\n\n- Should not be used until previous tool uses are confirmed successful (guideline, not enforced)\n- Limited to a single command for result demonstration\n- Cannot present multiple command options\n- Commands require user approval before execution\n- Limited to demonstrating results that can be shown via CLI commands\n- Cannot be used for partial task completion or progress updates\n- Result formatting strips XML closing tags through internal processing\n\n## How It Works\n\nWhen the `attempt_completion` tool is invoked, it follows this process:\n\n1. **Safety Consideration** (guideline, not enforced):\n - The AI is instructed to confirm previous tool uses were successful\n - This is a best practice rather than a programmatically enforced mechanism\n\n2. **Result Presentation**:\n - Displays the completion message to the user in a special \"completion_result\" UI format\n - Removes XML closing tags from the result text using the `removeClosingTag` function\n - Presents the result differently than regular messages for visual distinction\n\n3. **Command Execution** (if provided):\n - Requests user approval before executing the command\n - Only executes if the user approves\n - Executes the command using the system's command execution functionality\n - Shows the result of the command to the user\n\n4. **Feedback Collection**:\n - Waits for user feedback on the completion result\n - Processes this feedback and returns it to the AI\n - Enables continued refinement based on user input\n\n5. **Task Completion and Continuation**:\n - Signals the task as completed in the system\n - Captures telemetry data for the completed task\n - For subtasks, offers to finish the subtask and resume the parent task\n - Supports continued conversation through the feedback mechanism\n\n6. **Implementation Integration**:\n - Tool results are parsed through the system's parsing mechanism in `parse-assistant-message.ts`\n - The tool is part of the \"ALWAYS_AVAILABLE_TOOLS\" constant, making it available in all modes\n\n## Result Formatting Guidelines\n\nThe result message should follow these guidelines:\n\n- Clearly communicate what was accomplished\n- Be concise but complete\n- Focus on the value delivered to the user\n- Avoid unnecessary pleasantries or filler text\n- Maintain a professional, straightforward tone\n- Present information in a way that's easy to scan and understand\n- Acknowledge that the user may provide feedback for further refinements\n\nNote: The system automatically strips XML closing tags from the result text through the `removeClosingTag` function.\n\n## Command Selection Guidelines\n\nWhen including a command, follow these guidelines:\n\n- Choose commands that visually demonstrate the result\n- Prefer commands that show the user what was created or modified\n- Examples include:\n * `open index.html` to display a created website\n * `npm start` to launch a development server\n * `python app.py` to run a created application\n- Avoid commands that merely print text (like `echo` or `cat`)\n- Remember that commands require user approval before execution\n- Ensure the command is valid for the user's operating system\n\n## Feedback and UI Representation\n\nThe `attempt_completion` tool has a unique feedback mechanism:\n\n- Results appear in a special \"completion_result\" UI format distinct from regular messages\n- The system waits for user feedback after presenting the result\n- Feedback is processed and returned to Saarthi for further refinements\n- This creates an iterative improvement cycle rather than ending the conversation\n- The UI includes special elements for providing feedback\n- Results serve as clear checkpoints in the conversation flow\n\nWhen used within subtasks:\n- The system offers to finish the subtask and resume the parent task\n- If approved, the subtask is completed with a summary\n- The parent task resumes with context from the completed subtask\n- This enables complex, nested workflows while maintaining context\n\n## Examples When Used\n\n- When creating a website, Saarthi presents the solution and uses `open index.html` to show the result. The user provides feedback, and Saarthi refines the design.\n- When implementing a feature, Saarthi summarizes the changes and their impact. The user requests additional test coverage, and Saarthi continues the task.\n- When fixing bugs, Saarthi explains what was fixed and how the solution addresses the root cause. The user confirms the fix works.\n- When setting up a development environment, Saarthi confirms all components are installed. The user requests additional configuration, and Saarthi continues.\n- When creating a data processing script, Saarthi summarizes the functionality. The user suggests performance improvements, and Saarthi optimizes the code.\n\n## Usage Examples\n\n\nCompleting a task with just a result summary:\n```\n\n\nI've updated the CSS for the navigation bar to match the design mockup. The changes include:\n1. Updated color scheme to use the new brand colors\n2. Added responsive behavior for mobile devices\n3. Fixed the alignment issues in the dropdown menus\n\n\n```\n\nCompleting a task with a result and demonstration command:\n```\n\n\nI've created a simple React application with the following features:\n1. A responsive landing page with the requested hero section\n2. A working contact form with validation\n3. Navigation between all three requested pages\n4. Styling that matches the provided design mockup\n\nnpm start\n\n```\n\nCompleting a Python script task with execution:\n```\n\n\nI've created a data processing script that:\n1. Reads the CSV files from the data directory\n2. Cleans and normalizes the data as requested\n3. Generates the summary statistics in JSON format\n4. Outputs the processed data to the output directory\n\npython process_data.py\n\n```\n" }, "docs/saarthi/advanced-usage/available-tools/browser-action.md": { "content": "# browser_action\n\nThe `browser_action` tool enables web automation and interaction via a Puppeteer-controlled browser. It allows Saarthi to launch browsers, navigate to websites, click elements, type text, and scroll pages with visual feedback through screenshots.\n\n## Parameters\n\nThe tool accepts these parameters:\n\n- `action` (required): The action to perform:\n * `launch`: Start a new browser session at a URL\n * `click`: Click at specific x,y coordinates\n * `type`: Type text via the keyboard\n * `scroll_down`: Scroll down one page height\n * `scroll_up`: Scroll up one page height\n * `close`: End the browser session\n- `url` (optional): The URL to navigate to when using the `launch` action\n- `coordinate` (optional): The x,y coordinates for the `click` action (e.g., \"450,300\")\n- `text` (optional): The text to type when using the `type` action\n\n## What It Does\n\nThis tool creates an automated browser session that Saarthi can control to navigate websites, interact with elements, and perform tasks that require browser automation. Each action provides a screenshot of the current state, enabling visual verification of the process.\n\n## When is it used?\n\n- When Saarthi needs to interact with web applications or websites\n- When testing user interfaces or web functionality\n- When capturing screenshots of web pages\n- When demonstrating web workflows visually\n\n## Key Features\n\n- Provides visual feedback with screenshots after each action and captures console logs\n- Supports complete workflows from launching to page interaction to closing\n- Enables precise interactions via coordinates, keyboard input, and scrolling\n- Maintains consistent browser sessions with intelligent page loading detection\n- Operates in two modes: local (isolated Puppeteer instance) or remote (connects to existing Chrome)\n- Handles errors gracefully with automatic session cleanup and detailed messages\n- Optimizes visual output with support for various formats and quality settings\n- Tracks interaction state with position indicators and action history\n\n## Browser Modes\n\nThe tool operates in two distinct modes:\n\n### Local Browser Mode (Default)\n- Downloads and manages a local Chromium instance through Puppeteer\n- Creates a fresh browser environment with each launch\n- No access to existing user profiles, cookies, or extensions\n- Consistent, predictable behavior in a sandboxed environment\n- Completely closes the browser when the session ends\n\n### Remote Browser Mode\n- Connects to an existing Chrome/Chromium instance running with remote debugging enabled\n- Can access existing browser state, cookies, and potentially extensions\n- Faster startup as it reuses an existing browser process\n- Supports connecting to browsers in Docker containers or on remote machines\n- Only disconnects (doesn't close) from the browser when session ends\n- Requires Chrome to be running with remote debugging port open (typically port 9222)\n\n## Limitations\n\n- While the browser is active, only `browser_action` tool can be used\n- Browser coordinates are viewport-relative, not page-relative\n- Click actions must target visible elements within the viewport\n- Browser sessions must be explicitly closed before using other tools\n- Browser window has configurable dimensions (default 900x600)\n- Cannot directly interact with browser DevTools\n- Browser sessions are temporary and not persistent across Saarthi restarts\n- Works only with Chrome/Chromium browsers, not Firefox or Safari\n- Local mode has no access to existing cookies; remote mode requires Chrome with debugging enabled\n\n## How It Works\n\nWhen the `browser_action` tool is invoked, it follows this process:\n\n1. **Action Validation and Browser Management**:\n - Validates the required parameters for the requested action\n - For `launch`: Initializes a browser session (either local Puppeteer instance or remote Chrome)\n - For interaction actions: Uses the existing browser session\n - For `close`: Terminates or disconnects from the browser appropriately\n\n2. **Page Interaction and Stability**:\n - Ensures pages are fully loaded using DOM stability detection via `waitTillHTMLStable` algorithm\n - Executes requested actions (navigation, clicking, typing, scrolling) with proper timing\n - Monitors network activity after clicks and waits for navigation when necessary\n\n3. **Visual Feedback**:\n - Captures optimized screenshots using WebP format (with PNG fallback)\n - Records browser console logs for debugging purposes\n - Tracks mouse position and maintains paginated history of actions\n\n4. **Session Management**:\n - Maintains browser state across multiple actions\n - Handles errors and automatically cleans up resources\n - Enforces proper workflow sequence (launch → interactions → close)\n\n## Workflow Sequence\n\nBrowser interactions must follow this specific sequence:\n\n1. **Session Initialization**: All browser workflows must start with a `launch` action\n2. **Interaction Phase**: Multiple `click`, `type`, and scroll actions can be performed\n3. **Session Termination**: All browser workflows must end with a `close` action\n4. **Tool Switching**: After closing the browser, other tools can be used\n\n## Examples When Used\n\n- When creating a web form submission process, Saarthi launches a browser, navigates to the form, fills out fields with the `type` action, and clicks submit.\n- When testing a responsive website, Saarthi navigates to the site and uses scroll actions to examine different sections.\n- When capturing screenshots of a web application, Saarthi navigates through different pages and takes screenshots at each step.\n- When demonstrating an e-commerce checkout flow, Saarthi simulates the entire process from product selection to payment confirmation.\n\n## Usage Examples\n\nLaunching a browser and navigating to a website:\n```\n\nlaunch\nhttps://example.com\n\n```\n\nClicking at specific coordinates (e.g., a button):\n```\n\nclick\n450,300\n\n```\n\nTyping text into a focused input field:\n```\n\ntype\nHello, World!\n\n```\n\nScrolling down to see more content:\n```\n\nscroll_down\n\n```\n\nClosing the browser session:\n```\n\nclose\n\n```\n" }, "docs/saarthi/advanced-usage/available-tools/codebase-search.md": { "content": "# codebase_search\n\n:::warning Experimental Feature\nThe `codebase_search` tool is part of the experimental [Codebase Indexing](/saarthi/features/experimental/codebase-indexing) feature. This feature is under active development and may change significantly in future releases. It requires additional setup including an embedding provider and vector database.\n:::\n\nThe `codebase_search` tool performs semantic searches across your entire codebase using AI embeddings. Unlike traditional text-based search, it understands the meaning of your queries and finds relevant code even when exact keywords don't match.\n\n## Parameters\n\nThe tool accepts these parameters:\n\n- `query` (required): Natural language search query describing what you're looking for\n- `path` (optional): Directory path to limit search scope to a specific part of your codebase\n\n## What It Does\n\nThis tool searches through your indexed codebase using semantic similarity rather than exact text matching. It finds code blocks that are conceptually related to your query, even if they don't contain the exact words you searched for. Results include relevant code snippets with file paths, line numbers, and similarity scores.\n\n## When is it used?\n\n- When Saarthi needs to find code related to specific functionality across your project\n- When looking for implementation patterns or similar code structures\n- When searching for error handling, authentication, or other conceptual code patterns\n- When exploring unfamiliar codebases to understand how features are implemented\n- When finding related code that might be affected by changes or refactoring\n\n## Key Features\n\n- **Semantic Understanding**: Finds code by meaning rather than exact keyword matches\n- **Cross-Project Search**: Searches across your entire indexed codebase, not just open files\n- **Contextual Results**: Returns code snippets with file paths and line numbers for easy navigation\n- **Similarity Scoring**: Results ranked by relevance with similarity scores (0-1 scale)\n- **Scope Filtering**: Optional path parameter to limit searches to specific directories\n- **Intelligent Ranking**: Results sorted by semantic relevance to your query\n- **UI Integration**: Results displayed with syntax highlighting and navigation links\n- **Performance Optimized**: Fast vector-based search with configurable result limits\n\n## Requirements\n\nThis tool is only available when the experimental Codebase Indexing feature is properly configured:\n\n- **Feature Enabled**: Codebase Indexing must be enabled in experimental settings\n- **Embedding Provider**: OpenAI API key or Ollama configuration required\n- **Vector Database**: Qdrant instance running and accessible\n- **Index Status**: Codebase must be indexed (status: \"Indexed\" or \"Indexing\")\n\n## Limitations\n\n- **Experimental Feature**: Part of the experimental codebase indexing system\n- **Requires Configuration**: Depends on external services (embedding provider + Qdrant)\n- **Index Dependency**: Only searches through indexed code blocks\n- **Result Limits**: Maximum of 50 results per search to maintain performance\n- **Similarity Threshold**: Only returns results above 0.4 similarity score\n- **File Size Limits**: Limited to files under 1MB that were successfully indexed\n- **Language Support**: Effectiveness depends on Tree-sitter language support\n\n## How It Works\n\nWhen the `codebase_search` tool is invoked, it follows this process:\n\n1. **Availability Validation**:\n - Verifies that the CodeIndexManager is available and initialized\n - Confirms codebase indexing is enabled in settings\n - Checks that indexing is properly configured (API keys, Qdrant URL)\n - Validates the current index state allows searching\n\n2. **Query Processing**:\n - Takes your natural language query and generates an embedding vector\n - Uses the same embedding provider configured for indexing (OpenAI or Ollama)\n - Converts the semantic meaning of your query into a mathematical representation\n\n3. **Vector Search Execution**:\n - Searches the Qdrant vector database for similar code embeddings\n - Uses cosine similarity to find the most relevant code blocks\n - Applies the minimum similarity threshold (0.4) to filter results\n - Limits results to 50 matches for optimal performance\n\n4. **Path Filtering** (if specified):\n - Filters results to only include files within the specified directory path\n - Uses normalized path comparison for accurate filtering\n - Maintains relevance ranking within the filtered scope\n\n5. **Result Processing and Formatting**:\n - Converts absolute file paths to workspace-relative paths\n - Structures results with file paths, line ranges, similarity scores, and code content\n - Formats for both AI consumption and UI display with syntax highlighting\n\n6. **Dual Output Format**:\n - **AI Output**: Structured text format with query, file paths, scores, and code chunks\n - **UI Output**: JSON format with syntax highlighting and navigation capabilities\n\n## Search Query Best Practices\n\n### Effective Query Patterns\n\n**Good: Conceptual and specific**\n```xml\n\nuser authentication and password validation\n\n```\n\n**Good: Feature-focused**\n```xml\n\ndatabase connection pool setup\n\n```\n\n**Good: Problem-oriented**\n```xml\n\nerror handling for API requests\n\n```\n\n**Less effective: Too generic**\n```xml\n\nfunction\n\n```\n\n### Query Types That Work Well\n\n- **Functional Descriptions**: \"file upload processing\", \"email validation logic\"\n- **Technical Patterns**: \"singleton pattern implementation\", \"factory method usage\"\n- **Domain Concepts**: \"user profile management\", \"payment processing workflow\"\n- **Architecture Components**: \"middleware configuration\", \"database migration scripts\"\n\n## Directory Scoping\n\nUse the optional `path` parameter to focus searches on specific parts of your codebase:\n\n**Search within API modules:**\n```xml\n\nendpoint validation middleware\nsrc/api\n\n```\n\n**Search in test files:**\n```xml\n\nmock data setup patterns\ntests\n\n```\n\n**Search specific feature directories:**\n```xml\n\ncomponent state management\nsrc/components/auth\n\n```\n\n## Result Interpretation\n\n### Similarity Scores\n\n- **0.8-1.0**: Highly relevant matches, likely exactly what you're looking for\n- **0.6-0.8**: Good matches with strong conceptual similarity\n- **0.4-0.6**: Potentially relevant but may require review\n- **Below 0.4**: Filtered out as too dissimilar\n\n### Result Structure\n\nEach search result includes:\n- **File Path**: Workspace-relative path to the file containing the match\n- **Score**: Similarity score indicating relevance (0.4-1.0)\n- **Line Range**: Start and end line numbers for the code block\n- **Code Chunk**: The actual code content that matched your query\n\n## Examples When Used\n\n- When implementing a new feature, Saarthi searches for \"authentication middleware\" to understand existing patterns before writing new code.\n- When debugging an issue, Saarthi searches for \"error handling in API calls\" to find related error patterns across the codebase.\n- When refactoring code, Saarthi searches for \"database transaction patterns\" to ensure consistency across all database operations.\n- When onboarding to a new codebase, Saarthi searches for \"configuration loading\" to understand how the application bootstraps.\n\n## Usage Examples\n\nSearching for authentication-related code across the entire project:\n```xml\n\nuser login and authentication logic\n\n```\n\nFinding database-related code in a specific directory:\n```xml\n\ndatabase connection and query execution\nsrc/data\n\n```\n\nLooking for error handling patterns in API code:\n```xml\n\nHTTP error responses and exception handling\nsrc/api\n\n```\n\nSearching for testing utilities and mock setups:\n```xml\n\ntest setup and mock data creation\ntests\n\n```\n\nFinding configuration and environment setup code:\n```xml\n\nenvironment variables and application configuration\n" }, "docs/saarthi/advanced-usage/available-tools/execute-command.md": { "content": "# execute_command\n\nThe `execute_command` tool runs CLI commands on the user's system. It allows Saarthi to perform system operations, install dependencies, build projects, start servers, and execute other terminal-based tasks needed to accomplish user objectives.\n\n## Parameters\n\nThe tool accepts these parameters:\n\n- `command` (required): The CLI command to execute. Must be valid for the user's operating system.\n- `cwd` (optional): The working directory to execute the command in. If not provided, the current working directory is used.\n\n## What It Does\n\nThis tool executes terminal commands directly on the user's system, enabling a wide range of operations from file manipulations to running development servers. Commands run in managed terminal instances with real-time output capture, integrated with VS Code's terminal system for optimal performance and security.\n\n## When is it used?\n\n- When installing project dependencies (npm install, pip install, etc.)\n- When building or compiling code (make, npm run build, etc.)\n- When starting development servers or running applications\n- When initializing new projects (git init, npm init, etc.)\n- When performing file operations beyond what other tools provide\n- When running tests or linting operations\n- When needing to execute specialized commands for specific technologies\n\n## Key Features\n\n- Integrates with VS Code shell API for reliable terminal execution\n- Reuses terminal instances when possible through a registry system\n- Captures command output line by line with real-time feedback\n- Supports long-running commands that continue in the background\n- Allows specification of custom working directories\n- Maintains terminal history and state across command executions\n- Handles complex command chains appropriate for the user's shell\n- Provides detailed command completion status and exit code interpretation\n- Supports interactive terminal applications with user feedback loop\n- Shows terminals during execution for transparency\n- Validates commands for security using shell-quote parsing\n- Blocks potentially dangerous subshell execution patterns\n- Integrates with RooIgnore system for file access control\n- Handles terminal escape sequences for clean output\n\n## Limitations\n\n- Command access may be restricted by RooIgnore rules and security validations\n- Commands with elevated permission requirements may need user configuration\n- Behavior may vary across operating systems for certain commands\n- Very long-running commands may require specific handling\n- File paths should be properly escaped according to the OS shell rules\n- Not all terminal features may work with remote development scenarios\n\n## How It Works\n\nWhen the `execute_command` tool is invoked, it follows this process:\n\n1. **Command Validation and Security Checks**:\n - Parses the command using shell-quote to identify components\n - Validates against security restrictions (subshell usage, restricted files)\n - Checks against RooIgnore rules for file access permissions\n - Ensures the command meets system security requirements\n\n2. **Terminal Management**:\n - Gets or creates a terminal through TerminalRegistry\n - Sets up the working directory context\n - Prepares event listeners for output capture\n - Shows the terminal for user visibility\n\n3. **Command Execution and Monitoring**:\n - Executes via VS Code's shellIntegration API\n - Captures output with escape sequence processing\n - Throttles output handling (100ms intervals)\n - Monitors for command completion or errors\n - Detects \"hot\" processes like compilers for special handling\n\n4. **Result Processing**:\n - Strips ANSI/VS Code escape sequences for clean output\n - Interprets exit codes with detailed signal information\n - Updates working directory tracking if changed by command\n - Provides command status with appropriate context\n\n## Terminal Implementation Details\n\nThe tool uses a sophisticated terminal management system:\n\n1. **First Priority: Terminal Reuse**\n - The TerminalRegistry tries to reuse existing terminals when possible\n - This reduces proliferation of terminal instances and improves performance\n - Terminal state (working directory, history) is preserved across commands\n\n2. **Second Priority: Security Validation**\n - Commands are parsed using shell-quote for component analysis\n - Dangerous patterns like `$(...)` and backticks are blocked\n - Commands are checked against RooIgnore rules for file access control\n - A prefix-based allowlist system validates command patterns\n\n3. **Performance Optimizations**\n - Output is processed in 100ms throttled intervals to prevent UI overload\n - Zero-copy buffer management uses index-based tracking for efficiency\n - Special handling for compilation and \"hot\" processes\n - Platform-specific optimizations for Windows PowerShell\n\n4. **Error and Signal Handling**\n - Exit codes are mapped to detailed signal information (SIGTERM, SIGKILL, etc.)\n - Core dump detection for critical failures\n - Working directory changes are tracked and handled automatically\n - Clean recovery from terminal disconnection scenarios\n\n## Examples When Used\n\n- When setting up a new project, Saarthi runs initialization commands like `npm init -y` followed by installing dependencies.\n- When building a web application, Saarthi executes build commands like `npm run build` to compile assets.\n- When deploying code, Saarthi runs git commands to commit and push changes to a repository.\n- When troubleshooting, Saarthi executes diagnostic commands to gather system information.\n- When starting a development server, Saarthi launches the appropriate server command (e.g., `npm start`).\n- When running tests, Saarthi executes the test runner command for the project's testing framework.\n\n## Usage Examples\n\nRunning a simple command in the current directory:\n```\n\nnpm run dev\n\n```\n\nInstalling dependencies for a project:\n```\n\nnpm install express mongodb mongoose dotenv\n\n```\n\nRunning multiple commands in sequence:\n```\n\nmkdir -p src/components && touch src/components/App.js\n\n```\n\nExecuting a command in a specific directory:\n```\n\ngit status\n./my-project\n\n```\n\nBuilding and then starting a project:\n```\n\nnpm run build && npm start\n\n```\n" }, "docs/saarthi/advanced-usage/available-tools/insert-content.md": { "content": "# insert_content\n\nThe `insert_content` tool adds new lines of content into an existing file without modifying the original content. It's ideal for inserting code blocks, configuration entries, or log lines at specific locations.\n\n## Parameters\n\nThe tool accepts these parameters:\n\n- `path` (required): The relative path (from the workspace root) of the file to insert content into.\n- `line` (required): The 1-based line number *before* which the content should be inserted. Use `0` to append the content to the end of the file.\n- `content` (required): The text content to insert.\n\n## What It Does\n\nThis tool reads the target file, identifies the specified insertion point based on the `line` parameter, and inserts the provided `content` at that location. If `line` is `0`, the content is added to the end. Changes are presented in a diff view for user approval before being saved.\n\n## When is it used?\n\n- When adding new import statements at the beginning of a file.\n- When inserting new functions or methods into existing code.\n- When adding configuration blocks to settings files.\n- When appending log entries or data records.\n- When adding any multi-line text block without altering existing lines.\n\n## Key Features\n\n- **Targeted Insertion**: Adds content precisely at the specified line number or appends to the end.\n- **Preserves Existing Content**: Does not modify or delete original file lines.\n- **Interactive Approval**: Shows proposed insertions in a diff view, requiring explicit user approval.\n- **User Edit Support**: Allows editing the proposed content directly within the diff view before final approval.\n- **Handles Line Numbers**: Correctly interprets the `line` parameter (1-based or 0 for append).\n- **Context Tracking**: Records the file edit operation for context management.\n- **Error Handling**: Checks for missing parameters, invalid line numbers, and file access issues.\n\n## Limitations\n\n- **Insert Only**: Cannot replace or delete existing content. Use `apply_diff` or `search_and_replace` for modifications.\n- **Requires Existing File**: The target file specified by `path` must exist.\n- **Review Overhead**: The mandatory diff view approval adds an interactive step.\n\n## How It Works\n\nWhen the `insert_content` tool is invoked, it follows this process:\n\n1. **Parameter Validation**: Checks for required `path`, `line`, and `content`. Validates `line` is a non-negative integer.\n2. **File Reading**: Reads the content of the target file specified by `path`.\n3. **Insertion Point Calculation**: Converts the 1-based `line` parameter to a 0-based index for internal processing (`-1` for appending).\n4. **Content Insertion**: Uses an internal utility (`insertGroups`) to merge the original file lines with the new `content` at the calculated index.\n5. **Diff View Interaction**:\n * Opens the file in the diff view (`cline.diffViewProvider.open`).\n * Updates the diff view with the proposed content (`cline.diffViewProvider.update`).\n6. **User Approval**: Presents the change via `askApproval`. Reverts if rejected.\n7. **Saving Changes**: If approved, saves the changes using `cline.diffViewProvider.saveChanges`.\n8. **File Context Tracking**: Tracks the edit using `cline.getFileContextTracker().trackFileContext`.\n9. **Handling User Edits**: If the user edited the content in the diff view, reports the final merged content back.\n10. **Result Reporting**: Reports success (including user edits) or failure back to the AI model.\n\n## Usage Examples\n\nInserting import statements at the beginning of a file (`line: 1`):\n\n```xml\n\nsrc/utils.ts\n1\n\n// Add imports at start of file\nimport { sum } from './math';\nimport { parse } from 'date-fns';\n\n\n```\n\nAppending content to the end of a file (`line: 0`):\n\n```xml\n\nconfig/routes.yaml\n0\n\n- path: /new-feature\n component: NewFeatureComponent\n auth_required: true\n\n\n```\n\nInserting a function before line 50:\n\n```xml\n\nsrc/services/api.js\n50\n\nasync function fetchUserData(userId) {\n const response = await fetch(`/api/users/${userId}`);\n if (!response.ok) {\n throw new Error('Failed to fetch user data');\n }\n return response.json();\n}\n\n\n```" }, "docs/saarthi/advanced-usage/available-tools/list-code-definition-names.md": { "content": "# list_code_definition_names\n\nThe `list_code_definition_names` tool provides a structural overview of your codebase by listing code definitions from source files at the top level of a specified directory. It helps Saarthi understand code architecture by displaying line numbers and definition snippets.\n\n## Parameters\n\nThe tool accepts these parameters:\n\n- `path` (required): The path of the directory to list top level source code definitions for, relative to the current working directory\n\n## What It Does\n\nThis tool scans source code files at the top level of a specified directory and extracts code definitions like classes, functions, and interfaces. It displays the line numbers and actual code for each definition, providing a quick way to map the important components of your codebase.\n\n## When is it used?\n\n- When Saarthi needs to understand your codebase architecture quickly\n- When Saarthi needs to locate important code constructs across multiple files\n- When planning refactoring or extensions to existing code\n- Before diving into implementation details with other tools\n- When identifying relationships between different parts of your codebase\n\n## Key Features\n\n- Extracts classes, functions, methods, interfaces, and other definitions from source files\n- Displays line numbers and actual source code for each definition\n- Supports multiple programming languages including JavaScript, TypeScript, Python, Rust, Go, C++, C, C#, Ruby, Java, PHP, Swift, and Kotlin\n- Processes only files at the top level of the specified directory (not subdirectories)\n- Limits processing to a maximum of 50 files for performance\n- Focuses on top-level definitions to avoid overwhelming detail\n- Helps identify code organization patterns across the project\n- Creates a mental map of your codebase's architecture\n- Works in conjunction with other tools like `read_file` for deeper analysis\n\n## Limitations\n\n- Only identifies top-level definitions, not nested ones\n- Only processes files at the top level of the specified directory, not subdirectories\n- Limited to processing a maximum of 50 files per request\n- Dependent on language-specific parsers, with varying detection quality\n- May not recognize all definitions in languages with complex syntax\n- Not a substitute for reading code to understand implementation details\n- Cannot detect runtime patterns or dynamic code relationships\n- Does not provide information about how definitions are used\n- May have reduced accuracy with highly dynamic or metaprogrammed code\n- Limited to specific languages supported by the implemented Tree-sitter parsers\n\n## How It Works\n\nWhen the `list_code_definition_names` tool is invoked, it follows this process:\n\n1. **Parameter Validation**: Validates the required `path` parameter\n2. **Path Resolution**: Resolves the relative path to an absolute path\n3. **Directory Scanning**: Scans only the top level of the specified directory for source code files (not recursive)\n4. **File Filtering**: Limits processing to a maximum of 50 files\n5. **Language Detection**: Identifies file types based on extensions (.js, .jsx, .ts, .tsx, .py, .rs, .go, .cpp, .hpp, .c, .h, .cs, .rb, .java, .php, .swift, .kt, .kts)\n6. **Code Parsing**: Uses Tree-sitter to parse code and extract definitions through these steps:\n - Parsing file content into an Abstract Syntax Tree (AST)\n - Creating a query using a language-specific query string\n - Sorting the captures by their position in the file\n7. **Result Formatting**: Outputs definitions with line numbers and actual source code\n\n## Output Format\n\nThe output shows file paths followed by line numbers and the actual source code of each definition. For example:\n\n```\nsrc/utils.js:\n0--0 | export class HttpClient {\n5--5 | formatDate() {\n10--10 | function parseConfig(data) {\n\nsrc/models/User.js:\n0--0 | interface UserProfile {\n10--10 | export class User {\n20--20 | function createUser(data) {\n```\n\nEach line displays:\n- The start and end line numbers of the definition\n- The pipe symbol (|) as a separator\n- The actual source code of the definition\n\nThis output format helps you quickly see both where definitions are located in the file and their implementation details.\n\n## Examples When Used\n\n- When starting a new task, Saarthi first lists key code definitions to understand the overall structure of your project.\n- When planning refactoring work, Saarthi uses this tool to identify classes and functions that might be affected.\n- When exploring unfamiliar codebases, Saarthi maps the important code constructs before diving into implementation details.\n- When adding new features, Saarthi identifies existing patterns and relevant code definitions to maintain consistency.\n- When troubleshooting bugs, Saarthi maps the codebase structure to locate potential sources of the issue.\n- When planning architecture changes, Saarthi identifies all affected components across files.\n\n## Usage Examples\n\nListing code definitions in the current directory:\n```\n\n.\n\n```\n\nExamining a specific module's structure:\n```\n\nsrc/components\n\n```\n\nExploring a utility library:\n```\n\nlib/utils\n\n```\n" }, "docs/saarthi/advanced-usage/available-tools/list-files.md": { "content": "# list_files\n\nThe `list_files` tool displays the files and directories within a specified location. It helps Saarthi understand your project structure and navigate your codebase effectively.\n\n## Parameters\n\nThe tool accepts these parameters:\n\n- `path` (required): The path of the directory to list contents for, relative to the current working directory\n- `recursive` (optional): Whether to list files recursively. Use `true` for recursive listing, `false` or omit for top-level only.\n\n## What It Does\n\nThis tool lists all files and directories in a specified location, providing a clear overview of your project structure. It can either show just the top-level contents or recursively explore subdirectories.\n\n## When is it used?\n\n- When Saarthi needs to understand your project structure\n- When Saarthi explores what files are available before reading specific ones\n- When Saarthi maps a codebase to better understand its organization\n- Before using more targeted tools like `read_file` or `search_files`\n- When Saarthi needs to check for specific file types (like configuration files) across a project\n\n## Key Features\n\n- Lists both files and directories with directories clearly marked\n- Offers both recursive and non-recursive listing modes\n- Intelligently ignores common large directories like `node_modules` and `.git` in recursive mode\n- Respects `.gitignore` rules when in recursive mode\n- Marks files ignored by `.saarthiignore` with a lock symbol (🔒) when `showRooIgnoredFiles` is enabled\n- Optimizes file listing performance by leveraging the `ripgrep` tool.\n- Sorts results to show directories before their contents, maintaining a logical hierarchy\n- Presents results in a clean, organized format\n- Automatically creates a mental map of your project structure\n\n## Limitations\n\n- File listing is capped at about 200 files by default to prevent performance issues\n- The underlying `ripgrep` file listing process has a 10-second timeout; if exceeded, partial results may be returned.\n- When the file limit is hit, it adds a note suggesting to use `list_files` on specific subdirectories\n- Not designed for confirming the existence of files you've just created\n- May have reduced performance in very large directory structures\n- Cannot list files in root or home directories for security reasons\n\n## How It Works\n\nWhen the `list_files` tool is invoked, it follows this process:\n\n1. **Parameter Validation**: Validates the required `path` parameter and optional `recursive` parameter\n2. **Path Resolution**: Resolves the relative path to an absolute path\n3. **Security Checks**: Prevents listing files in sensitive locations like root or home directories\n4. **Directory/File Scanning**:\n - Uses the `ripgrep` tool to efficiently list files, applying a 10-second timeout.\n - Uses Node.js `fs` module to list directories.\n - Applies different filtering logic for recursive vs. non-recursive modes.\n5. **Result Filtering**:\n - In recursive mode, skips common large directories like `node_modules`, `.git`, etc.\n - Respects `.gitignore` rules when in recursive mode\n - Handles `.saarthiignore` patterns, either hiding files or marking them with a lock symbol\n6. **Formatting**:\n - Marks directories with a trailing slash (`/`)\n - Sorts results to show directories before their contents for logical hierarchy\n - Marks ignored files with a lock symbol (🔒) when `showRooIgnoredFiles` is enabled\n - Caps results at 200 files by default with a note about using subdirectories\n - Organizes results for readability\n\n## File Listing Format\n\nThe file listing results include:\n\n- Each file path is displayed on its own line\n- Directories are marked with a trailing slash (`/`)\n- Files ignored by `.saarthiignore` are marked with a lock symbol (🔒) when `showRooIgnoredFiles` is enabled\n- Results are sorted logically with directories appearing before their contents\n- When the file limit is reached, a message appears suggesting to use `list_files` on specific subdirectories\n\nExample output format:\n```\nsrc/\nsrc/components/\nsrc/components/Button.tsx\nsrc/components/Header.tsx\nsrc/utils/\nsrc/utils/helpers.ts\nsrc/index.ts\n...\nFile listing truncated (showing 200 of 543 files). Use list_files on specific subdirectories for more details.\n```\n\nWhen `.saarthiignore` files are used and `showRooIgnoredFiles` is enabled:\n```\nsrc/\nsrc/components/\nsrc/components/Button.tsx\nsrc/components/Header.tsx\n🔒 src/secrets.json\nsrc/utils/\nsrc/utils/helpers.ts\nsrc/index.ts\n```\n\n## Examples When Used\n\n- When starting a new task, Saarthi may list the project files to understand its structure before diving into specific code.\n- When asked to find specific types of files (like all JavaScript files), Saarthi first lists directories to know where to look.\n- When providing recommendations for code organization, Saarthi examines the current project structure first.\n- When setting up a new feature, Saarthi lists related directories to understand the project conventions.\n\n## Usage Examples\n\nListing top-level files in the current directory:\n```\n\n.\n\n```\n\nRecursively listing all files in a source directory:\n```\n\nsrc\ntrue\n\n```\n\nExamining a specific project subdirectory:\n```\n\nsrc/components\nfalse\n\n```\n" }, "docs/saarthi/advanced-usage/available-tools/new-task.md": { "content": "# new_task\n\nThe `new_task` tool creates subtasks with specialized modes while maintaining a parent-child relationship. It breaks down complex projects into manageable pieces, each operating in the mode best suited for specific work.\n\n## Parameters\n\nThe tool accepts these parameters:\n\n- `mode` (required): The slug of the mode to start the new task in (e.g., \"code\", \"ask\", \"architect\")\n- `message` (required): The initial user message or instructions for this new task\n\n## What It Does\n\nThis tool creates a new task instance with a specified starting mode and initial message. It allows complex workflows to be divided into subtasks with their own conversation history. Parent tasks are paused during subtask execution and resumed when the subtask completes, with results transferred back to the parent.\n\n## When is it used?\n\n- When breaking down complex projects into separate, focused subtasks\n- When different aspects of a task require different specialized modes\n- When different phases of work benefit from context separation\n- When organizing multi-phase development workflows\n\n## Key Features\n\n- Creates subtasks with their own conversation history and specialized mode\n- Pauses parent tasks for later resumption\n- Maintains hierarchical task relationships for navigation\n- Transfers results back to parent tasks upon completion\n- Supports workflow segregation for complex projects\n- Allows different parts of a project to use modes optimized for specific work\n- Requires explicit user approval for task creation\n- Provides clear task transition in the UI\n\n## Limitations\n\n- Cannot create tasks with modes that don't exist\n- Requires user approval before creating each new task\n- Task interface may become complex with deeply nested subtasks\n- Subtasks inherit certain workspace and extension configurations from parents\n- May require re-establishing context when switching between deeply nested tasks\n- Task completion needs explicit signaling to properly return to parent tasks\n\n## How It Works\n\nWhen the `new_task` tool is invoked, it follows this process:\n\n1. **Parameter Validation**:\n - Validates the required `mode` and `message` parameters\n - Verifies that the requested mode exists in the system\n\n2. **Task Stack Management**:\n - Maintains a task stack that tracks all active and paused tasks\n - Preserves the current mode for later resumption\n - Sets the parent task to paused state\n\n3. **Task Context Management**:\n - Creates a new task context with the provided message\n - Assigns unique taskId and instanceId identifiers for state management\n - Captures telemetry data on tool usage and task lifecycles\n\n4. **Mode Switching and Integration**:\n - Switches to the specified mode with appropriate role and capabilities\n - Initializes the new task with the provided message\n - Integrates with VS Code's command palette and code actions\n\n5. **Task Completion and Result Transfer**:\n - When subtask completes, result is passed back to parent task via `finishSubTask()`\n - Parent task resumes in its original mode\n - Task history and token usage metrics are updated\n - The `taskCompleted` event is emitted with performance data\n\n## Examples When Used\n\n- When a front-end developer needs to architect a new feature, implement the code, and document it, they can create separate tasks for each phase with results flowing from one phase to the next.\n- When debugging an issue before implementing a fix, the debugging task can document findings that are passed to the implementation task.\n- When developing a full-stack application, database schema designs from an architect-mode task inform implementation details in a subsequent code-mode task.\n- When documenting a system after implementation, the documentation task can reference the completed implementation while using documentation-specific features.\n\n## Usage Examples\n\nCreating a new task in code mode:\n```\n\ncode\nImplement a user authentication service with login, registration, and password reset functionality.\n\n```\n\nCreating a documentation task after completing implementation:\n```\n\ndocs\nCreate comprehensive API documentation for the authentication service we just built.\n\n```\n\nBreaking down a complex feature into architectural planning and implementation:\n```\n\narchitect\nDesign the database schema and system architecture for our new e-commerce platform.\n\n```\n" }, "docs/saarthi/advanced-usage/available-tools/read-file.md": { "content": "\n# read_file\n\nThe `read_file` tool examines the contents of files in a project. It allows Saarthi to understand code, configuration files, and documentation to provide better assistance.\n\n:::info Multi-File Support\nWhen the [Concurrent File Reads](../../features/experimental/concurrent-file-reads) experimental feature is enabled, this tool can read multiple files simultaneously using an enhanced XML parameter format. This significantly improves efficiency for tasks requiring analysis of multiple related files.\n:::\n\n## Parameters\n\nThe tool accepts parameters in two formats depending on your configuration:\n\n### Standard Format (Single File)\n\n- `path` (required): The path of the file to read relative to the current working directory\n- `start_line` (optional): The starting line number to read from (1-based indexing)\n- `end_line` (optional): The ending line number to read to (1-based, inclusive)\n\n### Enhanced Format (Multi-File - Experimental)\n\nWhen [Concurrent File Reads](../../features/experimental/concurrent-file-reads) is enabled, the tool accepts an `args` parameter containing multiple file entries:\n\n- `args` (required): Container for multiple file specifications\n - `file` (required): Individual file specification\n - `path` (required): The path of the file to read\n - `lines` (optional): Line range specification (e.g., \"1-50\" or \"100-150\")\n\n## What It Does\n\nThis tool reads the content of a specified file and returns it with line numbers for easy reference. It can read entire files or specific sections, and even extract text from PDFs and Word documents.\n\n## When is it used?\n\n- When Saarthi needs to understand existing code structure\n- When Saarthi needs to analyze configuration files\n- When Saarthi needs to extract information from text files\n- When Saarthi needs to see code before suggesting changes\n- When specific line numbers need to be referenced in discussions\n\n## Key Features\n\n- Displays file content with line numbers for easy reference\n- Can read specific portions of files by specifying line ranges\n- Extracts readable text from PDF and DOCX files\n- Automatically truncates large text files when no line range is specified, showing the beginning of the file\n- Provides method summaries with line ranges for truncated large code files\n- Efficiently streams only requested line ranges for better performance\n- Makes it easy to discuss specific parts of code with line numbering\n- **Multi-file support** (experimental): Read multiple files simultaneously with batch approval\n\n## Multi-File Capabilities (Experimental)\n\nWhen the [Concurrent File Reads](../../features/experimental/concurrent-file-reads) experimental feature is enabled, the `read_file` tool gains enhanced capabilities:\n\n### Batch Processing\n- Read up to 100 files in a single request (configurable, default 15)\n- Parallel processing for improved performance\n- Batch approval interface for user consent\n\n### Enhanced User Experience\n- Single approval dialog for multiple files\n- Individual file override options\n- Clear visibility into which files will be accessed\n- Graceful handling of mixed success/failure scenarios\n\n### Improved Efficiency\n- Reduces interruptions from multiple approval dialogs\n- Faster processing through parallel file reading\n- Smart batching of related files\n- Configurable concurrency limits to match system capabilities\n\n## Limitations\n\n- May not handle extremely large files efficiently without using line range parameters\n- For binary files (except PDF and DOCX), may return content that isn't human-readable\n- **Multi-file mode**: Requires experimental feature to be enabled and may have stability issues\n\n## How It Works\n\nWhen the `read_file` tool is invoked, it follows this process:\n\n1. **Parameter Validation**: Validates the required `path` parameter and optional parameters\n2. **Path Resolution**: Resolves the relative path to an absolute path\n3. **Reading Strategy Selection**:\n - The tool uses a strict priority hierarchy (explained in detail below)\n - It chooses between range reading, auto-truncation, or full file reading\n4. **Content Processing**:\n - Adds line numbers to the content (e.g., \"1 | const x = 13\") where `1 |` is the line number.\n - For truncated files, adds truncation notice and method definitions\n - For special formats (PDF, DOCX), extracts readable text\n\n## Reading Strategy Priority\n\nThe tool uses a clear decision hierarchy to determine how to read a file:\n\n1. **First Priority: Explicit Line Range**\n - If either `start_line` or `end_line` is provided, the tool always performs a range read\n - The implementation efficiently streams only the requested lines, making it suitable for processing large files\n - This takes precedence over all other options\n\n2. **Second Priority: Automatic Truncation for Large Text Files**\n - This applies only when **all** of the following conditions are met:\n - Neither `start_line` nor `end_line` is specified.\n - The file is identified as a text-based file (not binary like PDF/DOCX).\n - The file's total line count exceeds an internal limit (e.g., `maxReadFileLine`, often around 500 lines).\n - When automatic truncation occurs:\n - The tool reads only the *first* `maxReadFileLine` lines.\n - It appends a notice indicating truncation (e.g., `[Showing only 500 of 1200 total lines...]`).\n - For code files, it may also append a summary of source code definitions found within the truncated portion.\n\n3. **Default Behavior: Read Entire File**\n - If neither an explicit range is given nor automatic truncation applies (e.g., the file is within the line limit, or it's a supported binary type), the tool reads the entire content.\n - For supported formats like PDF and DOCX, it attempts to extract the full text content.\n\n## Examples When Used\n\n- When asked to explain or improve code, Saarthi first reads the relevant files to understand the current implementation.\n- When troubleshooting configuration issues, Saarthi reads config files to identify potential problems.\n- When working with documentation, Saarthi reads existing docs to understand the current content before suggesting improvements.\n\n## Usage Examples\n\nHere are several scenarios demonstrating how the `read_file` tool is used and the typical output you might receive.\n\n### Reading an Entire File\n\nTo read the complete content of a file:\n\n**Input:**\n```xml\n\nsrc/app.js\n\n```\n\n**Simulated Output (for a small file like `example_small.txt`):**\n```\n1 | This is the first line.\n2 | This is the second line.\n3 | This is the third line.\n```\n*(Output will vary based on the actual file content)*\n\n### Reading Specific Lines\n\nTo read only a specific range of lines (e.g., 46-68):\n\n**Input:**\n```xml\n\nsrc/app.js\n46\n68\n\n```\n\n**Simulated Output (for lines 2-3 of `example_five_lines.txt`):**\n```\n2 | Content of line two.\n3 | Content of line three.\n```\n*(Output shows only the requested lines with their original line numbers)*\n\n### Reading a Large Text File (Automatic Truncation)\n\nWhen reading a large text file without specifying a line range, the tool automatically truncates the content if it exceeds the internal line limit (e.g., 500 lines).\n\n**Input:**\n```xml\n\nlogs/large_app.log\n\n```\n\n**Simulated Output (for a 1500-line log file with a 500-line limit):**\n```\n1 | Log entry 1...\n2 | Log entry 2...\n...\n500 | Log entry 500...\n\n[Showing only 500 of 1500 total lines. Use start_line and end_line to read specific ranges.]\n// Optional: Source code definitions summary might appear here for code files\n```\n*(Output shows the beginning lines up to the internal limit, plus a truncation notice. Use line ranges for full access.)*\n\n### Attempting to Read a Non-Existent File\n\nIf the specified file does not exist:\n\n**Input:**\n```xml\n\nnon_existent_file.txt\n\n```\n\n**Simulated Output (Error):**\n```\nError: File not found at path 'non_existent_file.txt'.\n```\n\n### Attempting to Read a Blocked File\n\nIf the file is excluded by rules in a `.saarthiignore` file:\n\n**Input:**\n```xml\n\n.env\n\n```\n\n**Simulated Output (Error):**\n```\nError: Access denied to file '.env' due to .saarthiignore rules.\n```\n\n## Multi-File Examples (Experimental)\n\nWhen the [Concurrent File Reads](../../features/experimental/concurrent-file-reads) experimental feature is enabled, you can read multiple files simultaneously using the enhanced XML format.\n\n### Reading Multiple Complete Files\n\nTo read several complete files at once:\n\n**Input:**\n```xml\n\n\n \n src/app.ts\n \n \n src/utils.ts\n \n \n src/config.json\n \n\n\n```\n\n**Simulated Output:**\n```xml\n\n \n src/app.ts\n \n 1 | import React from 'react'\n 2 | import { Utils } from './utils'\n 3 | // ... rest of file content\n \n \n \n src/utils.ts\n \n 1 | export class Utils {\n 2 | static formatDate(date: Date): string {\n 3 | // ... utility functions\n \n \n \n src/config.json\n \n 1 | {\n 2 | \"apiUrl\": \"https://api.example.com\",\n 3 | \"timeout\": 5000\n 4 | }\n \n \n\n```\n\n### Reading Specific Line Ranges from Multiple Files\n\nTo read specific sections from multiple files:\n\n**Input:**\n```xml\n\n\n \n src/app.ts\n 1-20\n 45-60\n \n \n src/utils.ts\n 10-25\n \n\n\n```\n\n**Simulated Output:**\n```xml\n\n \n src/app.ts\n \n 1 | import React from 'react'\n 2 | import { Utils } from './utils'\n ...\n 20 | const App = () => {\n \n 45 | const handleSubmit = () => {\n 46 | // Handle form submission\n ...\n 60 | }\n \n \n \n src/utils.ts\n \n 10 | static formatDate(date: Date): string {\n 11 | return date.toISOString().split('T')[0]\n ...\n 25 | }\n \n \n\n```\n\n### Handling Mixed Results (Some Files Denied/Blocked)\n\nWhen some files are approved and others are denied or blocked:\n\n**Input:**\n```xml\n\n\n \n src/app.ts\n \n \n .env\n \n \n src/secret-config.ts\n \n\n\n```\n\n**Simulated Output:**\n```xml\n\n \n src/app.ts\n \n 1 | import React from 'react'\n 2 | // ... file content successfully read\n \n \n \n .env\n Access denied by .saarthiignore rules\n \n \n src/secret-config.ts\n User denied access\n \n\n```\n\n### Batch Approval Interface\n\nWhen requesting multiple files, you'll see a batch approval interface that allows you to:\n\n- **Approve All**: Grant access to all requested files\n- **Deny All**: Deny access to all requested files\n- **Individual Control**: Override decisions for specific files\n- **File Preview**: Click file headers to open them in your editor\n\nThe interface displays each file path clearly, making it easy to understand what Saarthi wants to access before granting permission.\n\n## Backward Compatibility\n\nThe enhanced multi-file format is fully backward compatible. Existing single-file requests using the `path` parameter continue to work exactly as before, regardless of whether the experimental feature is enabled or disabled.\n" }, "docs/saarthi/advanced-usage/available-tools/search-and-replace.md": { "content": "# search_and_replace\n\nThe `search_and_replace` tool finds and replaces text within a file, supporting both literal strings and regular expression patterns. It allows for targeted replacements across multiple locations, optionally within specific line ranges.\n\n## Parameters\n\n### Required Parameters\n\n- `path`: The relative path (from the workspace saarthit) of the file to modify.\n- `search`: The text string or regex pattern to find.\n- `replace`: The text to replace matches with.\n\n### Optional Parameters\n\n- `start_line`: The 1-based line number where the search scope begins.\n- `end_line`: The 1-based line number where the search scope ends (inclusive).\n- `use_regex`: Set to `\"true\"` to treat the `search` parameter as a regular expression pattern (default is `false`).\n- `ignore_case`: Set to `\"true\"` to perform a case-insensitive search (default is `false`).\n\n## What It Does\n\nThis tool reads the specified file and performs a search-and-replace operation based on the provided parameters. It can operate on the entire file or be restricted to a specific range of lines. Changes are presented in a diff view for user review and approval before being saved.\n\n## When is it used?\n\n- When renaming variables, functions, or classes across a file.\n- When updating specific text strings or values consistently.\n- When applying patterned changes using regular expressions.\n- When refactoring code requires replacing specific patterns.\n- When making targeted changes within a defined section of a file.\n\n## Key Features\n\n- **Flexible Searching**: Supports both literal text and regular expression patterns.\n- **Case Sensitivity Control**: Option to ignore case during search.\n- **Scoped Replacements**: Can limit replacements to a specific range of lines (`start_line`, `end_line`).\n- **Global Replacement**: Performs replacements across the entire file (or specified range) by default.\n- **Interactive Approval**: Shows proposed changes in a diff view for user review and approval.\n- **User Edit Support**: Allows editing the proposed content directly within the diff view.\n- **Context Tracking**: Records the file edit operation.\n- **Error Handling**: Checks for missing parameters, file access issues, and invalid line numbers.\n\n## Limitations\n\n- **Single File Operation**: Operates on only one file at a time. Use `search_files` to find patterns across multiple files first.\n- **Review Overhead**: The mandatory diff view approval adds an interactive step.\n- **Regex Complexity**: Complex regex patterns might require careful construction and testing.\n\n## How It Works\n\nWhen the `search_and_replace` tool is invoked, it follows this process:\n\n1. **Parameter Validation**: Checks for required `path`, `search`, `replace`, and validates optional parameters like line numbers and boolean flags.\n2. **File Reading**: Reads the content of the target file specified by `path`.\n3. **Regex Construction**:\n * If `use_regex` is `false`, the `search` string is escaped to treat it as literal text.\n * A `RegExp` object is created with the `g` (global) flag and optionally the `i` (ignore case) flag.\n4. **Replacement Execution**:\n * If `start_line` or `end_line` are provided, the file content is split into lines, the relevant section is isolated, the replacement is performed on that section, and the file content is reconstructed.\n * If no line range is specified, the replacement is performed on the entire file content string.\n5. **Diff View Interaction**:\n * Opens the file in the diff view showing original vs. proposed content.\n * Updates the diff view with the result of the replacement.\n6. **User Approval**: Presents the change via `askApproval`. Reverts if rejected.\n7. **Saving Changes**: If approved, saves the changes (including any user edits made in the diff view).\n8. **File Context Tracking**: Tracks the edit using `cline.getFileContextTracker().trackFileContext`.\n9. **Result Reporting**: Reports success (including user edits) or failure back to the AI model.\n\n## Usage Examples\n\nSimple text replacement throughout a file:\n\n```xml\n\nsrc/config.js\nAPI_KEY_OLD\nAPI_KEY_NEW\n\n```\n\nCase-insensitive regex replacement to update function calls:\n\n```xml\n\nsrc/app.ts\nprocessData\\((.*?)\\)\nhandleData($1)\ntrue\ntrue\n\n```\n\nReplacing text only within lines 10 to 20:\n\n```xml\n\nREADME.md\nDraft\nFinal\n10\n20\n\n```" }, "docs/saarthi/advanced-usage/available-tools/search-files.md": { "content": "\n# search_files\n\nThe `search_files` tool performs regex searches across multiple files in your project. It helps Saarthi locate specific code patterns, text, or other content throughout your codebase with contextual results.\n\n## Parameters\n\nThe tool accepts these parameters:\n\n- `path` (required): The path of the directory to search in, relative to the current working directory\n- `regex` (required): The regular expression pattern to search for (uses Rust regex syntax)\n- `file_pattern` (optional): Glob pattern to filter files (e.g., '*.ts' for TypeScript files)\n\n## What It Does\n\nThis tool searches across files in a specified directory using regular expressions, showing each match with surrounding context. It's like having a powerful \"Find in Files\" feature that works across the entire project structure.\n\n## When is it used?\n\n- When Saarthi needs to find where specific functions or variables are used\n- When Saarthi helps with refactoring and needs to understand usage patterns\n- When Saarthi needs to locate all instances of a particular code pattern\n- When Saarthi searches for text across multiple files with filtering capabilities\n\n## Key Features\n\n- Searches across multiple files in a single operation using high-performance Ripgrep\n- Shows context around each match (1 line before and after)\n- Filters files by type using glob patterns (e.g., only TypeScript files)\n- Provides line numbers for easy reference\n- Uses powerful regex patterns for precise searches\n- Automatically limits output to 300 results with notification\n- Truncates lines longer than 500 characters with \"[truncated...]\" marker\n- Intelligently combines nearby matches into single blocks for readability\n\n## Limitations\n\n- Works best with text-based files (not effective for binary files like images)\n- Performance may slow with extremely large codebases\n- Uses Rust regex syntax, which may differ slightly from other regex implementations\n- Cannot search within compressed files or archives\n- Default context size is fixed (1 line before and after)\n- May display varying context sizes when matches are close together due to result grouping\n\n## How It Works\n\nWhen the `search_files` tool is invoked, it follows this process:\n\n1. **Parameter Validation**: Validates the required `path` and `regex` parameters\n2. **Path Resolution**: Resolves the relative path to an absolute path\n3. **Search Execution**:\n - Uses Ripgrep (rg) for high-performance text searching\n - Applies file pattern filtering if specified\n - Collects matches with surrounding context\n4. **Result Formatting**:\n - Formats results with file paths, line numbers, and context\n - Displays 1 line of context before and after each match\n - Structures output for easy readability\n - Limits results to a maximum of 300 matches with notification\n - Truncates lines longer than 500 characters\n - Merges nearby matches into contiguous blocks\n\n## Search Results Format\n\nThe search results include:\n\n- Relative file paths for each matching file (prefixed with #)\n- Context lines before and after each match (1 line by default)\n- Line numbers padded to 3 spaces followed by ` | ` and the line content\n- A separator line (----) after each match group\n\nExample output format:\n```\n# rel/path/to/app.ts\n 11 | // Some processing logic here\n 12 | // TODO: Implement error handling\n 13 | return processedData;\n----\n\n# Showing first 300 of 300+ results. Use a more specific search if necessary.\n```\n\nWhen matches occur close to each other, they're merged into a single block rather than shown as separate results:\n\n```\n# rel/path/to/auth.ts\n 13 | // Some code here\n 14 | // TODO: Add proper validation\n 15 | function validateUser(credentials) {\n 16 | // TODO: Implement rate limiting\n 17 | return checkDatabase(credentials);\n----\n```\n\n## Examples When Used\n\n- When asked to refactor a function, Saarthi first searches for all places the function is used to ensure comprehensive changes.\n- When investigating bugs, Saarthi searches for similar patterns to identify related issues across the codebase.\n- When addressing technical debt, Saarthi locates all TODO comments across the project.\n- When analyzing dependencies, Saarthi finds all imports of a particular module.\n\n## Usage Examples\n\nSearching for TODO comments in all JavaScript files:\n```\n\nsrc\nTODO|FIXME\n*.js\n\n```\n\nFinding all usages of a specific function:\n```\n\n.\nfunction\\s+calculateTotal\n*.{js,ts}\n\n```\n\nSearching for a specific import pattern across the entire project:\n```\n\n.\nimport\\s+.*\\s+from\\s+['\"]@components/\n\n```\n" }, "docs/saarthi/advanced-usage/available-tools/switch-mode.md": { "content": "# switch_mode\n\nThe `switch_mode` tool enables Saarthi to change between different operational modes, each with specialized capabilities for specific types of tasks. This allows seamless transitions between modes like Code, Architect, Ask, or Debug when the current task requires different expertise.\n\n## Parameters\n\nThe tool accepts these parameters:\n\n- `mode_slug` (required): The slug of the mode to switch to (e.g., \"code\", \"ask\", \"architect\")\n- `reason` (optional): The reason for switching modes, providing context for the user\n\n## What It Does\n\nThis tool requests a mode change when the current task would be better handled by another mode's capabilities. It maintains context while shifting Saarthi's focus and available toolsets to match the requirements of the new task phase.\n\n## When is it used?\n\n- When transitioning from information gathering to code implementation\n- When shifting from coding to architecture or design\n- When the current task requires capabilities only available in a different mode\n- When specialized expertise is needed for a particular phase of a complex project\n\n## Key Features\n\n- Maintains context continuity across mode transitions\n- Provides clear reasoning for mode switch recommendations\n- Requires user approval for all mode changes\n- Enforces tool group restrictions specific to each mode\n- Seamlessly adapts tool availability based on the selected mode\n- Works with both standard and custom modes\n- Displays the mode switch and reasoning in the UI\n- Uses XML-style formatting for parameter specification\n- Handles file type restrictions specific to certain modes\n\n## Limitations\n\n- Cannot switch to modes that don't exist in the system\n- Requires explicit user approval for each mode transition\n- Cannot use tools specific to a mode until the switch is complete\n- Applies a 500ms delay after mode switching to allow the change to take effect\n- Some modes have file type restrictions (e.g., Architect mode can only edit markdown files)\n- Mode preservation for resumption applies only to the `new_task` functionality, not general mode switching\n\n## How It Works\n\nWhen the `switch_mode` tool is invoked, it follows this process:\n\n1. **Request Validation**:\n - Validates that the requested mode exists in the system\n - Checks that the `mode_slug` parameter is provided and valid\n - Verifies the user isn't already in the requested mode\n - Ensures the `reason` parameter (if provided) is properly formatted\n\n2. **Mode Transition Preparation**:\n - Packages the mode change request with the provided reason\n - Presents the change request to the user for approval\n\n3. **Mode Activation (Upon User Approval)**:\n - Updates the UI to reflect the new mode\n - Adjusts available tools based on the mode's tool group configuration\n - Applies the mode-specific prompt and behavior\n - Applies a 500ms delay to allow the change to take effect before executing next tool\n - Enforces any file restrictions specific to the mode\n\n4. **Continuation**:\n - Proceeds with the task using the capabilities of the new mode\n - Retains relevant context from the previous interaction\n\n## Tool Group Association\n\nThe `switch_mode` tool belongs to the \"modes\" tool group but is also included in the \"always available\" tools list. This means:\n\n- It can be used in any mode regardless of the mode's configured tool groups\n- It's available alongside other core tools like `ask_followup_question` and `attempt_completion`\n- It allows mode transitions at any point in a workflow when task requirements change\n\n## Mode Structure\n\nEach mode in the system has a specific structure:\n\n- `slug`: Unique identifier for the mode (e.g., \"code\", \"ask\")\n- `name`: Display name for the mode (e.g., \"Code\", \"Ask\")\n- `roleDefinition`: The specialized role and capabilities of the mode\n- `customInstructions`: Optional mode-specific instructions that guide behavior\n- `groups`: Tool groups available to the mode with optional restrictions\n\n## Mode Capabilities\n\nThe core modes provide these specialized capabilities:\n\n- **Code Mode**: Focused on coding tasks with full access to code editing tools\n- **Architect Mode**: Specialized for system design and architecture planning, limited to editing markdown files only\n- **Ask Mode**: Optimized for answering questions and providing information\n- **Debug Mode**: Equipped for systematic problem diagnosis and resolution\n\n## Custom Modes\n\nBeyond the core modes, the system supports custom project-specific modes:\n\n- Custom modes can be defined with specific tool groups enabled\n- They can specify custom role definitions and instructions\n- The system checks custom modes first before falling back to core modes\n- Custom mode definitions take precedence over core modes with the same slug\n\n## File Restrictions\n\nDifferent modes may have specific file type restrictions:\n\n- **Architect Mode**: Can only edit files matching the `.md` extension\n- Attempting to edit restricted file types results in a `FileRestrictionError`\n- These restrictions help enforce proper separation of concerns between modes\n\n## Examples When Used\n\n- When discussing a new feature, Saarthi switches from Ask mode to Architect mode to help design the system structure.\n- After completing architecture planning in Architect mode, Saarthi switches to Code mode to implement the designed features.\n- When encountering bugs during development, Saarthi switches from Code mode to Debug mode for systematic troubleshooting.\n\n## Usage Examples\n\nSwitching to Code mode for implementation:\n```\n\ncode\nNeed to implement the login functionality based on the architecture we've discussed\n\n```\n\nSwitching to Architect mode for design:\n```\n\narchitect\nNeed to design the system architecture before implementation\n\n```\n\nSwitching to Debug mode for troubleshooting:\n```\n\ndebug\nNeed to systematically diagnose the authentication error\n\n```\n\nSwitching to Ask mode for information:\n```\n\nask\nNeed to answer questions about the implemented feature\n\n```\n" }, "docs/saarthi/advanced-usage/available-tools/tool-use-overview.md": { "content": "# Tool Use Overview\n\nSaarthi implements a sophisticated tool system that allows AI models to interact with your development environment in a controlled and secure manner. This document explains how tools work, when they're called, and how they're managed.\n\n## Core Concepts\n\n### Tool Groups\n\nTools are organized into logical groups based on their functionality:\n\n| Category | Purpose | Tools | Common Use |\n|----------|---------|-------|------------|\n| **Read Group** | File system reading and exploration | [read_file](/advanced-usage/available-tools/read-file), [list_files](/advanced-usage/available-tools/list-files), [list_code_definition_names](/advanced-usage/available-tools/list-code-definition-names) | Code exploration and analysis |\n| **Search Group** | Pattern and semantic searching | [search_files](/advanced-usage/available-tools/search-files), [codebase_search](/advanced-usage/available-tools/codebase-search) | Finding code patterns and functionality |\n| **Edit Group** | File system modifications | [apply_diff](/advanced-usage/available-tools/apply-diff), [insert_content](/advanced-usage/available-tools/insert-content), [search_and_replace](/advanced-usage/available-tools/search-and-replace), [write_to_file](/advanced-usage/available-tools/write-to-file) | Code changes and file manipulation |\n| **Browser Group** | Web automation | [browser_action](/advanced-usage/available-tools/browser-action) | Web testing and interaction |\n| **Command Group** | System command execution | [execute_command](/advanced-usage/available-tools/execute-command) | Running scripts, building projects |\n| **MCP Group** | External tool integration | [use_mcp_tool](/advanced-usage/available-tools/use-mcp-tool), [access_mcp_resource](/advanced-usage/available-tools/access-mcp-resource) | Specialized functionality through external servers |\n| **Workflow Group** | Mode and task management | [switch_mode](/advanced-usage/available-tools/switch-mode), [new_task](/advanced-usage/available-tools/new-task), [ask_followup_question](/advanced-usage/available-tools/ask-followup-question), [attempt_completion](/advanced-usage/available-tools/attempt-completion) | Context switching and task organization |\n\n### Always Available Tools\n\nCertain tools are accessible regardless of the current mode:\n\n- [ask_followup_question](/advanced-usage/available-tools/ask-followup-question): Gather additional information from users\n- [attempt_completion](/advanced-usage/available-tools/attempt-completion): Signal task completion\n- [switch_mode](/advanced-usage/available-tools/switch-mode): Change operational modes\n- [new_task](/advanced-usage/available-tools/new-task): Create subtasks\n\n## Available Tools\n\n### Read Tools\nThese tools help Saarthi understand your code and project:\n\n- [read_file](/advanced-usage/available-tools/read-file) - Examines the contents of files\n- [list_files](/advanced-usage/available-tools/list-files) - Maps your project's file structure\n- [list_code_definition_names](/advanced-usage/available-tools/list-code-definition-names) - Creates a structural map of your code\n\n### Search Tools\nThese tools help Saarthi find patterns and functionality across your codebase:\n\n- [search_files](/advanced-usage/available-tools/search-files) - Finds patterns across multiple files using regex\n- [codebase_search](/advanced-usage/available-tools/codebase-search) - Performs semantic searches across your indexed codebase\n\n### Edit Tools\nThese tools help Saarthi make changes to your code:\n\n- [apply_diff](/advanced-usage/available-tools/apply-diff) - Makes precise, surgical changes to your code\n- [insert_content](/advanced-usage/available-tools/insert-content) - Adds new lines of content without modifying existing lines\n- [search_and_replace](/advanced-usage/available-tools/search-and-replace) - Finds and replaces text or regex patterns within a file\n- [write_to_file](/advanced-usage/available-tools/write-to-file) - Creates new files or completely rewrites existing ones\n\n### Browser Tools\nThese tools help Saarthi interact with web applications:\n\n- [browser_action](/advanced-usage/available-tools/browser-action) - Automates browser interactions\n\n### Command Tools\nThese tools help Saarthi execute commands:\n\n- [execute_command](/advanced-usage/available-tools/execute-command) - Runs system commands and programs\n\n### MCP Tools\nThese tools help Saarthi connect with external services:\n\n- [use_mcp_tool](/advanced-usage/available-tools/use-mcp-tool) - Uses specialized external tools\n- [access_mcp_resource](/advanced-usage/available-tools/access-mcp-resource) - Accesses external data sources\n\n### Workflow Tools\nThese tools help manage the conversation and task flow:\n\n- [ask_followup_question](/advanced-usage/available-tools/ask-followup-question) - Gets additional information from you\n- [attempt_completion](/advanced-usage/available-tools/attempt-completion) - Presents final results\n- [switch_mode](/advanced-usage/available-tools/switch-mode) - Changes to a different mode for specialized tasks\n- [new_task](/advanced-usage/available-tools/new-task) - Creates a new subtask\n\n## Tool Calling Mechanism\n\n### Handling Complex Tasks\n\nFor certain complex operations that require multiple steps, Saarthi doesn't just figure them out on the fly. Instead, it follows predefined, internal plans to ensure consistency and accuracy.\n\nA prime example is creating a new MCP server, identified internally by `create_mcp_server`. **This identifier does not represent a tool you will see being called.** Rather, when you ask Saarthi to create a server, it triggers this known, multi-step workflow.\n\nThis specific workflow is initiated by Saarthi using its internal `fetch_instructions` tool (with the task `create_mcp_server`) to retrieve a detailed plan. This plan then guides Saarthi to make calls to several standard, documented tools in sequence, such as:\n\n* [`execute_command`](/advanced-usage/available-tools/execute-command) for running setup scripts (e.g., `npx @modelcontextprotocol/create-server`).\n* [`write_to_file`](/advanced-usage/available-tools/write-to-file) or [`apply_diff`](/advanced-usage/available-tools/apply-diff) for creating or modifying server code and configuration files.\n* [`ask_followup_question`](/advanced-usage/available-tools/ask-followup-question) to gather necessary information like API keys from you.\n* Other standard tools as needed for steps like determining file locations or updating configuration entries.\n\nSo, while the overall task (like `create_mcp_server`) is complex, it's ultimately accomplished by intelligently orchestrating the standard tools available in your environment. This approach allows Saarthi to reliably perform complex operations by leveraging the tools documented here.\n\n### When Tools Are Called\n\nTools are invoked under specific conditions:\n\n1. **Direct Task Requirements**\n - When specific actions are needed to complete a task as decided by the LLM\n - In response to user requests\n - During automated workflows\n\n2. **Mode-Based Availability**\n - Different modes enable different tool sets\n - Mode switches can trigger tool availability changes\n - Some tools are restricted to specific modes\n\n3. **Context-Dependent Calls**\n - Based on the current state of the workspace\n - In response to system events\n - During error handling and recovery\n\n### Decision Process\n\nThe system uses a multi-step process to determine tool availability:\n\n1. **Mode Validation**\n ```typescript\n isToolAllowedForMode(\n tool: string,\n modeSlug: string,\n customModes: ModeConfig[],\n toolRequirements?: Record,\n toolParams?: Record\n )\n ```\n\n2. **Requirement Checking**\n - System capability verification\n - Resource availability\n - Permission validation\n\n3. **Parameter Validation**\n - Required parameter presence\n - Parameter type checking\n - Value validation\n\n## Technical Implementation\n\n### Tool Call Processing\n\n1. **Initialization**\n - Tool name and parameters are validated\n - Mode compatibility is checked\n - Requirements are verified\n\n2. **Execution**\n ```typescript\n const toolCall = {\n type: \"tool_call\",\n name: chunk.name,\n arguments: chunk.input,\n callId: chunk.callId\n }\n ```\n\n3. **Result Handling**\n - Success/failure determination\n - Result formatting\n - Error handling\n\n### Security and Permissions\n\n1. **Access Control**\n - File system restrictions\n - Command execution limitations\n - Network access controls\n\n2. **Validation Layers**\n - Tool-specific validation\n - Mode-based restrictions\n - System-level checks\n\n## Mode Integration\n\n### Mode-Based Tool Access\n\nTools are made available based on the current mode:\n\n- **Code Mode**: Full access to file system tools, code editing capabilities, command execution\n- **Ask Mode**: Limited to reading tools, information gathering capabilities, no file system modifications\n- **Architect Mode**: Design-focused tools, documentation capabilities, limited execution rights\n- **Custom Modes**: Can be configured with specific tool access for specialized workflows\n\n### Mode Switching\n\n1. **Process**\n - Current mode state preservation\n - Tool availability updates\n - Context switching\n\n2. **Impact on Tools**\n - Tool set changes\n - Permission adjustments\n - Context preservation\n\n## Best Practices\n\n### Tool Usage Guidelines\n\n1. **Efficiency**\n - Use the most specific tool for the task\n - Avoid redundant tool calls\n - Batch operations when possible\n\n2. **Security**\n - Validate inputs before tool calls\n - Use minimum required permissions\n - Follow security best practices\n\n3. **Error Handling**\n - Implement proper error checking\n - Provide meaningful error messages\n - Handle failures gracefully\n\n### Common Patterns\n\n1. **Information Gathering**\n ```\n [ask_followup_question](/advanced-usage/available-tools/ask-followup-question) → [read_file](/advanced-usage/available-tools/read-file) → [codebase_search](/advanced-usage/available-tools/codebase-search)\n ```\n\n2. **Code Modification**\n ```\n [read_file](/advanced-usage/available-tools/read-file) → [apply_diff](/advanced-usage/available-tools/apply-diff) → [attempt_completion](/advanced-usage/available-tools/attempt-completion)\n ```\n\n3. **Task Management**\n ```\n [new_task](/advanced-usage/available-tools/new-task) → [switch_mode](/advanced-usage/available-tools/switch-mode) → [execute_command](/advanced-usage/available-tools/execute-command)\n ```\n\n## Error Handling and Recovery\n\n### Error Types\n\n1. **Tool-Specific Errors**\n - Parameter validation failures\n - Execution errors\n - Resource access issues\n\n2. **System Errors**\n - Permission denied\n - Resource unavailable\n - Network failures\n\n3. **Context Errors**\n - Invalid mode for tool\n - Missing requirements\n - State inconsistencies\n\n### Recovery Strategies\n\n1. **Automatic Recovery**\n - Retry mechanisms\n - Fallback options\n - State restoration\n\n2. **User Intervention**\n - Error notifications\n - Recovery suggestions\n - Manual intervention options\n" }, "docs/saarthi/advanced-usage/available-tools/use-mcp-tool.md": { "content": "# use_mcp_tool\n\nThe `use_mcp_tool` tool enables interaction with external tools provided by connected Model Context Protocol (MCP) servers. It extends Saarthi's capabilities with domain-specific functionality through a standardized protocol.\n\n## Parameters\n\nThe tool accepts these parameters:\n\n- `server_name` (required): The name of the MCP server providing the tool\n- `tool_name` (required): The name of the tool to execute\n- `arguments` (required/optional): A JSON object containing the tool's input parameters, following the tool's input schema. May be optional for tools that require no input.\n\n## What It Does\n\nThis tool allows Saarthi to access specialized functionality provided by external MCP servers. Each MCP server can offer multiple tools with unique capabilities, extending Saarthi beyond its built-in functionality. The system validates arguments against schemas, manages server connections, and processes responses of various content types (text, image, resource).\n\n## When is it used?\n\n- When specialized functionality not available in core tools is needed\n- When domain-specific operations are required\n- When integration with external systems or services is needed\n- When working with data that requires specific processing or analysis\n- When accessing proprietary tools through a standardized interface\n\n## Key Features\n\n- Uses the standardized MCP protocol via the `@modelcontextprotocol/sdk` library\n- Supports multiple transport mechanisms (StdioClientTransport, StreamableHTTPClientTransport and SSEClientTransport)\n- Validates arguments using Zod schema validation on both client and server sides\n- Processes multiple response content types: text, image, and resource references\n- Manages server lifecycle with automatic restarts when server code changes\n- Provides an \"always allow\" mechanism to bypass approval for trusted tools\n- Works with the companion `access_mcp_resource` tool for resource retrieval\n- Maintains proper error tracking and handling for failed operations\n- Supports configurable timeouts (1-3600 seconds, default: 60 seconds)\n- Allows file watchers to automatically detect and reload server changes\n\n## Limitations\n\n- Depends on external MCP servers being available and connected\n- Limited to the tools provided by connected servers\n- Tool capabilities vary between different MCP servers\n- Network issues can affect reliability and performance\n- Requires user approval before execution (unless in the \"always allow\" list)\n- Cannot execute multiple MCP tool operations simultaneously\n\n## Server Configuration\n\nMCP servers can be configured globally or at the project level:\n\n- **Global Configuration**: Managed through the Saarthi extension settings in VS Code. These apply across all projects unless overridden.\n- **Project-level Configuration**: Defined in a `.saarthi/mcp.json` file within your project's root directory.\n - This allows project-specific server setups.\n - Project-level servers take precedence over global servers if they share the same name.\n - Since `.saarthi/mcp.json` can be committed to version control, it simplifies sharing configurations with your team.\n\n## How It Works\n\nWhen the `use_mcp_tool` tool is invoked, it follows this process:\n\n1. **Initialization and Validation**:\n - The system verifies that the MCP hub is available\n - Confirms the specified server exists and is connected\n - Validates the requested tool exists on the server\n - Arguments are validated against the tool's schema definition\n - Timeout settings are extracted from server configuration (default: 60 seconds)\n\n2. **Execution and Communication**:\n - The system selects the appropriate transport mechanism:\n - `StdioClientTransport`: For communicating with local processes via standard I/O\n - `SSEClientTransport`: For communicating with HTTP servers via Server-Sent Events\n - `StreamableHTTPClientTransport`: For communicating with HTTP servers via Streamable HTTP Events\n - A request is sent with validated server name, tool name, and arguments\n - Communication uses the `@modelcontextprotocol/sdk` library for standardized interactions\n - Request execution is tracked with timeout handling to prevent hanging operations\n\n3. **Response Processing**:\n - Responses can include multiple content types:\n - Text content: Plain text responses\n - Image content: Binary image data with MIME type information\n - Resource references: URIs to access server resources (works with `access_mcp_resource`)\n - The system checks the `isError` flag to determine if error handling is needed\n - Results are formatted for display in the Saarthi interface\n\n4. **Resource and Error Handling**:\n - The system uses WeakRef patterns to prevent memory leaks\n - A consecutive mistake counter tracks and manages errors\n - File watchers monitor for server code changes and trigger automatic restarts\n - The security model requires approval for tool execution unless in the \"always allow\" list\n\n## Security and Permissions\n\nThe MCP architecture provides several security features:\n\n- Users must approve tool usage before execution (by default)\n- Specific tools can be marked for automatic approval in the \"always allow\" list\n- Server configurations are validated with Zod schemas for integrity\n- Configurable timeouts prevent hanging operations (1-3600 seconds)\n- Server connections can be enabled or disabled through the UI\n\n## Examples When Used\n\n- Analyzing specialized data formats using server-side processing tools\n- Generating images or other media through AI models hosted on external servers\n- Executing complex domain-specific calculations without local implementation\n- Accessing proprietary APIs or services through a controlled interface\n- Retrieving data from specialized databases or data sources\n\n## Usage Examples\n\nRequesting weather forecast data with text response:\n```\n\nweather-server\nget_forecast\n\n{\n \"city\": \"San Francisco\",\n \"days\": 5,\n \"format\": \"text\"\n}\n\n\n```\n\nAnalyzing source code with a specialized tool that returns JSON:\n```\n\ncode-analysis\ncomplexity_metrics\n\n{\n \"language\": \"typescript\",\n \"file_path\": \"src/app.ts\",\n \"include_functions\": true,\n \"metrics\": [\"cyclomatic\", \"cognitive\"]\n}\n\n\n```\n\nGenerating an image with specific parameters:\n```\n\nimage-generation\ncreate_image\n\n{\n \"prompt\": \"A futuristic city with flying cars\",\n \"style\": \"photorealistic\",\n \"dimensions\": {\n \"width\": 1024,\n \"height\": 768\n },\n \"format\": \"webp\"\n}\n\n\n```\n\nAccessing a resource through a tool that returns a resource reference:\n```\n\ndatabase-connector\nquery_and_store\n\n{\n \"database\": \"users\",\n \"type\": \"select\",\n \"fields\": [\"name\", \"email\", \"last_login\"],\n \"where\": {\n \"status\": \"active\"\n },\n \"store_as\": \"active_users\"\n}\n\n\n```\n\nTool with no required arguments:\n```\n\nsystem-monitor\nget_current_status\n\n{}\n\n\n```\n" }, "docs/saarthi/advanced-usage/available-tools/write-to-file.md": { "content": "# write_to_file\n\nThe `write_to_file` tool creates new files or completely replaces existing file content with an interactive approval process. It provides a diff view for reviewing changes before they're applied.\n\n## Parameters\n\nThe tool accepts these parameters:\n\n- `path` (required): The path of the file to write to, relative to the current working directory\n- `content` (required): The complete content to write to the file\n- `line_count` (required): The number of lines in the file, including empty lines\n\n## What It Does\n\nThis tool writes content to a specified file, either creating a new file if it doesn't exist or completely overwriting an existing file. All changes require explicit user approval through a diff view interface, where users can review and even edit the proposed changes before they're applied.\n\n## When is it used?\n\n- When Saarthi needs to create a new file from scratch\n- When Saarthi needs to completely rewrite an existing file\n- When creating multiple files for a new project\n- When generating configuration files, documentation, or source code\n- When you need to review changes before they're applied\n\n## Key Features\n\n- Interactive Approval: Shows changes in a diff view requiring explicit approval before applying\n- User Edit Support: Allows editing the proposed content before final approval\n- Safety Measures: Detects code omission, validates paths, and prevents truncated content\n- Editor Integration: Opens a diff view that scrolls to the first difference automatically\n- Content Preprocessing: Handles artifacts from different AI models to ensure clean content\n- Access Control: Validates against `.saarthiignore` restrictions before making changes\n- Parent Directories: May handle directory creation through system dependencies\n- Complete Replacement: Provides a fully transformed file in a single operation\n\n## Limitations\n\n- Not suitable for existing files: Much slower and less efficient than `apply_diff` for modifying existing files\n- Performance with large files: Operation becomes significantly slower with larger files\n- Complete overwrite: Replaces entire file content, cannot preserve original content\n- Line count required: Needs accurate line count to detect potential content truncation\n- Review overhead: The approval process adds extra steps compared to direct edits\n- Interactive only: Cannot be used in automated workflows that require non-interactive execution\n\n## How It Works\n\nWhen the `write_to_file` tool is invoked, it follows this process:\n\n1. **Parameter Validation**: Validates the required parameters and permissions\n - Checks that `path`, `content`, and `line_count` are provided\n - If `line_count` is missing/invalid, reverts any diff view changes and returns an error suggesting alternative tools (`apply_diff`, `insert_content`, etc.) if modifying an existing file.\n - Validates the file is allowed (not restricted by `.saarthiignore`)\n - Ensures the path is within the workspace boundaries\n - Tracks consecutive mistake counts for missing parameters\n - Shows specific error messages for each validation failure\n\n2. **Content Preprocessing**:\n - Removes code block markers that might be added by AI models\n - Handles escaped HTML entities (specifically for non-Claude models)\n - Strips line numbers if accidentally included in content\n - Performs model-specific processing for different AI providers\n\n3. **Diff View Generation**:\n - Opens a diff view in the editor showing the proposed changes\n - Adds a 300ms delay to ensure UI responsiveness\n - Scrolls automatically to the first difference\n - Highlights changes for easy review\n\n4. **User Approval Process**:\n - Waits for explicit user approval to proceed\n - Allows users to edit the content in the diff view\n - Captures any user edits for the final content\n - Provides option to reject changes entirely\n - Detects and incorporates user modifications into the final result\n\n5. **Safety Validation**:\n - Detects potential content truncation by comparing with provided line count\n - Shows warnings if content appears incomplete\n - Validates file path and access permissions\n - Specifically checks if files are outside the workspace with `isOutsideWorkspace` flag\n\n6. **File Writing**:\n - Writes the approved content (with any user edits) to the file\n - Provides confirmation of successful write\n - Resets the consecutive mistakes counter on success\n\n## Examples When Used\n\n- When creating a new project, Saarthi generates multiple files but lets you review each before committing changes.\n- When setting up configuration files, Saarthi shows the proposed configuration in a diff view for approval.\n- When generating documentation, Saarthi creates markdown files but lets you make final adjustments in the diff view.\n- When developing a prototype, Saarthi shows complete source files in a diff view where you can fine-tune before saving.\n\n## Usage Examples\n\nCreating a new JSON configuration file:\n```\n\nconfig/settings.json\n\n{\n \"apiEndpoint\": \"https://api.example.com\",\n \"theme\": {\n \"primaryColor\": \"#007bff\",\n \"secondaryColor\": \"#6c757d\",\n \"fontFamily\": \"Arial, sans-serif\"\n },\n \"features\": {\n \"darkMode\": true,\n \"notifications\": true,\n \"analytics\": false\n },\n \"version\": \"1.0.0\"\n}\n\n14\n\n```\n\nCreating a simple HTML file:\n```\n\nsrc/index.html\n\n\n\n\n \n \n My Application\n \n\n\n
\n \n\n\n\n13\n\n```\n\nCreating a JavaScript module:\n```\n\nsrc/utils/helpers.js\n\n/**\n * Utility functions for the application\n */\n\nexport function formatDate(date) {\n return new Date(date).toLocaleDateString();\n}\n\nexport function calculateTotal(items) {\n return items.reduce((sum, item) => sum + item.price, 0);\n}\n\nexport function debounce(func, delay) {\n let timeout;\n return function(...args) {\n clearTimeout(timeout);\n timeout = setTimeout(() => func.apply(this, args), delay);\n };\n}\n\n18\n\n```\n" }, "docs/saarthi/advanced-usage/context-poisoning.md": { "content": "# Context Poisoning\n\n:::info\nContext poisoning is a persistent issue within a given session. Once a chat session's context is compromised, treat that session as disposable. Starting fresh with a clean context is crucial for maintaining the accuracy and effectiveness of your Saarthi agent.\n:::\n\nContext poisoning occurs when inaccurate or irrelevant data contaminates the language model's active context. This leads the model to draw incorrect conclusions, provide erroneous information to tools, and progressively deviate from the intended task with each interaction.\n\n## Symptoms of Context Poisoning\n\nIdentify context poisoning by observing these behaviors:\n\n* **Degraded Output Quality:** Suggestions become nonsensical, repetitive, or irrelevant.\n* **Tool Misalignment:** Tool calls no longer correspond to the user's requests.\n* **Orchestration Failures:** Orchestrator chains may stall, loop indefinitely, or fail to complete.\n* **Temporary Fixes:** Re-applying a clean prompt or instructions offers only brief respite before issues resurface.\n* **Tool Usage Confusion:** The model struggles to correctly use or recall how to use tools defined in the system prompt.\n\n## Common Causes\n\nContext poisoning can be triggered by several factors:\n\n* **Model Hallucination:** The model generates an incorrect piece of information and subsequently treats it as a factual part of the context.\n* **Code Comments:** Outdated, incorrect, or ambiguous comments in the codebase can be misinterpreted by the model, leading it down the wrong path.\n* **Contaminated User Input:** Copy-pasting logs or text containing hidden or rogue control characters.\n* **Context Window Overflow:** As a session grows, older, useful information may be pushed out of the model's limited context window, allowing \"poisoned\" data to have a greater relative impact.\n\nOnce bad data enters the context, it tends to persist. The model re-evaluates this tainted information in subsequent reasoning cycles, similar to a permanent flaw affecting its perception until the context is completely reset.\n\n## Can a \"Wake-Up Prompt\" Resolve Context Poisoning?\n\n**Short Answer:** No.\n\nA corrective prompt might temporarily suppress symptoms, but the problematic data remains in the conversational buffer. The model will likely revert to the poisoned state as soon as the interaction deviates from the narrow scope of the corrective prompt.\n\n**Detailed Explanation:**\n\n* Re-injecting the full set of tool definitions or core directives can sometimes mask the damage for one or some interactions following the initial context poisoning .\n* However, the underlying poisoned context remains. Any query or task outside the immediate \"patch\" will likely re-trigger the original issue.\n* This approach is unreliable, akin to placing a warning label on a leaking pipe instead of repairing it.\n\n## Effective Recovery Strategies\n\nTo reliably recover from context poisoning:\n\n* **Hard Reset the Session:** The most dependable solution is to start a new chat session. This clears the contaminated context entirely.\n* **Minimize Manual Data Dumps:** When pasting logs or other data, be selective. Only include the essential information the model requires.\n* **Manage Context Window Size:** For large or complex tasks, consider breaking them into smaller, focused chat sessions. This helps ensure that stale or irrelevant information ages out of the context window more quickly.\n* **Validate Tool Output:** If a tool returns nonsensical or clearly incorrect data, delete that message from the chat history before the model can process it and incorporate it into its context.\n\n## Addressing a Common Question: The \"Magic Bullet\" Prompt\n\nA frequent question from the community is:\n> \"Have you found a prompt that wakes it back up? Maybe a prompt that just has the tools instructions we can push back in manually?”\n\nAs explained, no single prompt offers a lasting fix. Any immediate improvement is superficial because the corrupted lines of text persist in the session's history, ready to cause further issues. The only robust solution is to discard the compromised session, initiate a new one, and provide it with a clean prompt and the correct tool definitions from the outset.\n" }, "docs/saarthi/advanced-usage/large-projects.md": { "content": "# Working with Large Projects\n\nSaarthi can be used with projects of any size, but large projects require some extra care to manage context effectively. Here are some tips for working with large codebases:\n\n## Understanding Context Limits\n\nSaarthi uses large language models (LLMs) that have a limited \"context window.\" This is the maximum amount of text (measured in tokens) that the model can process at once. If the context is too large, the model may not be able to understand your request or generate accurate responses.\n\nThe context window includes:\n\n* The system prompt (instructions for Saarthi).\n* The conversation history.\n* The content of any files you mention using `@`.\n* The output of any commands or tools Saarthi uses.\n\n## Strategies for Managing Context\n\n1. **Be Specific:** When referring to files or code, use specific file paths and function names. Avoid vague references like \"the main file.\"\n\n2. **Use Context Mentions Effectively:** Use `@/path/to/file.ts` to include specific files. Use `@problems` to include current errors and warnings. Use `@` followed by a commit hash to reference specific Git commits.\n\n3. **Break Down Tasks:** Divide large tasks into smaller, more manageable sub-tasks. This helps keep the context focused.\n\n4. **Summarize:** If you need to refer to a large amount of code, consider summarizing the relevant parts in your prompt instead of including the entire code.\n\n5. **Prioritize Recent History:** Saarthi automatically truncates older messages in the conversation history to stay within the context window. Be mindful of this, and re-include important context if needed.\n\n6. **Use Prompt Caching (if available):** Some API providers like Anthropic, OpenAI, OpenRouter and Requesty support \"prompt caching\". This caches your prompts for use in future tasks and helps reduce the cost and latency of requests.\n\n## Example: Refactoring a Large File\n\nLet's say you need to refactor a large TypeScript file (`src/components/MyComponent.tsx`). Here's a possible approach:\n\n1. **Initial Overview:**\n ```\n @/src/components/MyComponent.tsx List the functions and classes in this file.\n ```\n\n2. **Target Specific Functions:**\n ```\n @/src/components/MyComponent.tsx Refactor the `processData` function to use `async/await` instead of Promises.\n ```\n\n3. **Iterative Changes:** Make small, incremental changes, reviewing and approving each step.\n\nBy breaking down the task and providing specific context, you can work effectively with large files even with a limited context window." }, "docs/saarthi/advanced-usage/local-development-setup.mdx": { "content": "---\nsidebar_label: Local Development Setup\n---\n \n# Local Development Setup\n\n:::note Developer Information Only\nThis section is for developers contributing to Saarthi or those who wish to build and run the extension from source.\n:::\n\nTo set up Saarthi for local development:\n\n1. **Clone the repository:**\n ```bash\n git clone https://github.com/godspeedsystems/Saarthi.git\n cd Saarthi\n ```\n\n2. **Install dependencies:**\n Saarthi uses `pnpm` for package management.\n ```bash\n pnpm install\n ```\n\n3. **Run the extension for development:**\n You have two primary methods:\n\n * **Directly in VS Code (Recommended for active development):**\n * Open the cloned `Saarthi` folder in VS Code.\n * Press `F5` (or select **Run** → **Start Debugging** from the menu). This opens a new VS Code window (Extension Development Host) with Saarthi running.\n * Changes to the webview code (under `webview-ui/`) will often update live.\n * Changes to the core extension code (under `src/`) typically require restarting the Extension Development Host window.\n\n * **Building and Installing a VSIX package:**\n This method is useful for testing the packaged extension or distributing development builds.\n 1. Build the VSIX file:\n ```bash\n pnpm build\n ```\n A `.vsix` file (e.g., `saarthi-.vsix`) will be created in the `bin/` directory.\n\n 2. Install the VSIX in VS Code:\n * **Using the command line:**\n ```bash\n code --install-extension bin/saarthi-.vsix\n ```\n (Replace `` with the actual version number in the filename).\n * **Using the VS Code interface:**\n * Open the Extensions view (`Ctrl+Shift+X` or `Cmd+Shift+X`).\n * Click the \"...\" (More Actions) menu in the Extensions view sidebar.\n * Select \"Install from VSIX...\"\n * Browse to and select the generated `.vsix` file from the `bin/` directory.\n\n![VS Code's Install from VSIX dialog](/img/installing/installing-2.png)\n *Installing a development VSIX using VS Code's \"Install from VSIX...\" dialog.*\n" }, "docs/saarthi/advanced-usage/local-models.md": { "content": "# Using Local Models\n\nSaarthi supports running language models locally on your own machine using [Ollama](https://ollama.com/) and [LM Studio](https://lmstudio.ai/). This offers several advantages:\n\n* **Privacy:** Your code and data never leave your computer.\n* **Offline Access:** You can use Saarthi even without an internet connection.\n* **Cost Savings:** Avoid API usage fees associated with cloud-based models.\n* **Customization:** Experiment with different models and configurations.\n\n**However, using local models also has some drawbacks:**\n\n* **Resource Requirements:** Local models can be resource-intensive, requiring a powerful computer with a good CPU and, ideally, a dedicated GPU.\n* **Setup Complexity:** Setting up local models can be more complex than using cloud-based APIs.\n* **Model Performance:** The performance of local models can vary significantly. While some are excellent, they may not always match the capabilities of the largest, most advanced cloud models.\n* **Limited Features**: Local models (and many online models) often do not support advanced features such as prompt caching, computer use, and others.\n\n## Supported Local Model Providers\n\nSaarthi currently supports two main local model providers:\n\n1. **Ollama:** A popular open-source tool for running large language models locally. It supports a wide range of models.\n2. **LM Studio:** A user-friendly desktop application that simplifies the process of downloading, configuring, and running local models. It also provides a local server that emulates the OpenAI API.\n\n## Setting Up Local Models\n\nFor detailed setup instructions, see:\n* [Setting up Ollama](/providers/ollama)\n* [Setting up LM Studio](/providers/lmstudio)\n\nBoth providers offer similar capabilities but with different user interfaces and workflows. Ollama provides more control through its command-line interface, while LM Studio offers a more user-friendly graphical interface.\n\n## Troubleshooting\n\n* **\"No connection could be made because the target machine actively refused it\":** This usually means that the Ollama or LM Studio server isn't running, or is running on a different port/address than Saarthi is configured to use. Double-check the Base URL setting.\n\n* **Slow Response Times:** Local models can be slower than cloud-based models, especially on less powerful hardware. If performance is an issue, try using a smaller model.\n\n* **Model Not Found:** Ensure you have typed in the name of the model correctly. If you're using Ollama, use the same name that you provide in the `ollama run` command.\n" }, "docs/saarthi/advanced-usage/prompt-engineering.md": { "content": "# Prompt Engineering Tips\n\nPrompt engineering is the art of crafting effective instructions for AI models like Saarthi. Well-written prompts lead to better results, fewer errors, and a more efficient workflow.\n\n## General Principles\n\n* **Be Clear and Specific:** Clearly state what you want Saarthi to do. Avoid ambiguity.\n * **Bad:** Fix the code.\n * **Good:** Fix the bug in the `calculateTotal` function that causes it to return incorrect results.\n\n* **Provide Context:** Use [Context Mentions](/basic-usage/context-mentions) to refer to specific files, folders, or problems.\n * **Good:** `@/src/utils.ts` Refactor the `calculateTotal` function to use async/await.\n\n* **Break Down Tasks:** Divide complex tasks into smaller, well-defined steps.\n\n* **Give Examples:** If you have a specific coding style or pattern in mind, provide examples.\n\n* **Specify Output Format:** If you need the output in a particular format (e.g., JSON, Markdown), specify it in the prompt.\n\n* **Iterate:** Don't be afraid to refine your prompt if the initial results aren't what you expect.\n\n## Thinking vs. Doing\n\nIt's often helpful to guide Saarthi through a \"think-then-do\" process:\n\n1. **Analyze:** Ask Saarthi to analyze the current code, identify problems, or plan the approach.\n2. **Plan:** Have Saarthi outline the steps it will take to complete the task.\n3. **Execute:** Instruct Saarthi to implement the plan, one step at a time.\n4. **Review:** Carefully review the results of each step before proceeding.\n\n## Using Custom Instructions\n\nYou can provide custom instructions to further tailor Saarthi's behavior. There are two types of custom instructions:\n\n* **Global Custom Instructions:** Apply to all modes.\n* **Mode-Specific Custom Instructions:** Apply only to a specific mode (e.g., Code, Architect, Ask, Debug, or a custom mode).\n\nCustom instructions are added to the system prompt, providing persistent guidance to the AI model. You can use these to:\n\n* Enforce coding style guidelines.\n* Specify preferred libraries or frameworks.\n* Define project-specific conventions.\n* Adjust Saarthi's tone or personality.\n\nSee the [Custom Instructions](/features/custom-instructions) section for more details.\n\n## Handling Ambiguity\n\nIf your request is ambiguous or lacks sufficient detail, Saarthi might:\n\n* **Make Assumptions:** It might proceed based on its best guess, which may not be what you intended.\n* **Ask Follow-Up Questions:** It might use the `ask_followup_question` tool to clarify your request.\n\nIt's generally better to provide clear and specific instructions from the start to avoid unnecessary back-and-forth.\n\n## Providing Feedback\n\nIf Saarthi doesn't produce the desired results, you can provide feedback by:\n\n* **Rejecting Actions:** Click the \"Reject\" button when Saarthi proposes an action you don't want.\n* **Providing Explanations:** When rejecting, explain *why* you're rejecting the action. This helps Saarthi learn from its mistakes.\n* **Rewording Your Request:** Try rephrasing your initial task or providing more specific instructions.\n* **Manually Correcting:** If there are a few small issues, you can also directly modify the code before accepting the changes.\n\n## Examples\n\n**Good Prompt:**\n\n> `@/src/components/Button.tsx` Refactor the `Button` component to use the `useState` hook instead of the `useReducer` hook.\n\n**Bad Prompt:**\n\n> Fix the button.\n\n**Good Prompt:**\n\n> Create a new file named `utils.py` and add a function called `calculate_average` that takes a list of numbers and returns their average.\n\n**Bad Prompt:**\n\n> Write some Python code.\n\n**Good Prompt:**\n\n> `@problems` Address all errors and warnings in the current file.\n\n**Bad Prompt:**\n\n> Fix everything.\n\nBy following these tips, you can write effective prompts that get the most out of Saarthi's capabilities.\n" }, "docs/saarthi/advanced-usage/prompt-structure.md": { "content": "# Prompt Structure\n\nThis page explains the technical structure of prompts in Saarthi - how messages are constructed and sent to the Large Language Model (LLM).\n\n## Core Message Types\n\nSaarthi uses three primary message types when communicating with LLMs:\n\n- **System Prompt**: The initial instructions that define Saarthi's capabilities, persona, and operational rules\n- **User Messages**: Content sent by you (the user) to Saarthi\n- **Assistant Messages**: Responses generated by the LLM based on your requests\n\nAt the API level, there's also a fourth message role:\n\n- **Tool Messages**: Results returned from tool executions, sent back to the LLM as input\n\nUnderstanding these message types helps you work more effectively with Saarthi and can be valuable for troubleshooting or advanced customization.\n\n## System Prompt\n\nThe system prompt is the foundation of Saarthi's behavior. It contains:\n\n- **Role Definition**: The core persona instructions based on the selected mode (Code, Ask, Debug, etc.)\n- **Tool Descriptions**: Detailed information about available tools, including parameters and examples\n- **Tool Use Guidelines**: Rules for how tools should be used (sequential execution, waiting for results)\n- **Capabilities**: Description of what Saarthi can do in the current environment\n- **Available Modes**: List of all available modes and their descriptions\n- **Operational Rules**: Critical guidelines for handling files, project structure, and user interaction\n- **System Information**: Details about your environment (OS, shell, working directory)\n- **Custom Instructions**: Your global and mode-specific customizations\n\nThe system prompt is generated dynamically each time you interact with Saarthi, adapting to your current mode, available tools, and custom settings.\n\n### Custom System Prompts\n\nAdvanced users can create custom system prompts for specific modes by placing a `.saarthi/system-prompt-` file in their workspace. When present, Saarthi uses this file instead of generating the standard system prompt sections, allowing for complete customization of Saarthi's behavior in that mode.\n\n## User Messages\n\nUser messages contain your direct inputs to Saarthi, plus additional contextual information:\n\n- **Your Query**: The text you type in the chat interface\n- **Images**: Any images you include in your message (for supported models)\n- **Environment Details**: Automatically appended information about your workspace state:\n - Open files/tabs\n - Cursor position\n - Active terminals with output\n - Recently modified files\n - Current time\n - Token/cost information\n - Current mode\n - File listing (on initial connection)\n\nThis automatic context enrichment helps Saarthi understand your workspace without requiring you to explicitly describe it.\n\n## Assistant Messages\n\nAssistant messages are the LLM's responses, which may include:\n\n- **Text Responses**: Direct answers to your queries\n- **Thinking**: Internal reasoning process (visible when enabled)\n- **Tool Calls**: Requests to use specific tools like reading files or executing commands\n\nNote that while assistant messages contain tool calls, the results of those tools are sent back to the LLM in separate tool messages, not as part of the assistant message itself.\n\n## Message Flow\n\nHere's how these components work together:\n\n1. **Initial Setup**: Saarthi generates the system prompt based on your selected mode and configuration\n2. **User Input**: You send a message, which is enriched with environment details\n3. **LLM Processing**: The LLM receives all previous messages plus your new input\n4. **Assistant Response**: The LLM generates a response, potentially using tools\n5. **Tool Execution**: If the LLM requests a tool, Saarthi executes it and provides the result\n6. **Conversation History**: All messages are maintained in a structured history for context\n\n## Technical Implementation\n\nInternally, Saarthi's prompt construction is handled by several components:\n\n- **System Prompt Generation**: The `SYSTEM_PROMPT` function in `src/core/prompts/system.ts` assembles the complete system prompt\n- **Section Generators**: Specialized functions create each section of the system prompt\n- **Message Transformation**: Provider-specific transformers convert Saarthi's internal message format to the format required by each LLM API\n- **Custom Prompt Loading**: The `loadSystemPromptFile` function checks for and processes custom system prompt files\n\n## Support Prompts\n\nAlongside the main chat flow, Saarthi uses specialized templates for specific code actions:\n\n- **Code Action Prompts**: For commands like \"Explain\", \"Fix\", \"Improve\", or \"Add to Context\"\n- **Template-Based**: Generated from templates in `src/shared/support-prompt.ts`\n- **Independent Context**: Often operates without the main chat history\n- **Task-Specific Format**: Optimized for the specific code task being performed\n\nThese support prompts work outside the normal conversation flow to provide focused assistance for specific coding tasks.\n\n## Optimizing Your Interactions\n\nUnderstanding this structure can help you:\n\n- **Write Better Prompts**: Knowing what context Saarthi already has helps you avoid redundant information\n- **Troubleshoot Issues**: Understanding message flow helps identify where problems might occur\n- **Create Custom Modes**: With knowledge of the system prompt structure, you can create more effective custom modes\n- **Use Custom System Prompts**: Advanced users can create entirely custom system prompts for specialized use cases\n\nThis technical foundation powers all of Saarthi's capabilities, enabling it to understand your requests and effectively utilize available tools to complete tasks." }, "docs/saarthi/advanced-usage/rate-limits-costs.md": { "content": "# Rate Limits and Costs\n\nUnderstanding and managing API usage is crucial for a smooth and cost-effective experience with Saarthi. This section explains how to track your token usage and costs. Rate limits, which default to 0 (disabled) and typically don't need adjustment, are now configured per profile; see the [API Configuration Profiles](/features/api-configuration-profiles#creating-a-profile) documentation for details on how to set them if needed.\n\n## Token Usage\n\nSaarthi interacts with AI models using tokens. Tokens are essentially pieces of words. The number of tokens used in a request and response affects both the processing time and the cost.\n\n* **Input Tokens:** These are the tokens in your prompt, including the system prompt, your instructions, and any context provided (e.g., file contents).\n* **Output Tokens:** These are the tokens generated by the AI model in its response.\n\nYou can see the number of input and output tokens used for each interaction in the chat history.\n\n## Cost Calculation\n\nMost AI providers charge based on the number of tokens used. Pricing varies depending on the provider and the specific model.\n\nSaarthi automatically calculates the estimated cost of each API request based on the configured model's pricing. This cost is displayed in the chat history, next to the token usage.\n\n**Note:**\n\n* The cost calculation is an *estimate*. The actual cost may vary slightly depending on the provider's billing practices.\n* Some providers may offer free tiers or credits. Check your provider's documentation for details.\n* Some providers offer prompt caching which greatly lowers cost.\n\n### Limiting Auto-Approved Requests\n\nTo further help manage API costs and prevent unexpected expenses, Saarthi includes a \"Max Requests\" setting for auto-approved actions. This allows you to define a specific limit on how many consecutive API calls Saarthi can make without requiring your explicit re-approval during a task.\n\n* **How it works:** If you set a limit (e.g., 5 requests), Saarthi will perform up to 5 auto-approved API calls. Before making the 6th call, it will pause and prompt you to \"Reset and Continue,\" as shown below.\n![Warning message indicating the auto-approved request limit has been reached.](/img/v3.18.0/v3.18.0-1.png)\n *Notification when the auto-approved request limit is met.*\n* **Configuration:** This limit is configured within the \"Auto-approve actions\" settings. You can set a specific number or choose \"Unlimited.\" For detailed steps on configuring this and other auto-approval settings, see the [Auto-Approving Actions documentation](/features/auto-approving-actions).\n![Setting the Max Requests limit for auto-approved actions in Saarthi settings.](/img/v3.18.0/v3.18.0.png)\n *Setting the \"Max Requests\" for auto-approved actions.*\n\nThis feature provides an additional safeguard, particularly for complex or long-running tasks where multiple API calls might be involved.\n\n## Tips for Optimizing Token Usage\n\n* **Be Concise:** Use clear and concise language in your prompts. Avoid unnecessary words or details.\n* **Provide Only Relevant Context:** Use context mentions (`@file.ts`, `@folder/`) selectively. Only include the files that are directly relevant to the task.\n* **Break Down Tasks:** Divide large tasks into smaller, more focused sub-tasks.\n* **Use Custom Instructions:** Provide custom instructions to guide Saarthi's behavior and reduce the need for lengthy explanations in each prompt.\n* **Choose the Right Model:** Some models are more cost-effective than others. Consider using a smaller, faster model for tasks that don't require the full power of a larger model.\n* **Use Modes:** Different modes can access different tools, for example `Architect` can't modify code, which makes it a safe choice when analyzing a complex codebase, without worrying about accidentally allowing expensive operations.\n* **Disable MCP If Not Used:** If you're not using MCP (Model Context Protocol) features, consider [disabling it in the MCP settings](../features/mcp/using-mcp-in-saarthi#enabling-or-disabling-mcp-server-creation) to significantly reduce the size of the system prompt and save tokens.\n\nBy understanding and managing your API usage, you can use Saarthi effectively and efficiently.\n" }, "docs/saarthi/advanced-usage/roo-code-nightly.mdx": { "content": "---\nsidebar_label: Saarthi Nightly\n---\n\n# Saarthi Nightly\n\nFor users who want the latest features and updates, Saarthi offers a prerelease build called **Saarthi Nightly**, which is automatically published whenever changes are merged into our main development branch. This means it always reflects the latest code. It's designed for users who want to access the newest features and contribute to early testing.\n\n## Understanding Nightly vs. Other Prereleases\n\nIt's important to distinguish Saarthi Nightly from other types of prerelease software (like beta versions or release candidates):\n\n* **Saarthi Nightly:**\n * **Frequency:** Automated builds, generated on each merge to the main development branch.\n * **Content:** Reflects the very latest merged code.\n * **Stability:** Highly experimental; may contain bugs or incomplete features.\n * **Testing:** Minimal or no manual testing before release.\n * **Audience:** Primarily for developers, internal testing, or users comfortable with potentially unstable software who want to see the absolute latest changes and help identify issues early.\n\n* **General Prerelease (e.g., Beta, Release Candidate):**\n * **Frequency:** Released periodically (e.g., weekly, monthly) as specific milestones are met.\n * **Content:** Represents a more curated and planned intermediate version.\n * **Stability:** More stable than Nightly builds; often undergoes some manual testing or validation.\n * **Testing:** Typically receives more focused testing.\n * **Audience:** Intended for a wider group of early adopters or beta testers to gather feedback before a stable release.\n\nIn short: **Nightly = latest code, potentially unstable, frequent updates.** Other prereleases are generally less frequent, more validated, and aimed at broader testing.\n\n## Installing Saarthi Nightly\n\nTo install Saarthi Nightly:\n\n1. Open VS Code.\n2. Access Extensions: Click the Extensions icon in the Activity Bar or press `Ctrl+Shift+X` (Windows/Linux) or `Cmd+Shift+X` (macOS).\n3. Search for \"Saarthi Nightly\".\n4. Select \"Saarthi Nightly\" by Saarthi and click **Install**.\n5. Reload VS Code if prompted.\n\n![Saarthi Nightly extension in VS Code Marketplace](/img/installing/installing-5.png)\n*Saarthi Nightly in the VS Code Marketplace.*\n\nYou can have both the stable version of Saarthi and Saarthi Nightly installed simultaneously.\n" }, "docs/saarthi/basic-usage/context-mentions.md": { "content": "# Context Mentions\n\nContext mentions are a powerful way to provide Saarthi with specific information about your project, allowing it to perform tasks more accurately and efficiently. You can use mentions to refer to files, folders, problems, and Git commits. Context mentions start with the `@` symbol.\n\n![Context Mentions Overview - showing the @ symbol dropdown menu in the chat interface](/img/context-mentions/context-mentions.png)\n\n*Context mentions overview showing the @ symbol dropdown menu in the chat interface.*\n\n## Types of Mentions\n\n![File mention example showing a file being referenced with @ and its contents appearing in the conversation](/img/context-mentions/context-mentions-1.png)\n\n*File mentions add actual code content into the conversation for direct reference and analysis.*\n\n| Mention Type | Format | Description | Example Usage |\n|--------------|--------|-------------|--------------|\n| **File** | `@/path/to/file.ts` | Includes file contents in request context | \"Explain the function in @/src/utils.ts\" |\n| **Folder** | `@/path/to/folder` | Includes contents of all files directly in the folder (non-recursive) | \"Analyze the code in @/src/components\" |\n| **Problems** | `@problems` | Includes VS Code Problems panel diagnostics | \"@problems Fix all errors in my code\" |\n| **Terminal** | `@terminal` | Includes recent terminal command and output | \"Fix the errors shown in @terminal\" |\n| **Git Commit** | `@a1b2c3d` | References specific commit by hash | \"What changed in commit @a1b2c3d?\" |\n| **Git Changes** | `@git-changes` | Shows uncommitted changes | \"Suggest a message for @git-changes\" |\n| **URL** | `@https://example.com` | Imports website content | \"Summarize @https://docusaurus.io/\" |\n\n### File Mentions\n\n![File mention example showing a file being referenced with @ and its contents appearing in the conversation](/img/context-mentions/context-mentions-1.png)\n\n*File mentions incorporate source code with line numbers for precise references.*\n| Capability | Details |\n|------------|---------|\n| **Format** | `@/path/to/file.ts` (always start with `/` from workspace root) |\n| **Provides** | Complete file contents with line numbers |\n| **Supports** | Text files, PDFs, and DOCX files (with text extraction) |\n| **Works in** | Initial requests, feedback responses, and follow-up messages |\n| **Limitations** | Very large files may be truncated; binary files not supported |\n\n### Folder Mentions\n\n![Folder mention example showing directory contents being referenced in the chat](/img/context-mentions/context-mentions-2.png)\n\n*Folder mentions include the content of all files within the specified directory.*\n| Capability | Details |\n|------------|---------|\n| **Format** | `@/path/to/folder` (no trailing slash) |\n| **Provides** | Complete contents of all files within the directory |\n| **Includes** | Contents of non-binary text files directly within the folder (not recursive) |\n| **Best for** | Providing context from multiple files in a directory |\n| **Tip** | Be mindful of context window limits when mentioning large directories |\n\n### Problems Mention\n\n![Problems mention example showing VS Code problems panel being referenced with @problems](/img/context-mentions/context-mentions-3.png)\n\n*Problems mentions import diagnostics directly from VS Code's problems panel.*\n| Capability | Details |\n|------------|---------|\n| **Format** | `@problems` |\n| **Provides** | All errors and warnings from VS Code's problems panel |\n| **Includes** | File paths, line numbers, and diagnostic messages |\n| **Groups** | Problems organized by file for better clarity |\n| **Best for** | Fixing errors without manual copying |\n\n### Terminal Mention\n![Terminal mention example showing terminal output being included in Saarthi's context](/img/context-mentions/context-mentions-4.png)\n\n*Terminal mentions capture recent command output for debugging and analysis.*\n\n| Capability | Details |\n|------------|---------|\n| **Format** | `@terminal` |\n| **Captures** | Last command and its complete output |\n| **Preserves** | Terminal state (doesn't clear the terminal) |\n| **Limitation** | Limited to visible terminal buffer content |\n| **Best for** | Debugging build errors or analyzing command output |\n\n### Git Mentions\n\n![Git commit mention example showing commit details being analyzed by Saarthi](/img/context-mentions/context-mentions-5.png)\n\n*Git mentions provide commit details and diffs for context-aware version analysis.*\n| Type | Format | Provides | Limitations |\n|------|--------|----------|------------|\n| **Commit** | `@a1b2c3d` | Commit message, author, date, and complete diff | Only works in Git repositories |\n| **Working Changes** | `@git-changes` | `git status` output and diff of uncommitted changes | Only works in Git repositories |\n\n### URL Mentions\n\n*URL mentions import external web content and convert it to readable Markdown format.*\n\n| Capability | Details |\n|------------|---------|\n| **Format** | `@https://example.com` |\n| **Processing** | Uses headless browser to fetch content |\n| **Cleaning** | Removes scripts, styles, and navigation elements |\n| **Output** | Converts content to Markdown for readability |\n| **Limitation** | Complex pages may not convert perfectly |\n\n## How to Use Mentions\n\n1. Type `@` in the chat input to trigger the suggestions dropdown\n2. Continue typing to filter suggestions or use arrow keys to navigate\n3. Select with Enter key or mouse click\n4. Combine multiple mentions in a request: \"Fix @problems in @/src/component.ts\"\n\nThe dropdown automatically suggests:\n- Recently opened files\n- Visible folders\n- Recent git commits\n- Special keywords (`problems`, `terminal`, `git-changes`)\n- **All currently open files** (regardless of ignore settings or directory filters)\n\nThe dropdown automatically filters out common directories like `node_modules`, `.git`, `dist`, and `out` to reduce noise, even though their content could be included if manually typed.\n\n## Important Behaviors\n\n### Ignore File Interactions\n\n| Behavior | Description |\n|----------|-------------|\n| **`.saarthiignore` bypass** | File and folder `@mentions` bypass `.saarthiignore` checks when fetching content for context. Content from ignored files will be included if directly mentioned. |\n| **`.gitignore` bypass** | Similarly, file and folder `@mentions` do not respect `.gitignore` rules when fetching content. |\n| **Git command respect** | Git-related mentions (`@git-changes`, `@commit-hash`) do respect `.gitignore` since they rely on Git commands. |\n\n\n" }, "docs/saarthi/basic-usage/how-tools-work.md": { "content": "# How Tools Work\n\nSaarthi uses tools to interact with your code and environment. These specialized helpers perform specific actions like reading files, making edits, running commands, or searching your codebase. Tools provide automation for common development tasks without requiring manual execution.\n\n## Tool Workflow\n\nDescribe what you want to accomplish in natural language, and Saarthi will:\n\n1. Select the appropriate tool based on your request\n2. Present the tool with its parameters for your review\n3. Execute the approved tool and show you the results\n4. Continue this process until your task is complete\n\n## Tool Categories\n\n| Category | Purpose | Tool Names |\n| :------- | :------ | :--------- |\n| Read | Access file content and code structure | `read_file`, `search_files`, `list_files`, `list_code_definition_names` |\n| Edit | Create or modify files and code | `write_to_file`, `apply_diff` |\n| Execute | Run commands and perform system operations | `execute_command` |\n| Browser | Interact with web content | `browser_action` |\n| Workflow | Manage task flow and context | `ask_followup_question`, `attempt_completion`, `switch_mode`, `new_task` |\n\n## Example: Using Tools\n\nHere's how a typical tool interaction works:\n\n![Tool approval interface showing Save and Reject buttons along with Auto-approve checkbox](/img/how-tools-work/how-tools-work.png)\n\n*The tool approval interface shows Save/Reject buttons and Auto-approve options.*\n\n**User:** Create a file named `greeting.js` that logs a greeting message\n\n**Saarthi:** (Proposes the `write_to_file` tool as shown in the image above)\n```xml\n\ngreeting.js\n\nfunction greet(name) {\n console.log(`Hello, ${name}!`);\n}\n\ngreet('World');\n\n5\n\n```\n\n**User:** (Clicks \"Save\" in the interface)\n\n**Saarthi:** (Confirms file creation)\n\n## Tool Safety and Approval\n\n\nEvery tool use requires your explicit approval. When Saarthi proposes a tool, you'll see:\n\n* A \"Save\" button to approve and execute the tool\n* A \"Reject\" button to decline the proposed tool\n* An optional \"Auto-approve\" setting for trusted operations\n\nThis safety mechanism ensures you maintain control over which files are modified, what commands are executed, and how your codebase is changed. Always review tool proposals carefully before saving them.\n\n## Core Tools Reference\n\n| Tool Name | Description | Category |\n| :-------- | :---------- | :------- |\n| `read_file` | Reads the content of a file with line numbers | Read |\n| `search_files` | Searches for text or regex patterns across files | Read |\n| `list_files` | Lists files and directories in a specified location | Read |\n| `list_code_definition_names` | Lists code definitions like classes and functions | Read |\n| `write_to_file` | Creates new files or overwrites existing ones | Edit |\n| `apply_diff` | Makes precise changes to specific parts of a file | Edit |\n| `execute_command` | Runs commands in the VS Code terminal | Execute |\n| `browser_action` | Performs actions in a headless browser | Browser |\n| `ask_followup_question` | Asks you a clarifying question | Workflow |\n| `attempt_completion` | Indicates the task is complete | Workflow |\n| `switch_mode` | Changes to a different operational mode | Workflow |\n| `new_task` | Creates a new subtask with a specific starting mode | Workflow |\n\n## Learn More About Tools\n\nFor more detailed information about each tool, including complete parameter references and advanced usage patterns, see the [Tool Use Overview](../advanced-usage/available-tools/tool-use-overview) documentation.\n" }, "docs/saarthi/basic-usage/the-chat-interface.md": { "content": "---\nsidebar_label: The Chat Interface\n---\n\n\n\n# The Chat Interface\n\nThe Saarthi chat interface is your primary way of interacting with it. It's located in the Saarthi panel, which you can open by clicking the Saarthi icon in the VS Code Activity Bar.\n\n## Components of the Chat Interface\n\nThe chat interface consists of the following main elements:\n\n1. **Chat History:** This area displays the conversation history between you and Saarthi. It shows your requests, Saarthi's responses, and any actions taken (like file edits or command executions).\n\n2. **Input Field:** This is where you type your tasks and questions for Saarthi. You can use plain English to communicate.\n\n3. **Action Buttons:** These buttons appear above the input field and allow you to approve or reject Saarthi's proposed actions. The available buttons change depending on the context.\n\n4. **Send Button:** This looks like a small plane and it's located to the far right of the input field. This sends messages to Saarthi after you've typed them.\n\n5. **Plus Button:** The plus button is located at the top in the header, and it resets the current session.\n\n6. **Settings Button:** The settings button is a gear, and it's used for opening the settings to customize features or behavior.\n\n7. **Mode Selector:** The mode selector is a dropdown located to the left of the chat input field. It is used for selecting which mode Saarthi should use for your tasks.\n\n*Numbered interface elements showing the key components of the Saarthi chat interface.*\n\n## Interacting with Messages\n\n* **Clickable Links:** File paths, URLs, and other mentions in the chat history are clickable. Clicking a file path will open the file in the editor. Clicking a URL will open it in your default browser.\n* **Copying Text:** You can copy text from the chat history by selecting it and using the standard copy command (Ctrl/Cmd + C). Some elements, like code blocks, have a dedicated \"Copy\" button.\n* **Expanding and Collapsing**: Click on a message to expand or collapse it.\n\n## Status Indicators\n\n* **Loading Spinner:** When Saarthi is processing a request, you'll see a loading spinner.\n* **Error Messages:** If an error occurs, a red error message will be displayed.\n* **Success Messages:** Green messages indicate successful completion of actions.\n" }, "docs/saarthi/basic-usage/typing-your-requests.md": { "content": "# Typing Your Requests\n\nSaarthi is designed to understand natural language. You don't need to use any special commands or syntax to communicate with it. Just type your request in plain English, as if you were talking to a human developer.\n\n![Example of typing a request in Saarthi](/img/typing-your-requests/naturally.gif)\n\n## Effective Request Strategies\n\nClearly state what you want Saarthi to do. Avoid vague or ambiguous language.\n\n| Strategy | Implementation |\n|----------|----------------|\n| **Be specific** | \"Fix the bug in `calculateTotal` that returns incorrect results\" instead of \"Fix the code\" |\n| **Provide context** | Use @ [Context Mentions](../basic-usage/context-mentions) for file and code references |\n| **Break down tasks** | Submit complex tasks in smaller manageable steps |\n| **Include examples** | Provide sample code when you need specific formatting or style |\n\n## Example Requests\n\n```\ncreate a new file named `utils.py` and add a function called `add` that takes two numbers as arguments and returns their sum\n```\n```\nin the file @src/components/Button.tsx, change the color of the button to blue\n```\n```\nfind all instances of the variable `oldValue` in @/src/App.js and replace them with `newValue`\n```\n```\nrun the command `npm install` in the terminal\n```\n```\nexplain the function `calculateTotal` in @/src/utils.ts\n```\n```\n@problems address all detected problems\n```\n\n## Common Pitfalls to Avoid\n\n| DON'T | DO |\n|-------|---------|\n| Vague requests | Specify exactly what needs to be done |\n| Assuming context | Explicitly reference files and functions |\n| Excessive technical jargon | Use clear, straightforward language |\n| Multiple unrelated tasks | Submit one focused request at a time |\n| Proceeding without confirmation | Check the code to make sure it's complete |" }, "docs/saarthi/faq.md": { "content": "\n\n# Frequently Asked Questions\n\nThis page answers some common questions about Saarthi.\n\n## General\n\n### What is Saarthi?\n\nSaarthi is an AI-powered autonomous coding agent that lives in your editor.\n\n### How does Saarthi work?\n\nSaarthi uses large language models (LLMs) to understand your requests and translate them into actions. It can:\n\n* Read and write files in your project.\n* Execute commands in your VS Code terminal.\n* Perform web browsing (if enabled).\n* Use external tools via the Model Context Protocol (MCP).\n\nYou interact with Saarthi through a chat interface, where you provide instructions and review/approve its proposed actions.\n\n### What can Saarthi do?\n\nSaarthi can help with a variety of coding tasks, including:\n\n* Generating code from natural language descriptions.\n* Refactoring existing code.\n* Fixing bugs.\n* Writing documentation.\n* Explaining code.\n* Answering questions about your codebase.\n* Automating repetitive tasks.\n* Creating new files and projects.\n\n### Is Saarthi free to use?\n\nThe Saarthi extension itself is free and open-source. However, Saarthi relies on external API providers (like [Anthropic](providers/anthropic), [OpenAI](providers/openai), [OpenRouter](providers/openrouter), [Requesty](providers/requesty), etc.) for its AI capabilities. These providers typically charge for API usage based on the number of tokens processed. You will need to create an account and obtain an API key from your chosen provider. See [Setting Up Your First AI Provider](./getting-started/connecting-api-provider) for details.\n\n### What are the risks of using Saarthi?\n\nSaarthi is a powerful tool, and it's important to use it responsibly. Here are some things to keep in mind:\n\n* **Saarthi can make mistakes.** Always review Saarthi's proposed changes carefully before approving them.\n* **Saarthi can execute commands.** Be very cautious about allowing Saarthi to run commands, especially if you're using auto-approval.\n* **Saarthi can access the internet.** If you're using a provider that supports web browsing, be aware that Saarthi could potentially access sensitive information.\n\n## Setup & Installation\n\n### How do I install Saarthi?\n\nSee the [Installation Guide](./getting-started/installing) for detailed instructions.\n\n### Which API providers are supported?\n\nSaarthi supports a wide range of API providers, including:\n* [Anthropic (Claude)](./providers/anthropic)\n* [OpenAI](./providers/openai)\n* [OpenRouter](./providers/openrouter)\n* [Google Gemini](./providers/gemini)\n* [Glama](./providers/glama)\n* [AWS Bedrock](./providers/bedrock)\n* [GCP Vertex AI](./providers/vertex)\n* [Ollama](./providers/ollama)\n* [LM Studio](./providers/lmstudio)\n* [DeepSeek](./providers/deepseek)\n* [Mistral](./providers/mistral)\n* [Unbound](./providers/unbound)\n* [Requesty](./providers/requesty)\n* [VS Code Language Model API](./providers/vscode-lm)\n\n### How do I get an API key?\nEach API provider has its own process for obtaining an API key. See the [Setting Up Your First AI Provider](./getting-started/connecting-api-provider) for links to the relevant documentation for each provider.\n\n### Can I use Saarthi with local models?\nYes, Saarthi supports running models locally using [Ollama](./providers/ollama) and [LM Studio](./providers/lmstudio). See [Using Local Models](./advanced-usage/local-models) for instructions.\n\n## Usage\n\n### How do I start a new task?\nOpen the Saarthi panel and type your task in the chat box. Be clear and specific about what you want Saarthi to do. See [Typing Your Requests](./basic-usage/typing-your-requests) for best practices.\n\n### What are modes in Saarthi?\n[Modes](./basic-usage/using-modes) are different personas that Saarthi can adopt, each with a specific focus and set of capabilities. The built-in modes are:\n\n* **Code:** For general-purpose coding tasks.\n* **Architect:** For planning and technical leadership.\n* **Ask:** For answering questions and providing information.\n* **Debug:** For systematic problem diagnosis.\nYou can also create [Custom Modes](./features/custom-modes).\n\n### How do I switch between modes?\n\nUse the dropdown menu in the chat input area to select a different mode, or use the `/` command to switch to a specific mode.\n\n### What are tools and how do I use them?\n[Tools](./basic-usage/how-tools-work) are how Saarthi interacts with your system. Saarthi automatically selects and uses the appropriate tools to complete your tasks. You don't need to call tools directly. You will be prompted to approve or reject each tool use.\n\n### What are context mentions?\n[Context mentions](./basic-usage/context-mentions) are a way to provide Saarthi with specific information about your project, such as files, folders, or problems. Use the \"@\" symbol followed by the item you want to mention (e.g., `@/src/file.ts`, `@problems`).\n\n### Can Saarthi access the internet?\n\nYes, if you are using a provider with a model that support web browsing. Be mindful of the security implications of allowing this.\n\n### Can Saarthi run commands in my terminal?\n\nYes, Saarthi can execute commands in your VS Code terminal. You will be prompted to approve each command before it's executed, unless you've enabled auto-approval for commands. Be extremely cautious about auto-approving commands. If you're experiencing issues with terminal commands, see the [Shell Integration Guide](./features/shell-integration) for troubleshooting.\n\n### How do I provide feedback to Saarthi?\n\nYou can provide feedback by approving or rejecting Saarthi's proposed actions. You can provide additional feedback by using the feedback field.\n\n### Can I customize Saarthi's behavior?\n\nYes, you can customize Saarthi in several ways:\n\n* **Custom Instructions:** Provide general instructions that apply to all modes, or mode-specific instructions.\n* **Custom Modes:** Create your own modes with tailored prompts and some tool permissions.\n* **`.saarthirules` Files:** Create `.saarthirules` files in your project to provide additional guidelines.\n* **Settings:** Adjust various settings, such as auto-approval, diff editing, and more.\n\n### Does Saarthi have any auto approval settings?\nYes, Saarthi has a few settings that when enabled will automatically approve actions. Find out more [here](./features/auto-approving-actions).\n\n## Advanced Features\n\n### Can I use saarthi offline?\nYes, if you use a [local model](./advanced-usage/local-models).\n\n### What is MCP (Model Context Protocol)?\n[MCP](./features/mcp/overview) is a protocol that allows Saarthi to communicate with external servers, extending its capabilities with custom tools and resources.\n\n### Can I create my own MCP servers?\n\nYes, you can create your own MCP servers to add custom functionality to Saarthi. See the [MCP documentation](https://github.com/modelcontextprotocol) for details.\n\n### What is Codebase Indexing?\n\n[Codebase Indexing](./features/experimental/codebase-indexing) is an experimental feature that creates a semantic search index of your project using AI embeddings. This enables Saarthi to better understand and navigate large codebases by finding relevant code based on meaning rather than just keywords.\n\n### How much does Codebase Indexing cost?\n\nCodebase Indexing requires an OpenAI API key for generating embeddings and a Qdrant vector database for storage. Costs depend on your project size and the embedding model used. Initial indexing is the most expensive part; subsequent updates are incremental and much cheaper.\n\n## Troubleshooting\n\n### Saarthi isn't responding. What should I do?\n\n* Make sure your API key is correct and hasn't expired.\n* Check your internet connection.\n* Check the status of your chosen API provider.\n* Try restarting VS Code.\n* If the problem persists, report the issue on [GitHub](https://github.com/godspeedsystems/saarthi)\n\n### I'm seeing an error message. What does it mean?\n\nThe error message should provide some information about the problem. \nIf you're unsure how to resolve it, seek help in [Discord]\n\n### Saarthi made changes I didn't want. How do I undo them?\n\nSaarthi uses VS Code's built-in file editing capabilities. You can use the standard \"Undo\" command (Ctrl/Cmd + Z) to revert changes. Also, if experimental checkpoints are enabled, Saarthi can revert changes made to a file.\n\n### Saarthi can't write to markdown files. What's wrong?\n\nIf Saarthi fails to write to `.md` files with errors like \"Failed to open diff editor\" or \"write_to_file tool failed\", this is typically caused by VS Code extensions or settings that interfere with file editing:\n\n**Common causes:**\n- Extensions with \"format on save\" functionality\n- VS Code settings that open markdown files in preview mode by default\n- The Markdown Preview extension or similar markdown processing extensions\n\n**Solutions:**\n- Disable any extensions that automatically format files on save\n- Remove these settings from your VS Code `settings.json`:\n ```json\n \"markdown.preview.openMarkdownLinks\": \"inPreview\",\n \"workbench.editorAssociations\": {\n \"*.md\": \"vscode.markdown.preview.editor\"\n }\n ```\n- Temporarily disable markdown-related extensions to test if they're causing the issue\n- Restart VS Code after making these changes\n\n### How do I report a bug or suggest a feature?\n\nPlease report bugs or suggest features on the Saarthi [Issues page](https://github.com/godspeedsystems/Saarthi/issues).\n" }, "docs/saarthi/features/api-configuration-profiles.mdx": { "content": "import Codicon from '@site/src/components/Codicon';\n\n# API Configuration Profiles\n\nAPI Configuration Profiles allow you to create and switch between different sets of AI settings. Each profile can have different configurations for each mode, letting you optimize your experience based on the task at hand.\n\n:::info\nHaving multiple configuration profiles lets you quickly switch between different AI providers, models, and settings without reconfiguring everything each time you want to change your setup.\n:::\n\n\n## How It Works\n\nConfiguration profiles can have their own:\n- API providers (OpenAI, Anthropic, OpenRouter, Glama, etc.)\n- API keys and authentication details\n- Model selections (o3-mini-high, Claude 3.7 Sonnet, DeepSeek R1, etc.)\n- [Temperature settings](model-temperature) for controlling response randomness\n- Thinking budgets\n- Provider-specific settings\n- [Diff editing configuration](fast-edits)\n- Rate limit settings\n\nNote that available settings vary by provider and model. Each provider offers different configuration options, and even within the same provider, different models may support different parameter ranges or features.\n\n## Creating and Managing Profiles\n\n### Creating a Profile\n\n1. Open Settings by clicking the gear icon → Providers\n2. Click the \"+\" button next to the profile selector\n![Profile selector with plus button](/img/api-configuration-profiles/api-configuration-profiles-1.png)\n\n3. Enter a name for your new profile\n![Creating a new profile dialog](/img/api-configuration-profiles/api-configuration-profiles.png)\n\n4. Configure the profile settings:\n - Select your API provider\n![Provider selection dropdown](/img/api-configuration-profiles/api-configuration-profiles-2.png)\n\n - Enter API key\n![API key entry field](/img/api-configuration-profiles/api-configuration-profiles-3.png)\n \n - Choose a model\n![Model selection interface](/img/api-configuration-profiles/api-configuration-profiles-8.png)\n \n - Configure the **Rate Limit** for this profile:\n - **Default is 0 (disabled), which is suitable for most users.** If needed, you can set a minimum time (in seconds) between API requests *for this profile* to help manage costs or avoid provider rate limits.\n - A value of 0 disables rate limiting (default).\n - Requests using other profiles follow their own rate limits.\n\n![Rate limit slider control within API profile settings](/img/api-configuration-profiles/api-configuration-profiles-12.png)\n \n - Adjust model parameters (like [temperature](model-temperature))\n \n### Switching Profiles\n\nSwitch profiles in two ways:\n1. From Settings panel: Select a different profile from the dropdown\n\n![Profile selection dropdown in Settings](/img/api-configuration-profiles/api-configuration-profiles-7.png)\n\n2. During chat: Access the API Configuration dropdown in the chat interface\n\n![API Configuration dropdown in chat interface](/img/api-configuration-profiles/api-configuration-profiles-6.png)\n\n### Pinning and Sorting Profiles\n\nThe API configuration dropdown now supports pinning your favorite profiles for quicker access:\n\n1. Hover over any profile in the dropdown to reveal the pin icon\n2. Click the pin icon to add the profile to your pinned list\n3. Pinned profiles appear at the top of the dropdown, sorted alphabetically\n4. Unpinned profiles appear below a separator, also sorted alphabetically\n5. You can unpin a profile by clicking the same icon again\n\n![Pinning API configuration profiles](/img/api-configuration-profiles/api-configuration-profiles-4.png)\n\nThis feature makes it easier to navigate between commonly used profiles, especially when you have many configurations.\n\n\n### Editing and Deleting Profiles\n\n![Profile editing interface](/img/api-configuration-profiles/api-configuration-profiles-10.png)\n\n- Select the profile in Settings to modify any settings\n- Click the pencil icon to rename a profile\n- Click the trash icon to delete a profile (you cannot delete the only remaining profile)\n\n## Linking Profiles to Modes\nIn the Prompts tab, you can explicitly associate a specific Configuration Profile with each Mode. The system also automatically remembers which profile you last used with each mode, making your workflow more efficient.\n\n## Security Note\n\nAPI keys are stored securely in VSCode's Secret Storage and are never exposed in plain text.\n\n## Related Features\n\n- Works with [custom modes](custom-modes) you create\n- Integrates with [local models](../advanced-usage/local-models) for offline work\n- Supports [temperature settings](model-temperature) per mode\n- Supports per-profile rate limits (configured here) and general [usage tracking/cost info](../advanced-usage/rate-limits-costs)\n- Supports [Diff/Fast Edits](fast-edits) for tailored code editing behavior.\n" }, "docs/saarthi/features/auto-approving-actions.mdx": { "content": "import Codicon from '@site/src/components/Codicon';\n\n# Auto-Approving Actions\n\n> ⚠️ **SECURITY WARNING:** Auto-approve settings bypass confirmation prompts, giving Saarthi direct access to your system. This can result in **data loss, file corruption, or worse**. Command line access is particularly dangerous, as it can potentially execute harmful operations that could damage your system or compromise security. Only enable auto-approval for actions you fully trust.\n\nAuto-approve settings speed up your workflow by eliminating repetitive confirmation prompts, but they significantly increase security risks.\n\n\n## Quick Start Guide\n\n1. Click the Auto-Approve Toolbar above the chat input\n2. Select which actions Saarthi can perform without asking permission\n3. Use the master toggle (leftmost checkbox) to quickly enable/disable all permissions\n\n## Auto-Approve Toolbar\n\n![Auto-approve toolbar collapsed state](/img/auto-approving-actions/auto-approving-actions.png)\n\n*Prompt box and Auto-Approve Toolbar showing enabled permissions*\n\nClick the toolbar to expand it and configure individual permissions:\n\n![Auto-approve toolbar expanded state](/img/auto-approving-actions/auto-approving-actions-1.png)\n\n*Prompt text box and Expanded toolbar with all options*\n\n### Available Permissions\n\n| Permission | What it does | Risk level |\n|------------|--------------|------------|\n| **Read files and directories** | Lets Saarthi access files without asking | Medium |\n| **Edit files** | Lets Saarthi modify files without asking | **High** |\n| **Execute approved commands** | Runs whitelisted terminal commands automatically | **High** |\n| **Use the browser** | Allows headless browser interaction | Medium |\n| **Use MCP servers** | Lets Saarthi use configured MCP services | Medium-High |\n| **Switch modes** | Changes between Saarthi modes automatically | Low |\n| **Create & complete subtasks** | Manages subtasks without confirmation | Low |\n| **Retry failed requests** | Automatically retries failed API requests | Low |\n\n## Master Toggle for Quick Control\n\nThe leftmost checkbox works as a master toggle:\n\n![Master toggle in Auto-approve toolbar](/img/auto-approving-actions/auto-approving-actions-14.png)\n\n*Master toggle (checkbox) controls all auto-approve permissions at once*\n\nUse the master toggle when:\n- Working in sensitive code (turn off)\n- Doing rapid development (turn on)\n- Switching between exploration and editing tasks\n\n## Advanced Settings Panel\n\nThe settings panel provides detailed control with important security context:\n\n> **Allow Saarthi to automatically perform operations without requiring approval. Enable these settings only if you fully trust the AI and understand the associated security risks.**\n\nTo access these settings:\n\n1. Click in the top-right corner\n2. Navigate to Auto-Approve Settings\n\n*Complete settings panel view*\n\n### Read Operations\n\n:::caution Read Operations\n\n**Setting:** \"Always approve read-only operations\"\n\n**Description:** \"When enabled, Saarthi will automatically view directory contents and read files without requiring you to click the Approve button.\"\n\n**Risk level:** Medium\n\nWhile this setting only allows reading files (not modifying them), it could potentially expose sensitive data. Still recommended as a starting point for most users, but be mindful of what files Saarthi can access.\n:::\n\n### Write Operations\n\n:::caution Write Operations\n![Write operations setting with delay slider](/img/auto-approving-actions/auto-approving-actions-7.png)\n\n**Setting:** \"Always approve write operations\"\n\n**Description:** \"Automatically create and edit files without requiring approval\"\n\n**Delay slider:** \"Delay after writes to allow diagnostics to detect potential problems\" (Default: 1000ms)\n\n**Risk level:** High\n\nThis setting allows Saarthi to modify your files without confirmation. The delay timer is crucial:\n- Higher values (2000ms+): Recommended for complex projects where diagnostics take longer\n- Default (1000ms): Suitable for most projects\n- Lower values: Use only when speed is critical and you're in a controlled environment\n- Zero: No delay for diagnostics (not recommended for critical code)\n\n#### Write Delay & Problems Pane Integration\n\n![VSCode Problems pane showing diagnostic information](/img/auto-approving-actions/auto-approving-actions-5.png)\n\n*VSCode Problems pane that Saarthi checks during the write delay*\n\nWhen you enable auto-approval for writing files, the delay timer works with VSCode's Problems pane:\n\n1. Saarthi makes a change to your file\n2. VSCode's diagnostic tools analyze the change\n3. The Problems pane updates with any errors or warnings\n4. Saarthi notices these issues before continuing\n\nThis works like a human developer pausing to check for errors after changing code. You can adjust the delay time based on:\n\n- Project complexity\n- Language server speed\n- How important error detection is for your workflow\n:::\n\n### Browser Actions\n\n:::info Browser Actions\n![Browser actions setting](/img/auto-approving-actions/auto-approving-actions-8.png)\n\n**Setting:** \"Always approve browser actions\"\n\n**Description:** \"Automatically perform browser actions without requiring approval\"\n\n**Note:** \"Only applies when the model supports computer use\"\n\n**Risk level:** Medium\n\nAllows Saarthi to control a headless browser without confirmation. This can include:\n- Opening websites\n- Navigating pages\n- Interacting with web elements\n\nConsider the security implications of allowing automated browser access.\n:::\n\n### API Requests\n\n:::info API Requests\n![API requests retry setting with delay slider](/img/auto-approving-actions/auto-approving-actions-9.png)\n\n**Setting:** \"Always retry failed API requests\"\n\n**Description:** \"Automatically retry failed API requests when server returns an error response\"\n\n**Delay slider:** \"Delay before retrying the request\" (Default: 5s)\n\n**Risk level:** Low\n\nThis setting automatically retries API calls when they fail. The delay controls how long Saarthi waits before trying again:\n- Longer delays are gentler on API rate limits\n- Shorter delays give faster recovery from transient errors\n:::\n\n### MCP Tools\n\n:::caution MCP Tools\n![MCP tools setting](/img/auto-approving-actions/auto-approving-actions-10.png)\n\n**Setting:** \"Always approve MCP tools\"\n\n**Description:** \"Enable auto-approval of individual MCP tools in the MCP Servers view (requires both this setting and the tool's individual 'Always allow' checkbox)\"\n\n**Risk level:** Medium-High (depends on configured MCP tools)\n\nThis setting works in conjunction with individual tool permissions in the MCP Servers view. Both this global setting and the tool-specific permission must be enabled for auto-approval.\n:::\n\n### Mode Switching\n\n:::info Mode Switching\n![Mode switching setting](/img/auto-approving-actions/auto-approving-actions-11.png)\n\n**Setting:** \"Always approve mode switching\"\n\n**Description:** \"Automatically switch between different modes without requiring approval\"\n\n**Risk level:** Low\n\nAllows Saarthi to change between different modes (Code, Architect, etc.) without asking for permission. This primarily affects the AI's behavior rather than system access.\n:::\n\n### Subtasks\n\n:::info Subtasks\n![Subtasks setting](/img/auto-approving-actions/auto-approving-actions-12.png)\n\n**Setting:** \"Always approve creation & completion of subtasks\"\n\n**Description:** \"Allow creation and completion of subtasks without requiring approval\"\n\n**Risk level:** Low\n\nEnables Saarthi to create and complete subtasks automatically. This relates to workflow organization rather than system access.\n:::\n\n### Command Execution\n\n:::caution Command Execution\n![Command execution setting with whitelist interface](/img/auto-approving-actions/auto-approving-actions-13.png)\n\n**Setting:** \"Always approve allowed execute operations\"\n\n**Description:** \"Automatically execute allowed terminal commands without requiring approval\"\n\n**Command management:** \"Command prefixes that can be auto-executed when 'Always approve execute operations' is enabled. Add * to allow all commands (use with caution).\"\n\n**Risk level:** High\n\nThis setting allows terminal command execution with controls. While risky, the whitelist feature limits what commands can run. Important security features:\n\n- Whitelist specific command prefixes (recommended)\n- Never use * wildcard in production or with sensitive data\n- Consider security implications of each allowed command\n- Always verify commands that interact with external systems\n\n**Interface elements:**\n- Text field to enter command prefixes (e.g., 'git')\n- \"Add\" button to add new prefixes\n- Clickable command buttons with X to remove them\n:::\n" }, "docs/saarthi/features/boomerang-tasks.mdx": { "content": "---\nsidebar_label: 'Boomerang Tasks'\n---\n\n# Boomerang Tasks: Orchestrate Complex Workflows\n\nBoomerang Tasks (also known as subtasks or task orchestration) allow you to break down complex projects into smaller, manageable pieces using the built-in **`🪃 Orchestrator` Mode (aka Boomerang Mode)**. Think of it like delegating parts of your work to specialized assistants. Each subtask runs in its own context, often using a different Saarthi mode tailored for that specific job (like [`💻 Code`](/basic-usage/using-modes#code-mode-default), [`🏗️ Architect`](/basic-usage/using-modes#architect-mode), or [`🪲 Debug`](/basic-usage/using-modes#debug-mode)). The Orchestrator mode manages this process.\n\n:::info Orchestrator Mode is Built-In\nThe `🪃 Orchestrator` mode (previously achieved via a custom \"Boomerang Mode\") is now a built-in mode specifically designed to orchestrate workflows by breaking down tasks and delegating them to other modes. You no longer need to create a custom mode for this functionality.\n\nLearn more about [Built-in Modes](/basic-usage/using-modes#built-in-modes).\n:::\n\n## Why Use Boomerang Tasks?\n\n- **Tackle Complexity:** Break large, multi-step projects (e.g., building a full feature) into focused subtasks (e.g., design, implementation, documentation).\n- **Use Specialized Modes:** Automatically delegate subtasks to the mode best suited for that specific piece of work, leveraging specialized capabilities for optimal results.\n- **Maintain Focus & Efficiency:** Each subtask operates in its own isolated context with a separate conversation history. This prevents the parent (orchestrator) task from becoming cluttered with the detailed execution steps (like code diffs or file analysis results), allowing it to focus efficiently on the high-level workflow and manage the overall process based on concise summaries from completed subtasks.\n- **Streamline Workflows:** Results from one subtask can be automatically passed to the next, creating a smooth flow (e.g., architectural decisions feeding into the coding task).\n\n## How It Works\n\n1. When in the [`🪃 Orchestrator`](/basic-usage/using-modes#orchestrator-mode-aka-boomerang-mode) mode (aka Boomerang Mode), Saarthi analyzes a complex task and suggests breaking it down into a subtask[^1].\n\n2. The parent task (in Orchestrator mode) pauses, and the new subtask begins in a different, specialized mode[^2].\n3. When the subtask's goal is achieved, Saarthi signals completion.\n4. The parent task resumes with only the summary[^3] of the subtask. The parent uses this summary to continue the main workflow.\n\n## Key Considerations\n\n- **Approval Required:** By default, you must approve the creation and completion of each subtask. This can be automated via the [Auto-Approving Actions](/features/auto-approving-actions#subtasks) settings if desired.\n- **Context Isolation and Transfer:** Each subtask operates in complete isolation with its own conversation history. It does not automatically inherit the parent's context. Information must be explicitly passed:\n * **Down:** Via the initial instructions provided when the subtask is created.\n * **Up:** Via the final summary provided when the subtask finishes. Be mindful that only this summary returns to the parent.\n- **Navigation:** Saarthi's interface helps you see the hierarchy of tasks (which task is the parent, which are children). You can typically navigate between active and paused tasks.\n\nBoomerang Tasks provide a powerful way to manage complex development workflows directly within Saarthi, leveraging specialized modes for maximum efficiency.\n\n:::tip Keep Tasks Focused\nUse subtasks (delegated via Orchestrator mode) to maintain clarity. If a request significantly shifts focus or requires a different expertise (mode), consider creating a subtask rather than overloading the current one.\n:::\n\n## Frequently Asked Questions\n\n### Why can't Orchestrator mode read files, write files, call MCPs, or run commands?\n\nThe Orchestrator mode is intentionally limited to focus on high-level workflow management. Giving it the ability to read files by default causes the context to become filled with file reads, hampering its ability to remain focused on orchestration. The design philosophy is that subtasks should handle the detailed work and return only the necessary information (via their completion summaries) for the orchestrator to delegate further tasks effectively.\n\nThis limitation helps prevent [context poisoning](/advanced-usage/context-poisoning), where irrelevant or excessive information contaminates the model's active context, leading to degraded performance and task deviation.\n\n### How can I override Orchestrator mode's limitations?\n\nYou can [customize the Orchestrator mode](/features/custom-modes) to add capabilities like file reading by following the [configuration precedence](/features/custom-modes#configuration-precedence) system:\n\n1. Open the Command Palette and select **\"Edit Global Modes\"**\n2. Copy and paste this configuration:\n\n```json\n{\n \"customModes\": [\n {\n \"slug\": \"orchestrator\",\n \"name\": \"🪃 Orchestrator\",\n \"roleDefinition\": \"You are Saarthi, a strategic workflow orchestrator who coordinates complex tasks by delegating them to appropriate specialized modes. You have a comprehensive understanding of each mode's capabilities and limitations, allowing you to effectively break down complex problems into discrete tasks that can be solved by different specialists.\",\n \"customInstructions\": \"Your role is to coordinate complex workflows by delegating tasks to specialized modes. As an orchestrator, you should:\\n\\n1. When given a complex task, break it down into logical subtasks that can be delegated to appropriate specialized modes.\\n\\n2. For each subtask, use the `new_task` tool to delegate. Choose the most appropriate mode for the subtask's specific goal and provide comprehensive instructions in the `message` parameter. These instructions must include:\\n * All necessary context from the parent task or previous subtasks required to complete the work.\\n * A clearly defined scope, specifying exactly what the subtask should accomplish.\\n * An explicit statement that the subtask should *only* perform the work outlined in these instructions and not deviate.\\n * An instruction for the subtask to signal completion by using the `attempt_completion` tool, providing a concise yet thorough summary of the outcome in the `result` parameter, keeping in mind that this summary will be the source of truth used to keep track of what was completed on this project. \\n * A statement that these specific instructions supersede any conflicting general instructions the subtask's mode might have.\\n\\n3. Track and manage the progress of all subtasks. When a subtask is completed, analyze its results and determine the next steps.\\n\\n4. Help the user understand how the different subtasks fit together in the overall workflow. Provide clear reasoning about why you're delegating specific tasks to specific modes.\\n\\n5. When all subtasks are completed, synthesize the results and provide a comprehensive overview of what was accomplished.\\n\\n6. Ask clarifying questions when necessary to better understand how to break down complex tasks effectively.\\n\\n7. Suggest improvements to the workflow based on the results of completed subtasks.\\n\\nUse subtasks to maintain clarity. If a request significantly shifts focus or requires a different expertise (mode), consider creating a subtask rather than overloading the current one.\",\n \"groups\": [\n \"read\"\n ],\n \"source\": \"global\"\n }\n ]\n}\n```\n\n3. Save the file. Your global Orchestrator mode will now have read capabilities.\n\n:::warning\nAdding capabilities to the Orchestrator mode should be done thoughtfully. The limited default capabilities help maintain focus on orchestration rather than implementation details.\n:::\n\n\n[^1]: This context is passed via the `message` parameter of the [`new_task`](/advanced-usage/available-tools/new-task) tool when the Orchestrator mode delegates the task.\n[^2]: The mode for the subtask is specified via the `mode` parameter of the [`new_task`](/advanced-usage/available-tools/new-task) tool during initiation by the Orchestrator mode.\n[^3]: This summary is passed via the `result` parameter of the [`attempt_completion`](/advanced-usage/available-tools/attempt-completion) tool when the subtask finishes." }, "docs/saarthi/features/browser-use.mdx": { "content": "import Codicon from '@site/src/components/Codicon';\n\n# Browser Use\n\nSaarthi provides sophisticated browser automation capabilities that let you interact with websites directly from VS Code. This feature enables testing web applications, automating browser tasks, and capturing screenshots without leaving your development environment.\n\n\n:::caution Model Support Required\nBrowser Use within Saarthi requires the use of Claude Sonnet 3.5 or 3.7\n:::\n\n## How Browser Use Works\n\nBy default, Saarthi uses a built-in browser that:\n- Launches automatically when you ask Saarthi to visit a website\n- Captures screenshots of web pages\n- Allows Saarthi to interact with web elements\n- Runs invisibly in the background\n\nAll of this happens directly within VS Code, with no setup required.\n\n## Using Browser Use\n\nA typical browser interaction follows this pattern:\n\n**Important:** Browser Use requires Claude Sonnet 3.5 or 3.7 model.\n\n1. Ask Saarthi to visit a website\n2. Saarthi launches the browser and shows you a screenshot\n3. Request additional actions (clicking, typing, scrolling)\n4. Saarthi closes the browser when finished\n\nFor example:\n\n```\nOpen the browser and view our site.\n```\n\n```\nCan you check if my website at https://saarthi.com is displaying correctly?\n```\n\n```\nBrowse http://localhost:3000, scroll down to the bottom of the page and check if the footer information is displaying correctly.\n```\n\n## How Browser Actions Work\n\nThe browser_action tool controls a browser instance that returns screenshots and console logs after each action, allowing you to see the results of interactions.\n\nKey characteristics:\n- Each browser session must start with `launch` and end with `close`\n- Only one browser action can be used per message\n- While the browser is active, no other tools can be used\n- You must wait for the response (screenshot and logs) before performing the next action\n\n### Available Browser Actions\n\n| Action | Description | When to Use |\n|--------|-------------|------------|\n| `launch` | Opens a browser at a URL | Starting a new browser session |\n| `click` | Clicks at specific coordinates | Interacting with buttons, links, etc. |\n| `type` | Types text into active element | Filling forms, search boxes |\n| `scroll_down` | Scrolls down by one page | Viewing content below the fold |\n| `scroll_up` | Scrolls up by one page | Returning to previous content |\n| `close` | Closes the browser | Ending a browser session |\n\n## Browser Use Configuration/Settings\n\n:::info Default Browser Settings\n- **Enable browser tool**: Enabled\n- **Viewport size**: Small Desktop (900x600)\n- **Screenshot quality**: 75%\n- **Use remote browser connection**: Disabled\n:::\n\n### Accessing Settings\n\nTo change Browser / Computer Use settings in Saarthi:\n\n1. Open Settings by clicking the gear icon → Browser / Computer Use\n\n### Enable/Disable Browser Use\n\n**Purpose**: Master toggle that enables Saarthi to interact with websites using a Puppeteer-controlled browser.\n\nTo change this setting:\n1. Check or uncheck the \"Enable browser tool\" checkbox within your Browser / Computer Use settings\n\n![Enable browser tool setting](/img/browser-use/browser-use-2.png)\n\n### Viewport Size\n\n**Purpose**: Determines the resolution of the browser session Saarthi uses.\n\n**Tradeoff**: Higher values provide a larger viewport but increase token usage.\n\nTo change this setting:\n1. Click the dropdown menu under \"Viewport size\" within your Browser / Computer Use settings\n2. Select one of the available options:\n - Large Desktop (1280x800)\n - Small Desktop (900x600) - Default\n - Tablet (768x1024)\n - Mobile (360x640)\n2. Select your desired resolution.\n\n![Viewport size setting](/img/browser-use/browser-use-3.png)\n\n### Screenshot Quality\n\n**Purpose**: Controls the WebP compression quality of browser screenshots.\n\n**Tradeoff**: Higher values provide clearer screenshots but increase token usage.\n\nTo change this setting:\n1. Adjust the slider under \"Screenshot quality\" within your Browser / Computer Use settings\n2. Set a value between 1-100% (default is 75%)\n3. Higher values provide clearer screenshots but increase token usage:\n - 40-50%: Good for basic text-based websites\n - 60-70%: Balanced for most general browsing\n - 80%+: Use when fine visual details are critical\n\n![Screenshot quality setting](/img/browser-use/browser-use-4.png)\n\n### Remote Browser Connection\n\n**Purpose**: Connect Saarthi to an existing Chrome browser instead of using the built-in browser.\n\n**Benefits**:\n- Works in containerized environments and remote development workflows\n- Maintains authenticated sessions between browser uses\n- Eliminates repetitive login steps\n- Allows use of custom browser profiles with specific extensions\n\n**Requirements**: Chrome must be running with remote debugging enabled.\n\nTo enable this feature:\n1. Check the \"Use remote browser connection\" box in Browser / Computer Use settings\n2. Click \"Test Connection\" to verify\n\n![Remote browser connection setting](/img/browser-use/browser-use-5.png)\n\n#### Common Use Cases\n\n- **DevContainers**: Connect from containerized VS Code to host Chrome browser\n- **Remote Development**: Use local Chrome with remote VS Code server\n- **Custom Chrome Profiles**: Use profiles with specific extensions and settings\n\n#### Connecting to a Visible Chrome Window\n\nConnect to a visible Chrome window to observe Saarthi's interactions in real-time:\n\n**macOS**\n```bash\n/Applications/Google\\ Chrome.app/Contents/MacOS/Google\\ Chrome --remote-debugging-port=9222 --user-data-dir=/tmp/chrome-debug --no-first-run\n```\n\n**Windows**\n```bash\n\"C:\\Program Files\\Google\\Chrome\\Application\\chrome.exe\" --remote-debugging-port=9222 --user-data-dir=C:\\chrome-debug --no-first-run\n```\n\n**Linux**\n```bash\ngoogle-chrome --remote-debugging-port=9222 --user-data-dir=/tmp/chrome-debug --no-first-run\n```\n" }, "docs/saarthi/features/checkpoints.mdx": { "content": "import Codicon from '@site/src/components/Codicon';\n\n# Checkpoints\n\nCheckpoints automatically version your workspace files during Saarthi tasks, enabling non-destructive exploration of AI suggestions and easy recovery from unwanted changes.\n\nCheckpoints let you:\n- Safely experiment with AI-suggested changes\n- Easily recover from undesired modifications\n- Compare different implementation approaches\n- Revert to previous project states without losing work\n\n\n:::info Important Notes\n- **Checkpoints are enabled by default.**\n- **Git must be installed** for checkpoints to function - [see installation instructions](#git-installation)\n- No GitHub account or repository is required\n- No Git personal information configuration is needed\n- The shadow Git repository operates independently from your project's existing Git configuration\n:::\n\n## Configuration Options\n\nAccess checkpoint settings in Saarthi settings under the \"Checkpoints\" section:\n\n1. Open Settings by clicking the gear icon → Checkpoints\n2. Check or uncheck the \"Enable automatic checkpoints\" checkbox\n\n## How Checkpoints Work\n\nSaarthi captures snapshots of your project's state using a shadow Git repository, separate from your main version control system. These snapshots, called checkpoints, automatically record changes throughout your AI-assisted workflow—whenever tasks begin, files change, or commands run.\n\nCheckpoints are stored as Git commits in the shadow repository, capturing:\n\n- File content changes\n- New files added\n- Deleted files\n- Renamed files\n- Binary file changes\n\n## Working with Checkpoints\n\nCheckpoints are integrated directly into your workflow through the chat interface.\n\nCheckpoints appear directly in your chat history in two forms:\n\n- **Initial checkpoint** marks your starting project state\n![Initial checkpoint indicator in chat](/img/checkpoints/checkpoints-1.png)\n\n- **Regular checkpoints** appear after file modifications or command execution\n![Regular checkpoint indicator in chat](/img/checkpoints/checkpoints-2.png)\n\nEach checkpoint provides two primary functions:\n\n### Viewing Differences\n\nTo compare your current workspace with a previous checkpoint:\n\n1. Locate the checkpoint in your chat history\n2. Click the checkpoint's `View Differences` button\n\n![View Differences button interface](/img/checkpoints/checkpoints-6.png)\n\n3. Review the differences in the comparison view:\n - Added lines are highlighted in green\n - Removed lines are highlighted in red\n - Modified files are listed with detailed changes\n - Renamed and moved files are tracked with their path changes\n - New or deleted files are clearly marked\n\n![View differences option for checkpoints](/img/checkpoints/checkpoints-3.png)\n\n### Restoring Checkpoints\n\nTo restore a project to a previous checkpoint state:\n\n1. Locate the checkpoint in your chat history\n2. Click the checkpoint's `Restore Checkpoint` button\n![Restore checkpoint button interface](/img/checkpoints/checkpoints-7.png)\n3. Choose one of these restoration options:\n \n![Restore checkpoint option](/img/checkpoints/checkpoints-4.png)\n\n - **Restore Files Only** - Reverts only workspace files to checkpoint state without modifying conversation history. Ideal for comparing alternative implementations while maintaining chat context, allowing you to seamlessly switch between different project states. This option does not require confirmation and lets you quickly switch between different implementations.\n \n - **Restore Files & Task** - Reverts both workspace files AND removes all subsequent conversation messages. Use when you want to completely reset both your code and conversation back to the checkpoint's point in time. This option requires confirmation in a dialog as it cannot be undone.\n\n![Confirmation dialog for restoring checkpoint with files & task](/img/checkpoints/checkpoints-9.png)\n\n### Limitations and Considerations\n\n- **Scope**: Checkpoints only capture changes made during active Saarthi tasks\n- **External changes**: Modifications made outside of tasks (manual edits, other tools) aren't included\n- **Large files**: Very large binary files may impact performance\n- **Unsaved work**: Restoration will overwrite any unsaved changes in your workspace\n\n## Technical Implementation\n\n### Checkpoint Architecture\n\nThe checkpoint system consists of:\n\n1. **Shadow Git Repository**: A separate Git repository created specifically for checkpoint tracking that functions as the persistent storage mechanism for checkpoint state.\n\n2. **Checkpoint Service**: Handles Git operations and state management through:\n - Repository initialization\n - Checkpoint creation and storage\n - Diff computation\n - State restoration\n\n3. **UI Components**: Interface elements displayed in the chat that enable interaction with checkpoints.\n\n### Restoration Process\n\nWhen restoration executes, Saarthi:\n- Performs a hard reset to the specified checkpoint commit\n- Copies all files from the shadow repository to your workspace\n- Updates internal checkpoint tracking state\n\n### Storage Type\n\nCheckpoints are task-scoped, meaning they are specific to a single task.\n\n### Diff Computation\n\nCheckpoint comparison uses Git's underlying diff capabilities to produce structured file differences:\n- Modified files show line-by-line changes\n- Binary files are properly detected and handled\n- Renamed and moved files are tracked correctly\n- File creation and deletion are clearly identified\n\n### File Exclusion and Ignore Patterns\n\nThe checkpoint system uses intelligent file exclusion to track only relevant files:\n\n#### Built-in Exclusions\n\nThe system has comprehensive built-in exclusion patterns that automatically ignore:\n- Build artifacts and dependency directories (`node_modules/`, `dist/`, `build/`)\n- Media files and binary assets (images, videos, audio)\n- Cache and temporary files (`.cache/`, `.tmp/`, `.bak`)\n- Configuration files with sensitive information (`.env`)\n- Large data files (archives, executables, binaries)\n- Database files and logs\n\nThese patterns are written to the shadow repository's `.git/info/exclude` file during initialization.\n\n#### .gitignore Support\n\nThe checkpoint system respects `.gitignore` patterns in your workspace:\n- Files excluded by `.gitignore` won't trigger checkpoint creation\n- Excluded files won't appear in checkpoint diffs\n- Standard Git ignore rules apply when staging file changes\n\n#### .saarthiignore Behavior\n\nThe `.saarthiignore` file (which controls AI access to files) is separate from checkpoint tracking:\n- Files excluded by `.saarthiignore` but not by `.gitignore` will still be checkpointed\n- Changes to AI-inaccessible files can still be restored through checkpoints\n\nThis separation is intentional, as `.saarthiignore` limits which files the AI can access, not which files should be tracked for version history.\n\n#### Nested Git Repositories\n\nThe checkpoint system includes special handling for nested Git repositories:\n- Temporarily renames nested `.git` directories to `.git_disabled` during operations\n- Restores them after operations complete\n- Allows proper tracking of files in nested repositories\n- Ensures nested repositories remain functional and unaffected\n\n### Concurrency Control\n\nOperations are queued to prevent concurrent Git operations that might corrupt repository state. This ensures that rapid checkpoint operations complete safely even when requested in quick succession.\n\n## Git Installation\n\nCheckpoints require Git to be installed on your system. The implementation uses the `simple-git` library, which relies on Git command-line tools to create and manage shadow repositories.\n\n### macOS\n\n1. **Install with Homebrew (recommended)**:\n ```\n brew install git\n ```\n\n2. **Alternative: Install with Xcode Command Line Tools**:\n ```\n xcode-select --install\n ```\n\n3. **Verify installation**:\n - Open Terminal\n - Type `git --version`\n - You should see a version number like `git version 2.40.0`\n\n### Windows\n\n1. **Download Git for Windows**:\n - Visit https://git-scm.com/download/win\n - The download should start automatically\n\n2. **Run the installer**:\n - Accept the license agreement\n - Choose installation location (default is recommended)\n - Select components (default options are typically sufficient)\n - Choose the default editor\n - Choose how to use Git from the command line (recommended: Git from the command line and also from 3rd-party software)\n - Configure line ending conversions (recommended: Checkout Windows-style, commit Unix-style)\n - Complete the installation\n\n3. **Verify installation**:\n - Open Command Prompt or PowerShell\n - Type `git --version`\n - You should see a version number like `git version 2.40.0.windows.1`\n\n### Linux\n\n**Debian/Ubuntu**:\n```\nsudo apt update\nsudo apt install git\n```\n\n**Fedora**:\n```\nsudo dnf install git\n```\n\n**Arch Linux**:\n```\nsudo pacman -S git\n```\n\n**Verify installation**:\n- Open Terminal\n- Type `git --version`\n- You should see a version number\n" }, "docs/saarthi/features/code-actions.md": { "content": "import Codicon from '@site/src/components/Codicon';\n\n# Code Actions\n\nCode Actions are a powerful feature of VS Code that provide quick fixes, refactorings, and other code-related suggestions directly within the editor. Saarthi integrates with this system to offer AI-powered assistance for common coding tasks.\n\n## What are Code Actions?\n\nCode Actions appear as a lightbulb icon (💡) in the editor gutter (the area to the left of the line numbers). They can also be accessed via the right-click context menu, or via keyboard shortcut. They are triggered when:\n\n* You select a range of code.\n* Your cursor is on a line with a problem (error, warning, or hint).\n* You invoke them via command.\n\nClicking the lightbulb, right-clicking and selecting \"Saarthi\", or using the keyboard shortcut (`Ctrl+.` or `Cmd+.` on macOS, by default), displays a menu of available actions.\n\n## Saarthi's Code Actions\n\nSaarthi provides the following Code Actions:\n\n* **Add to Context:** Quickly adds the selected code to your chat with Saarthi, including the filename and line numbers so Saarthi knows exactly where the code is from. It's listed first in the menu for easy access.\n* **Explain Code:** Asks Saarthi to explain the selected code.\n* **Improve Code:** Asks Saarthi to suggest improvements to the selected code.\n\n### Add to Context Deep Dive\n\nThe **Add to Context** action is listed first in the Code Actions menu so you can quickly add code snippets to your conversation. When you use it, Saarthi includes the filename and line numbers along with the code.\n\nThis helps Saarthi understand the exact context of your code within the project, allowing it to provide more relevant and accurate assistance.\n\n**Example Chat Input:**\n\n```\nCan you explain this function?\n@myFile.js:15:25\n```\n\n*(Where `@myFile.js:15:25` represents the code added via \"Add to Context\")*\n\n## Using Code Actions\n\nThere are three main ways to use Saarthi's Code Actions:\n\n### 1. From the Lightbulb (💡)\n\n1. **Select Code:** Select the code you want to work with. You can select a single line, multiple lines, or an entire block of code.\n2. **Look for the Lightbulb:** A lightbulb icon will appear in the gutter next to the selected code (or the line with the error/warning).\n3. **Click the Lightbulb:** Click the lightbulb icon to open the Code Actions menu.\n4. **Choose an Action:** Select the desired Saarthi action from the menu.\n5. **Review and Approve:** Saarthi will propose a solution in the chat panel. Review the proposed changes and approve or reject them.\n\n### 2. From the Right-Click Context Menu\n\n1. **Select Code:** Select the code you want to work with.\n2. **Right-Click:** Right-click on the selected code to open the context menu.\n3. **Choose \"Saarthi\":** Select the \"Saarthi\" option from the context menu. A submenu will appear with the available Saarthi actions.\n4. **Choose an Action:** Select the desired action from the submenu.\n5. **Review and Approve:** Saarthi will propose a solution in the chat panel. Review the proposed changes and approve or reject them.\n\n### 3. From the Command Palette\n\n1. **Select Code:** Select the code you want to work with.\n2. **Open the Command Palette:** Press `Ctrl+Shift+P` (Windows/Linux) or `Cmd+Shift+P` (macOS).\n3. **Type a Command:** Type \"Saarthi\" to filter the commands, then choose the relevant code action (e.g., \"Saarthi: Explain Code\"). The action will apply in the most logical context (usually the current active chat task, if one exists).\n4. **Review and Approve:** Saarthi will propose a solution in the chat panel. Review the proposed changes and approve or reject them.\n\n## Customizing Code Action Prompts\n\nYou can customize the prompts used for each Code Action by modifying the \"Support Prompts\" in the **Prompts** tab. This allows you to fine-tune the instructions given to the AI model and tailor the responses to your specific needs.\n\n1. **Open the Prompts Tab:** Click the icon in the Saarthi top menu bar.\n2. **Find \"Support Prompts\":** You will see the support prompts, including \"Enhance Prompt\", \"Explain Code\", and \"Improve Code\".\n3. **Edit the Prompts:** Modify the text in the text area for the prompt you want to customize. You can use placeholders like `${filePath}` and `${selectedText}` to include information about the current file and selection.\n4. **Click \"Done\":** Save your changes.\n\nBy using Saarthi's Code Actions, you can quickly get AI-powered assistance directly within your coding workflow. This can save you time and help you write better code.\n" }, "docs/saarthi/features/custom-instructions.md": { "content": "import Codicon from '@site/src/components/Codicon';\n\n# Custom Instructions\n\nCustom Instructions allow you to personalize how Saarthi behaves, providing specific guidance that shapes responses, coding style, and decision-making processes.\n\n:::info Instruction File Locations\nYou can provide custom instructions using dedicated files or directories within your workspace. This allows for better organization and version control.\n\n**Workspace-Wide Instructions:** Apply to all modes in the project.\n* **Preferred Method: Directory (`.saarthi/rules/`)**\n ```\n .\n ├── .saarthi/\n │ └── rules/ # Workspace-wide rules\n │ ├── 01-general.md\n │ └── 02-coding-style.txt\n └── ... (other project files)\n ```\n* **Fallback Method: Single File (`.saarthirules`)**\n ```\n .\n ├── .saarthirules # Workspace-wide rules (single file)\n └── ... (other project files)\n ```\n\n**Mode-Specific Instructions:** Apply only to a specific mode (e.g., `code`).\n* **Preferred Method: Directory (`.saarthi/rules-{modeSlug}/`)**\n ```\n .\n ├── .saarthi/\n │ └── rules-code/ # Rules for \"code\" mode\n │ ├── 01-js-style.md\n │ └── 02-ts-style.md\n └── ... (other project files)\n ```\n* **Fallback Method: Single File (`.saarthirules-{modeSlug}`)**\n ```\n .\n ├── .saarthirules-code # Rules for \"code\" mode (single file)\n └── ... (other project files)\n ```\nThe directory methods take precedence if they exist and contain files. See [Workspace-Level Instructions](#workspace-level-instructions) and [Mode-Specific Instructions](#mode-specific-instructions) for details.\n:::\n\n## What Are Custom Instructions?\n\nCustom Instructions define specific behaviors, preferences, and constraints beyond Saarthi's basic role definition. Examples include coding style, documentation standards, testing requirements, and workflow guidelines.\n\n## Setting Custom Instructions\n\n### Global Custom Instructions\n\nThese instructions apply across all workspaces and maintain your preferences regardless of which project you're working on.\n\n**How to set them:**\n\n1. **Open Prompts Tab:** Click the icon in the Saarthi top menu bar\n2. **Find Section:** Find the \"Custom Instructions for All Modes\" section\n3. **Enter Instructions:** Enter your instructions in the text area\n4. **Save Changes:** Click \"Done\" to save your changes\n\n### Workspace-Level Instructions\n\nThese instructions only apply within your current workspace, allowing you to customize Saarthi's behavior for specific projects.\n\n#### Workspace-Wide Instructions via Files/Directories\n\nWorkspace-wide instructions apply to all modes within the current project and can be defined using files:\n\n* **Preferred Method: Directory-Based (`.saarthi/rules/`)**\n * Create a directory named `.saarthi/rules/` in your workspace root.\n * Place instruction files (e.g., `.md`, `.txt`) inside. Saarthi reads files recursively, appending their content to the system prompt in **alphabetical order** based on filename.\n * This method takes precedence if the directory exists and contains files.\n* **Fallback Method: File-Based (`.saarthirules`)**\n * If `.saarthi/rules/` doesn't exist or is empty, Saarthi looks for a single `.saarthirules` file in the workspace root.\n * If found, its content is loaded.\n\n#### Mode-Specific Instructions\n\nMode-specific instructions can be set in two independent ways that can be used simultaneously:\n\n1. **Using the Prompts Tab:**\n\n![Saarthi Prompts tab showing mode-specific custom instructions interface](/img/custom-instructions/custom-instructions-2.png)\n * **Open Tab:** Click the icon in the Saarthi top menu bar\n * **Select Mode:** Under the Modes heading, click the button for the mode you want to customize\n * **Enter Instructions:** Enter your instructions in the text area under \"Mode-specific Custom Instructions (optional)\"\n * **Save Changes:** Click \"Done\" to save your changes\n\n :::info Global Mode Rules\n If the mode itself is global (not workspace-specific), any custom instructions you set for it will also apply globally for that mode across all workspaces.\n :::\n\n2. **Using Rule Files/Directories:** Provide mode-specific instructions via files:\n * **Preferred Method: Directory-Based (`.saarthi/rules-{modeSlug}/`)**\n * Create a directory named `.saarthi/rules-{modeSlug}/` (e.g., `.saarthi/rules-docs-writer/`) in your workspace root.\n * Place instruction files inside (recursive loading). Files are read and appended to the system prompt in **alphabetical order** by filename.\n * This method takes precedence over the fallback file method for the specific mode if the directory exists and contains files.\n * **Fallback Method: File-Based (`.saarthirules-{modeSlug}`)**\n * If `.saarthi/rules-{modeSlug}/` doesn't exist or is empty, Saarthi looks for a single `.saarthirules-{modeSlug}` file (e.g., `.saarthirules-code`) in the workspace root.\n * If found, its content is loaded for that mode.\n\nInstructions from the Prompts tab, the mode-specific directory/file, and the workspace-wide directory/file are all combined. See the section below for the exact order.\n\n## How Instructions are Combined\n\nInstructions are placed in the system prompt in this exact format:\n\n```\n====\nUSER'S CUSTOM INSTRUCTIONS\n\nThe following additional instructions are provided by the user, and should be followed to the best of your ability without interfering with the TOOL USE guidelines.\n\n[Language Preference (if set)]\n\n[Global Instructions (from Prompts Tab)]\n\n[Mode-specific Instructions (from Prompts Tab for the current mode)]\n\nMode-Specific Instructions (from Files/Directories):\n[Contents of files in .saarthi/rules-{modeSlug}/ (if directory exists and is not empty)]\n[Contents of .saarthirules-{modeSlug} file (if .saarthi/rules-{modeSlug}/ does not exist or is empty, and file exists)]\n\nWorkspace-Wide Instructions (from Files/Directories):\n[Contents of files in .saarthi/rules/ (if directory exists and is not empty)]\n[Contents of .saarthirules file (if .saarthi/rules/ does not exist or is empty, and file exists)]\n\n====\n```\n\n*Note: The exact order ensures that more specific instructions (mode-level) appear before more general ones (workspace-wide), and directory-based rules take precedence over file-based fallbacks within each level.*\n\n## Rules about .rules files\n\n* **File Location:** The preferred method uses directories within `.saarthi/` (`.saarthi/rules/` and `.saarthi/rules-{modeSlug}/`). The fallback method uses single files (`.saarthirules` and `.saarthirules-{modeSlug}`) located directly in the workspace root.\n* **Empty Files:** Empty or missing rule files are silently skipped\n* **Source Headers:** Each rule file's contents are included with a header indicating its source\n* **Rule Interaction:** Mode-specific rules complement global rules rather than replacing them\n\n## Examples of Custom Instructions\n\n* \"Always use spaces for indentation, with a width of 4 spaces\"\n* \"Use camelCase for variable names\"\n* \"Write unit tests for all new functions\"\n* \"Explain your reasoning before providing code\"\n* \"Focus on code readability and maintainability\"\n* \"Prioritize using the most common library in the community\"\n* \"When adding new features to websites, ensure they are responsive and accessible\"\n\n:::tip Pro Tip: File-Based Team Standards\nWhen working in team environments, using the `.saarthi/rules/` directory structure (and potentially `.saarthi/rules-{modeSlug}/` directories for specific modes) under version control is the recommended way to standardize Saarthi's behavior across your team. This allows for better organization of multiple instruction files and ensures consistent code style, documentation practices, and development workflows. The older `.saarthirules` file method can still be used but offers less flexibility.\n:::\n\n## Combining with Custom Modes\n\nFor advanced customization, combine with [Custom Modes](custom-modes) to create specialized environments with specific tool access, file restrictions, and tailored instructions.\n" }, "docs/saarthi/features/custom-modes.mdx": { "content": "import Codicon from '@site/src/components/Codicon';\n\n# Customizing Modes\n\nSaarthi allows you to create **custom modes** to tailor Saarthi's behavior to specific tasks or workflows. Custom modes can be either **global** (available across all projects) or **project-specific** (defined within a single project).\n\n
\n\n:::tip Sticky Models for Efficient Workflow\nEach mode—including custom ones—features **Sticky Models**. This means Saarthi automatically remembers and selects the last model you used with a particular mode. This lets you assign different preferred models to different tasks without constant reconfiguration, as Saarthi switches between models when you change modes.\n:::\n\n## Why Use Custom Modes?\n\n* **Specialization:** Create modes optimized for specific tasks, like \"Documentation Writer,\" \"Test Engineer,\" or \"Refactoring Expert.\"\n* **Safety:** Restrict a mode's access to sensitive files or commands. For example, a \"Review Mode\" could be limited to read-only operations.\n* **Experimentation:** Safely experiment with different prompts and configurations without affecting other modes.\n* **Team Collaboration:** Share custom modes with your team to standardize workflows.\n\n\n## What's Included in a Custom Mode?\n\nCustom modes are defined by several key properties. Understanding these concepts will help you tailor Saarthi's behavior effectively.\n\n| UI Field / YAML Property | Conceptual Description |\n| ---------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |\n| Slug (`slug`) | A **unique internal identifier** for the mode. It's used by Saarthi to reference the mode, especially for associating [mode-specific instruction files](#mode-specific-instructions-via-filesdirectories). |\n| Name (`name`) | The **display name** for the mode as it appears in the Saarthi user interface. This should be human-readable and descriptive. |\n| Role Definition (`roleDefinition`) | Defines the **core identity and expertise** of the mode. This text is placed at the beginning of the system prompt.
- Its primary function is to define Saarthi's personality and behavior when this mode is active.
- The **first sentence** (up to the first period `.`) serves as a default concise summary for Saarthi to understand the mode's general purpose.
- **However, if the `whenToUse` property is defined, `whenToUse` takes precedence** for summarizing the mode's function, especially in contexts like task orchestration or mode switching. In such cases, the first sentence of `roleDefinition` is less critical for this specific summarization task, though the entire `roleDefinition` is still used when the mode is active to guide its overall behavior. |\n| Available Tools (`groups`) | Defines the **allowed toolsets and file access permissions** for the mode.
- In the UI, this corresponds to selecting which general categories of tools (like reading files, editing files, browsing, or executing commands) the mode can use.
- File type restrictions for the \"edit\" group are typically managed via manual YAML/JSON configuration or by asking Saarthi to set them up, as detailed in the [Property Details for `groups`](#groups). |\n| When to Use (optional) (`whenToUse`) | (Optional) Provides **guidance for Saarthi to understand the mode's purpose**, especially for automated decisions.
- This text is used by Saarthi, particularly the [`🪃 Orchestrator`](/basic-usage/using-modes#orchestrator-mode-aka-boomerang-mode) mode, for [orchestrating tasks](/features/boomerang-tasks) (e.g., via the [`new_task`](/advanced-usage/available-tools/new-task) tool).
- It also helps Saarthi decide which mode is appropriate when [switching modes](/basic-usage/using-modes#switching-between-modes) (e.g., via the [`switch_mode`](/advanced-usage/available-tools/switch-mode) tool).
- If populated, this description is used by Saarthi to understand the mode's function; otherwise, the first sentence of the `roleDefinition` is used by default. |\n| Custom Instructions (optional) (`customInstructions`) | **Specific behavioral guidelines** or rules for the mode.
- These instructions are added near the end of the system prompt to further refine Saarthi's behavior beyond the `roleDefinition`.
- This can be provided directly in the configuration or via separate instruction files. |\n\n## Methods for Creating and Configuring Custom Modes\n\nYou can create and configure custom modes in several ways:\n\n### 1. Ask Saarthi! (Recommended)\n\nYou can quickly create a basic custom mode by asking Saarthi to do it for you. For example:\n```\nCreate a new mode called \"Documentation Writer\". It should only be able to read files and write Markdown files.\n```\nSaarthi will guide you through the process, prompting for necessary information for the properties described in the [What's Included in a Custom Mode?](#whats-included-in-a-custom-mode) table. Saarthi will create the mode using the preferred YAML format. For fine-tuning or making specific adjustments later, you can use the Prompts tab or manual configuration.\n\n### 2. Using the Prompts Tab\n\n1. **Open Prompts Tab:** Click the icon in the Saarthi top menu bar.\n2. **Create New Mode:** Click the button to the right of the Modes heading.\n3. **Fill in Fields:**\n\n![Custom mode creation interface in the Prompts tab](/img/custom-modes/custom-modes-4.png)\n\n *The custom mode creation interface showing fields for name, slug, save location, role definition, available tools, custom instructions.*\n\n The interface provides fields for `Name`, `Slug`, `Save Location`, `Role Definition`, `When to Use (optional)`, `Available Tools`, and `Custom Instructions`. After filling these, click the \"Create Mode\" button. Saarthi will save the new mode in YAML format.\n\n*Refer to the [What's Included in a Custom Mode?](#whats-included-in-a-custom-mode) table for conceptual explanations of each property. File type restrictions for the \"edit\" tool group can be added by asking Saarthi or through manual YAML/JSON configuration.*\n\n### 3. Manual Configuration (YAML & JSON)\n\nYou can directly edit the configuration files to create or modify custom modes. This method offers the most control over all properties. Saarthi now supports both YAML (preferred) and JSON formats.\n\n* **Global Modes:** Edit the `custom_modes.yaml` (preferred) or `custom_modes.json` file. Access it via **Prompts Tab** > (Settings Menu icon next to \"Global Prompts\") > \"Edit Global Modes\".\n* **Project Modes:** Edit the `.saarthimodes` file (which can be YAML or JSON) in your project root. Access it via **Prompts Tab** > (Settings Menu icon next to \"Project Prompts\") > \"Edit Project Modes\".\n\nThese files define an array/list of custom modes.\n\n**YAML Example (`custom_modes.yaml` or `.saarthimodes`):**\n```yaml\ncustomModes:\n - slug: docs-writer\n name: 📝 Documentation Writer\n roleDefinition: You are a technical writer specializing in clear documentation.\n whenToUse: Use this mode for writing and editing documentation.\n customInstructions: Focus on clarity and completeness in documentation.\n groups:\n - read\n - - edit # This group allows editing specific files\n - fileRegex: \\.(md|mdx)$ # Regex for Markdown files\n description: Markdown files only\n - browser\n - slug: another-mode\n name: Another Mode\n # ... other properties\n```\n\n**JSON Alternative (`custom_modes.json` or `.saarthimodes`):**\n```json\n{\n \"customModes\": [\n {\n \"slug\": \"docs-writer\",\n \"name\": \"📝 Documentation Writer\",\n \"roleDefinition\": \"You are a technical writer specializing in clear documentation.\",\n \"whenToUse\": \"Use this mode for writing and editing documentation.\",\n \"customInstructions\": \"Focus on clarity and completeness in documentation.\",\n \"groups\": [\n \"read\",\n [\"edit\", { \"fileRegex\": \"\\\\.(md|mdx)$\", \"description\": \"Markdown files only\" }],\n \"browser\"\n ]\n },\n {\n \"slug\": \"another-mode\",\n \"name\": \"Another Mode\"\n }\n ]\n}\n```\n\n### YAML/JSON Property Details\n\n##### `slug`\n* **Purpose:** A unique identifier for the mode.\n* **Format:** Use lowercase letters, numbers, and hyphens.\n* **Usage:** Used internally and in file/directory names for mode-specific rules (e.g., `.saarthi/rules-{slug}/`).\n* **Recommendation:** Keep it short and descriptive.\n* *YAML Example:* `slug: docs-writer`\n* *JSON Example:* `\"slug\": \"docs-writer\"`\n\n##### `name`\n* **Purpose:** The display name shown in the Saarthi UI.\n* **Format:** Can include spaces and proper capitalization.\n* *YAML Example:* `name: 📝 Documentation Writer`\n* *JSON Example:* `\"name\": \"Documentation Writer\"`\n\n##### `roleDefinition`\n* **Purpose:** Detailed description of the mode's role, expertise, and personality.\n* **Placement:** This text is placed at the beginning of the system prompt when the mode is active.\n* **Important First Sentence:** The first sentence (up to the first period `.`) serves as a default concise summary for Saarthi to understand the mode's general purpose. **However, if the `whenToUse` property is defined, `whenToUse` takes precedence** for summarizing the mode's function, especially in contexts like task orchestration or mode selection.\n* *YAML Example (multi-line):*\n ```yaml\n roleDefinition: >-\n You are a test engineer with expertise in:\n - Writing comprehensive test suites\n - Test-driven development\n ```\n* *JSON Example:* `\"roleDefinition\": \"You are a technical writer specializing in clear documentation.\"`\n\n##### `groups`\n* **Purpose:** Array/list defining which tool groups the mode can access and any file restrictions.\n* **Available Tool Groups (Strings):** `\"read\"`, `\"edit\"`, `\"browser\"`, `\"command\"`, `\"mcp\"`.\n* **File Restrictions for \"edit\" group:**\n * To apply file restrictions, the \"edit\" entry becomes a list (YAML) or array (JSON) where the first element is `\"edit\"` and the second is a map/object defining the restrictions.\n * `fileRegex`: A regular expression string to control which files the mode can edit.\n * In YAML, typically use single backslashes for regex special characters (e.g., `\\.md$`).\n * In JSON, backslashes must be double-escaped (e.g., `\\\\.md$`).\n * `description`: An optional string describing the restriction.\n * For more complex patterns, see [Understanding Regex in Custom Modes](#understanding-regex-in-custom-modes).\n* *YAML Example:*\n ```yaml\n groups:\n - read\n - - edit # Start of \"edit\" tool with restrictions\n - fileRegex: \\.(js|ts)$ # Restriction map for JS/TS files\n description: JS/TS files only\n - command\n ```\n* *JSON Example:*\n ```json\n \"groups\": [\n \"read\",\n [\"edit\", { \"fileRegex\": \"\\\\.(js|ts)$\", \"description\": \"JS/TS files only\" }],\n \"command\"\n ]\n ```\n\n##### `whenToUse`\n* **Purpose:** (Optional) Provides guidance for Saarthi to understand what this mode does. Used by the Orchestrator mode and for mode switching.\n* **Format:** A string describing ideal scenarios or task types for this mode.\n* **Usage:** If populated, Saarthi uses this description. Otherwise, the first sentence of `roleDefinition` is used.\n* *YAML Example:* `whenToUse: This mode is best for refactoring Python code.`\n* *JSON Example:* `\"whenToUse\": \"This mode is best for refactoring Python code.\"`\n\n##### `customInstructions`\n* **Purpose:** A string containing additional behavioral guidelines for the mode.\n* **Placement:** This text is added near the end of the system prompt.\n* **Supplementing:** Can be supplemented by [Mode-Specific Instructions via Files/Directories](#mode-specific-instructions-via-filesdirectories).\n* *YAML Example (multi-line):*\n ```yaml\n customInstructions: |-\n When writing tests:\n - Use describe/it blocks\n - Include meaningful descriptions\n ```\n* *JSON Example:* `\"customInstructions\": \"Focus on explaining concepts and providing examples.\"`\n\n### Benefits of YAML Format\n\nYAML is now the preferred format for defining custom modes due to several advantages over JSON:\n\n* **Readability:** YAML's indentation-based structure is often easier for humans to read and understand complex configurations.\n* **Comments:** YAML allows for comments (lines starting with `#`), making it possible to annotate your mode definitions.\n ```yaml\n customModes:\n - slug: security-review\n name: 🔒 Security Reviewer\n # This mode is restricted to read-only access\n roleDefinition: You are a security specialist reviewing code for vulnerabilities.\n whenToUse: Use for security reviews and vulnerability assessments.\n # Only allow reading files, no editing permissions\n groups:\n - read\n - browser\n ```\n* **Multi-line Strings:** YAML provides cleaner syntax for multi-line strings (e.g., for `roleDefinition` or `customInstructions`) using `|` (literal block) or `>` (folded block).\n ```yaml\n customModes:\n - slug: test-engineer\n name: 🧪 Test Engineer\n roleDefinition: >-\n You are a test engineer with expertise in:\n - Writing comprehensive test suites\n - Test-driven development\n - Integration testing\n - Performance testing\n customInstructions: |-\n When writing tests:\n - Use describe/it blocks\n - Include meaningful descriptions\n - Test edge cases\n - Ensure proper coverage\n # ... other properties\n ```\n* **Less Punctuation:** YAML generally requires less punctuation (like commas and braces) compared to JSON, reducing syntax errors.\n* **Editor Support:** Most modern code editors provide excellent syntax highlighting and validation for YAML files, further enhancing readability and reducing errors.\n\nWhile JSON is still fully supported, new modes created via the UI or by asking Saarthi will default to YAML.\n\n#### Tips for Working with YAML\n\nWhen editing YAML manually, keep these points in mind:\n\n* **Indentation is Key:** YAML uses indentation (spaces, not tabs) to define structure. Incorrect indentation is the most common source of errors. Ensure consistent spacing for nested elements.\n* **Colons for Key-Value Pairs:** Keys must be followed by a colon and a space (e.g., `slug: my-mode`).\n* **Hyphens for List Items:** List items start with a hyphen and a space (e.g., `- read`).\n* **Validate Your YAML:** If you encounter issues, use an online YAML validator or your editor's built-in validation to check for syntax errors.\n\n### Migration to YAML Format\n\n* **Global Modes:** The migration from `custom_modes.json` to `custom_modes.yaml` happens automatically when Saarthi starts up, under these conditions:\n 1. Saarthi starts up.\n 2. A `custom_modes.json` file exists.\n 3. No `custom_modes.yaml` file exists yet.\n The migration process reads the existing JSON file, converts it to YAML format, creates a new `custom_modes.yaml` file, and preserves the original JSON file (e.g., by renaming it) for rollback purposes. If `custom_modes.yaml` already exists, it will be used, and no automatic migration of `custom_modes.json` will occur.\n\n* **Project Modes (`.saarthimodes`):**\n * **No automatic startup migration:** Unlike global modes, project-specific `.saarthimodes` files are not automatically converted from JSON to YAML simply when Saarthi starts.\n * **Format Detection:** Saarthi can read `.saarthimodes` files in either YAML or JSON format.\n * **Conversion on UI Edit:** If you edit a project-specific mode through the Saarthi UI (e.g., via the Prompts Tab), and the existing `.saarthimodes` file is in JSON format, Saarthi will save the changes in YAML format. This effectively converts the file to YAML. The original JSON content will be overwritten with YAML.\n * **Manual Conversion:** If you want to convert an existing `.saarthimodes` JSON file to YAML without making UI edits, you'll need to do this manually. You can:\n 1. Open your existing JSON `.saarthimodes` file.\n 2. Convert its content to YAML (you can ask Saarthi to help with this, or use an online converter).\n 3. Replace the content of your `.saarthimodes` file with the new YAML content, or rename the old file (e.g., `.saarthimodes.json.bak`) and save the new content into a file named `.saarthimodes`.\n Ensure the resulting YAML is valid.\n\n:::tip\nFor manual conversions of `.saarthimodes` files, you can use online JSON to YAML converters or ask Saarthi to help reformat a specific mode configuration from JSON to YAML. Always validate your YAML before saving.\n:::\n## Mode-Specific Instructions via Files/Directories\n\n:::info Mode-Specific Instruction File Locations\nYou can provide instructions for custom modes using dedicated files or directories within your workspace. This allows for better organization and version control compared to only using the `customInstructions` property.\n\n**Preferred Method: Directory (`.saarthi/rules-{mode-slug}/`)**\n```\n.\n├── .saarthi/\n│ └── rules-docs-writer/ # Example for mode slug \"docs-writer\"\n│ ├── 01-style-guide.md\n│ └── 02-formatting.txt\n└── ... (other project files)\n```\n\n**Fallback Method: Single File (`.saarthirules-{mode-slug}`)**\n```\n.\n├── .saarthirules-docs-writer # Example for mode slug \"docs-writer\"\n└── ... (other project files)\n```\nThe directory method takes precedence if it exists and contains files.\n:::\n\nIn addition to the `customInstructions` property, you can provide mode-specific instructions via files in your workspace. This is particularly useful for:\n\n* Organizing lengthy or complex instructions into multiple, manageable files.\n* Managing instructions easily with version control.\n* Allowing non-technical team members to modify instructions without editing YAML/JSON.\n\nThere are two ways Saarthi loads these instructions, with a clear preference for the newer directory-based method:\n\n**1. Preferred Method: Directory-Based Instructions (`.saarthi/rules-{mode-slug}/`)**\n\n* **Structure:** Create a directory named `.saarthi/rules-{mode-slug}/` in your workspace root. Replace `{mode-slug}` with your mode's slug (e.g., `.saarthi/rules-docs-writer/`).\n* **Content:** Place one or more files (e.g., `.md`, `.txt`) containing your instructions inside this directory. You can organize instructions further using subdirectories; Saarthi reads files recursively, appending their content to the system prompt in **alphabetical order** based on filename.\n* **Loading:** All instruction files found within this directory structure will be loaded and applied to the specified mode.\n\n**2. Fallback (Backward Compatibility): File-Based Instructions (`.saarthirules-{mode-slug}`)**\n\n* **Structure:** If the `.saarthi/rules-{mode-slug}/` directory **does not exist or is empty**, Saarthi will look for a single file named `.saarthirules-{mode-slug}` in your workspace root (e.g., `.saarthirules-docs-writer`).\n* **Loading:** If found, the content of this single file will be loaded as instructions for the mode.\n\n**Precedence:**\n\n* The **directory-based method (`.saarthi/rules-{mode-slug}/`) takes precedence**. If this directory exists and contains files, any corresponding root-level `.saarthirules-{mode-slug}` file will be **ignored** for that mode.\n* This ensures that projects migrated to the new directory structure behave predictably, while older projects using the single-file method remain compatible.\n\n**Combining with `customInstructions`:**\n\n* Instructions loaded from either the directory or the fallback file are combined with the `customInstructions` property defined in the mode's configuration.\n* Typically, the content from the files/directories is appended after the content from the `customInstructions` property.\n\n## Configuration Precedence\n\nMode configurations are applied in this order:\n\n1. Project-level mode configurations (from `.saarthimodes` - YAML or JSON)\n2. Global mode configurations (from `custom_modes.yaml`, then `custom_modes.json` if YAML not found)\n3. Default mode configurations\n\nThis means that project-specific configurations will override global configurations, which in turn override default configurations. You can override any default mode by including a mode with the same slug in your global or project-specific configuration.\n\n* **Note on Instruction Files:** Within the loading of mode-specific instructions from the filesystem, the directory `.saarthi/rules-{mode-slug}/` takes precedence over the single file `.saarthirules-{mode-slug}` found in the workspace root.\n\n## Overriding Default Modes\n\nYou can override Saarthi's built-in modes (like `💻 Code`, `🪲 Debug`, `❓ Ask`, `🏗️ Architect`, `🪃 Orchestrator`) with customized versions. This is done by creating a custom mode with the same slug as a default mode (e.g., `code`, `debug`).\n\n### Overriding Modes Globally\n\nTo customize a default mode across all your projects:\n\n1. **Open Prompts Tab:** Click the icon.\n2. **Access Settings Menu:** Click the icon next to \"Global Prompts\".\n3. **Edit Global Modes:** Select \"Edit Global Modes\" to edit `custom_modes.yaml` (or `custom_modes.json`).\n4. **Add Your Override:**\n\n**YAML Example:**\n```yaml\ncustomModes:\n - slug: code # Matches the default 'code' mode slug\n name: 💻 Code (Global Override) # Custom display name\n roleDefinition: You are a software engineer with global-specific constraints.\n whenToUse: This globally overridden code mode is for JS/TS tasks.\n customInstructions: Focus on project-specific JS/TS development.\n groups:\n - read\n - - edit\n - fileRegex: \\.(js|ts)$\n description: JS/TS files only\n```\n\n**JSON Alternative:**\n```json\n{\n \"customModes\": [{\n \"slug\": \"code\",\n \"name\": \"💻 Code (Global Override)\",\n \"roleDefinition\": \"You are a software engineer with global-specific constraints\",\n \"whenToUse\": \"This globally overridden code mode is for JS/TS tasks.\",\n \"customInstructions\": \"Focus on project-specific JS/TS development\",\n \"groups\": [\n \"read\",\n [\"edit\", { \"fileRegex\": \"\\\\.(js|ts)$\", \"description\": \"JS/TS files only\" }]\n ]\n }]\n}\n```\nThis example replaces the default `💻 Code` mode with a version restricted to JavaScript and TypeScript files.\n\n### Project-Specific Mode Override\n\nTo override a default mode for just one project:\n\n1. **Open Prompts Tab:** Click the icon.\n2. **Access Settings Menu:** Click the icon next to \"Project Prompts\".\n3. **Edit Project Modes:** Select \"Edit Project Modes\" to edit the `.saarthimodes` file (YAML or JSON).\n4. **Add Your Override:**\n\n**YAML Example:**\n```yaml\ncustomModes:\n - slug: code # Matches the default 'code' mode slug\n name: 💻 Code (Project-Specific) # Custom display name\n roleDefinition: You are a software engineer with project-specific constraints for this project.\n whenToUse: This project-specific code mode is for Python tasks within this project.\n customInstructions: Adhere to PEP8 and use type hints.\n groups:\n - read\n - - edit\n - fileRegex: \\.py$\n description: Python files only\n - command\n```\n\n**JSON Alternative:**\n```json\n{\n \"customModes\": [{\n \"slug\": \"code\",\n \"name\": \"💻 Code (Project-Specific)\",\n \"roleDefinition\": \"You are a software engineer with project-specific constraints for this project.\",\n \"whenToUse\": \"This project-specific code mode is for Python tasks within this project.\",\n \"customInstructions\": \"Adhere to PEP8 and use type hints.\",\n \"groups\": [\n \"read\",\n [\"edit\", { \"fileRegex\": \"\\\\.py$\", \"description\": \"Python files only\" }],\n \"command\"\n ]\n }]\n}\n```\nProject-specific overrides take precedence over global overrides.\n\n### Common Use Cases for Overriding Default Modes\n* **Restricting file access:** Limit a mode to specific file types.\n* **Specializing behavior:** Customize expertise for your tech stack.\n* **Adding custom instructions:** Integrate project standards.\n* **Changing available tools:** Remove tools to prevent unwanted operations.\n\n:::tip\nWhen overriding default modes, test carefully. Consider backing up configurations before major changes.\n:::\n\n## Understanding Regex in Custom Modes\n\nRegular expressions (`fileRegex`) offer fine-grained control over file editing permissions.\n\n:::tip\n**Let Saarthi Build Your Regex Patterns**\n\nInstead of writing complex regex manually, ask Saarthi:\n```\nCreate a regex pattern that matches JavaScript files but excludes test files\n```\nSaarthi will generate the pattern. Remember to adapt it for YAML (usually single backslashes) or JSON (double backslashes).\n:::\n\nWhen you specify `fileRegex`, you're creating a pattern that file paths must match.\n\n**Important Rules for `fileRegex`:**\n* **Escaping in JSON:** In JSON strings, backslashes (`\\`) must be double-escaped (e.g., `\\\\.md$`).\n* **Escaping in YAML:** In unquoted or single-quoted YAML strings, a single backslash is usually sufficient for regex special characters (e.g., `\\.md$`).\n* **Path Matching:** Patterns match against the full relative file path from your workspace root (e.g., `src/components/button.js`).\n* **Case Sensitivity:** Regex patterns are case-sensitive by default.\n\n**Common Pattern Examples:**\n(Note: JSON examples show double backslashes; YAML would typically use single backslashes for these.)\n\n| Pattern (Conceptual / YAML-like) | JSON `fileRegex` Value | Matches | Doesn't Match |\n| -------------------------------- | ------------------------------- | ----------------------------------------- | ------------------------------------- |\n| `\\.md$` | `\"\\\\.md$\"` | `readme.md`, `docs/guide.md` | `script.js`, `readme.md.bak` |\n| `^src/.*` | `\"^src/.*\"` | `src/app.js`, `src/components/button.tsx` | `lib/utils.js`, `test/src/mock.js` |\n| `\\.(css|scss)$` | \"\\\\.(css|scss)$\" | `styles.css`, `theme.scss` | `styles.less`, `styles.css.map` |\n| `docs/.*\\.md$` | `\"docs/.*\\\\.md$\"` | `docs/guide.md`, `docs/api/reference.md` | `guide.md`, `src/docs/notes.md` |\n| `^(?!.*(test\\|spec))\\.(js\\|ts)$` | `\"^(?!.*(test\\|spec))\\\\.(js\\|ts)$\"` | `app.js`, `utils.ts` | `app.test.js`, `utils.spec.js`, `app.jsx` |\n\n\n**Key Regex Building Blocks:**\n* `\\.`: Matches a literal dot. (YAML: `\\.`, JSON: `\\\\.`)\n* `$`: Matches the end of the string.\n* `^`: Matches the beginning of the string.\n* `.*`: Matches any character (except newline) zero or more times.\n* `(a|b)`: Matches either \"a\" or \"b\". (e.g., `\\.(js|ts)$`)\n* `(?!...)`: Negative lookahead.\n\n**Testing Your Patterns:**\n1. Test on sample file paths. Online regex testers are helpful.\n2. Remember the escaping rules for JSON vs. YAML.\n3. Start simple and build complexity.\n\n## Community Gallery\nReady to explore more? Check out the [Custom Modes Gallery](/community/#custom-modes-gallery) section on the main community page to discover and share custom modes created by the community!\n" }, "docs/saarthi/features/enhance-prompt.md": { "content": "import Codicon from '@site/src/components/Codicon';\n\n# Enhance Prompt\n\nThe \"Enhance Prompt\" feature in Saarthi helps you improve the quality and effectiveness of your prompts before sending them to the AI model. By clicking the icon in the chat input, you can automatically refine your initial request, making it clearer, more specific, and more likely to produce the desired results.\n\n## Why Use Enhance Prompt?\n\n* **Improved Clarity:** Saarthi can rephrase your prompt to make it more understandable for the AI model.\n* **Added Context:** The enhancement process can add relevant context to your prompt, such as the current file path or selected code.\n* **Better Instructions:** Saarthi can add instructions to guide the AI towards a more helpful response (e.g., requesting specific formatting or a particular level of detail).\n* **Reduced Ambiguity:** Enhance Prompt helps to eliminate ambiguity and ensure that Saarthi understands your intent.\n* **Consistency**: Saarthi will consistently format prompts the same way to the AI.\n\n## How to Use Enhance Prompt\n\n1. **Type your initial prompt:** Enter your request in the Saarthi chat input box as you normally would. This can be a simple question, a complex task description, or anything in between.\n2. **Click the Icon:** Instead of pressing Enter, click the icon located in the bottom right of the chat input box.\n3. **Review the Enhanced Prompt:** Saarthi will replace your original prompt with an enhanced version. Review the enhanced prompt to make sure it accurately reflects your intent. You can further refine the enhanced prompt before sending.\n4. **Send the Enhanced Prompt:** Press Enter or click the Send icon () to send the enhanced prompt to Saarthi.\n\n## Customizing the Enhancement Process\n\nThe \"Enhance Prompt\" feature uses a customizable prompt template. You can modify this template to tailor the enhancement process to your specific needs.\n\n1. **Open the Prompts Tab:** Click the icon in the Saarthi top menu bar.\n2. **Select \"ENHANCE\" Tab:** You should see listed out support prompts, including \"ENHANCE\". Click on this tab.\n3. **Edit the Prompt Template:** Modify the text in the \"Prompt\" field.\n\nThe default prompt template includes the placeholder `${userInput}`, which will be replaced with your original prompt. You can modify this to fit the model's prompt format, and instruct it how to enhance your request.\n\n## API Configuration\n\nThe API configuration used for Enhance Prompt is, by default, the same one that is selected for Saarthi tasks,\nbut it can be changed:\n\n1. **Open the Prompts Tab:** Click the icon in the Saarthi top menu bar.\n2. **Select \"ENHANCE\" Tab:** You should see an \"API Configuration\" dropdown\n3. **Select an API Configuration:** Choose an existing configuration, and future Enhance Prompt requests will be sent to that configured provider/model.\n\n## Limitations and Best Practices\n\n* **Experimental Feature:** Prompt enhancement is an experimental feature. The quality of the enhanced prompt may vary depending on the complexity of your request and the capabilities of the underlying model.\n* **Review Carefully:** Always review the enhanced prompt before sending it. Saarthi may make changes that don't align with your intentions.\n* **Iterative Process:** You can use the \"Enhance Prompt\" feature multiple times to iteratively refine your prompt.\n* **Not a Replacement for Clear Instructions:** While \"Enhance Prompt\" can help, it's still important to write clear and specific prompts from the start.\n\nBy using the \"Enhance Prompt\" feature, you can improve the quality of your interactions with Saarthi and get more accurate and helpful responses.\n" }, "docs/saarthi/features/experimental/codebase-indexing.mdx": { "content": "import Codicon from '@site/src/components/Codicon';\n\n# Codebase Indexing\n\n**⚠️ Experimental Feature:** This feature is under active development and may change significantly in future releases.\n\nCodebase Indexing enables semantic code search across your entire project using AI embeddings. Instead of searching for exact text matches, it understands the *meaning* of your queries, helping Saarthi find relevant code even when you don't know specific function names or file locations.\n\n## What It Does\n\nWhen enabled, the indexing system:\n\n1. **Parses your code** using Tree-sitter to identify semantic blocks (functions, classes, methods)\n2. **Creates embeddings** of each code block using AI models\n3. **Stores vectors** in a Qdrant database for fast similarity search\n4. **Provides the [`codebase_search`](./advanced-usage/available-tools/codebase-search) tool** to Saarthi for intelligent code discovery\n\nThis enables natural language queries like \"user authentication logic\" or \"database connection handling\" to find relevant code across your entire project.\n\n## Key Benefits\n\n- **Semantic Search**: Find code by meaning, not just keywords\n- **Enhanced AI Understanding**: Saarthi can better comprehend and work with your codebase\n- **Cross-Project Discovery**: Search across all files, not just what's open\n- **Pattern Recognition**: Locate similar implementations and code patterns\n\n## Setup Requirements\n\n### Embedding Provider\n\nChoose one of these options for generating embeddings:\n\n**OpenAI (Recommended)**\n- Requires OpenAI API key\n- Supports all OpenAI embedding models\n- Default: `text-embedding-3-small`\n- Processes up to 100,000 tokens per batch\n\n**Ollama (Local)**\n- Requires local Ollama installation\n- No API costs or internet dependency\n- Supports any Ollama-compatible embedding model\n- Requires Ollama base URL configuration\n\n### Setting Up Ollama for Embeddings\n\n1. **Install Ollama**\n - **macOS**: `brew install ollama` or download from [ollama.com](https://ollama.com)\n - **Linux**: `curl -fsSL https://ollama.com/install.sh | sh`\n - **Windows**: Download installer from [ollama.com](https://ollama.com)\n\n2. **Start Ollama Service**\n ```bash\n ollama serve\n ```\n This starts Ollama on `http://localhost:11434` (default port)\n\n3. **Install Embedding Model**\n ```bash\n ollama pull nomic-embed-text\n ```\n This downloads the recommended embedding model (~274MB)\n\n4. **Verify Installation**\n ```bash\n ollama list\n ```\n You should see `nomic-embed-text` in the list\n\n5. **Configure in Saarthi**\n - Set Ollama Base URL: `http://localhost:11434`\n - Select Model: `nomic-embed-text`\n\n### Vector Database\n\n**Qdrant** is required for storing and searching embeddings:\n- **Local**: `http://localhost:6333` (recommended for testing)\n- **Cloud**: Qdrant Cloud or self-hosted instance\n- **Authentication**: Optional API key for secured deployments\n\n## Setting Up Qdrant\n\n### Quick Local Setup\n\n**Using Docker:**\n```bash\ndocker run -p 6333:6333 qdrant/qdrant\n```\n\n**Using Docker Compose:**\n```yaml\nversion: '3.8'\nservices:\n qdrant:\n image: qdrant/qdrant\n ports:\n - \"6333:6333\"\n volumes:\n - qdrant_storage:/qdrant/storage\nvolumes:\n qdrant_storage:\n```\n\n### Production Deployment\n\nFor team or production use:\n- [Qdrant Cloud](https://cloud.qdrant.io/) - Managed service\n- Self-hosted on AWS, GCP, or Azure\n- Local server with network access for team sharing\n\n## Configuration\n\n1. Open Saarthi settings ( icon)\n2. Navigate to **Experimental** section\n3. Enable **\"Enable Codebase Indexing\"**\n4. Configure your embedding provider:\n - **OpenAI**: Enter API key and select model\n - **Ollama**: Enter base URL and select model\n5. Set Qdrant URL and optional API key\n6. Click **Save** to start initial indexing\n\n## Understanding Index Status\n\nThe interface shows real-time status with color indicators:\n\n- **Standby** (Gray): Not running, awaiting configuration\n- **Indexing** (Yellow): Currently processing files\n- **Indexed** (Green): Up-to-date and ready for searches\n- **Error** (Red): Failed state requiring attention\n\n## How Files Are Processed\n\n### Smart Code Parsing\n- **Tree-sitter Integration**: Uses AST parsing to identify semantic code blocks\n- **Language Support**: All languages supported by Tree-sitter\n- **Fallback**: Line-based chunking for unsupported file types\n- **Block Sizing**: \n - Minimum: 100 characters\n - Maximum: 1,000 characters\n - Splits large functions intelligently\n\n### Automatic File Filtering\nThe indexer automatically excludes:\n- Binary files and images\n- Large files (>1MB)\n- Git repositories (`.git` folders)\n- Dependencies (`node_modules`, `vendor`, etc.)\n- Files matching `.gitignore` and `.saarthiignore` patterns\n\n### Incremental Updates\n- **File Watching**: Monitors workspace for changes\n- **Smart Updates**: Only reprocesses modified files\n- **Hash-based Caching**: Avoids reprocessing unchanged content\n- **Branch Switching**: Automatically handles Git branch changes\n\n## Best Practices\n\n### Model Selection\n\n**For OpenAI:**\n- **`text-embedding-3-small`**: Best balance of performance and cost\n- **`text-embedding-3-large`**: Higher accuracy, 5x more expensive\n- **`text-embedding-ada-002`**: Legacy model, lower cost\n\n**For Ollama:**\n- **`mxbai-embed-large`**: The largest and highest-quality embedding model. \n- **`nomic-embed-text`**: Best balance of performance and embedding quality.\n- **`all-minilm`**: Compact model with lower quality but faster performance.\n\n### Security Considerations\n- **API Keys**: Stored securely in VS Code's encrypted storage\n- **Code Privacy**: Only small code snippets sent for embedding (not full files)\n- **Local Processing**: All parsing happens locally\n- **Qdrant Security**: Use authentication for production deployments\n\n## Current Limitations\n\n- **File Size**: 1MB maximum per file\n- **Markdown**: Not currently supported due to parsing complexity\n- **Single Workspace**: One workspace at a time\n- **Dependencies**: Requires external services (embedding provider + Qdrant)\n- **Language Coverage**: Limited to Tree-sitter supported languages\n\n## Using the Search Feature\n\nOnce indexed, Saarthi can use the [`codebase_search`](/saarthi/advanced-usage/available-tools/codebase-search) tool to find relevant code:\n\n**Example Queries:**\n- \"How is user authentication handled?\"\n- \"Database connection setup\"\n- \"Error handling patterns\"\n- \"API endpoint definitions\"\n\nThe tool provides Saarthi with:\n- Relevant code snippets\n- File paths and line numbers\n- Similarity scores\n- Contextual information\n\n## Privacy & Security\n\n- **Code stays local**: Only small code snippets sent for embedding\n- **Embeddings are numeric**: Not human-readable representations\n- **Secure storage**: API keys encrypted in VS Code storage\n- **Local option**: Use Ollama for completely local processing\n- **Access control**: Respects existing file permissions\n\n## Future Enhancements\n\nPlanned improvements:\n- Additional embedding providers\n- Improved markdown and documentation support\n- Multi-workspace indexing\n- Enhanced filtering and configuration options\n- Team sharing capabilities\n- Integration with VS Code's native search\n" }, "docs/saarthi/features/experimental/concurrent-file-reads.md": { "content": "\nimport Codicon from '@site/src/components/Codicon';\n\n# Concurrent File Reads\n\n:::warning Experimental Feature\nConcurrent file reads is an experimental feature that allows Saarthi to read multiple files simultaneously in a single request. This feature is under active development and may change significantly in future releases.\n:::\n\nThis feature dramatically improves your workflow by allowing Saarthi to gather context from multiple files in a single conversation turn, rather than asking to read files one by one. Instead of waiting through multiple back-and-forth exchanges, Saarthi can understand your entire project structure much faster.\n\n## Why This Matters\n\n**Faster Context Building**: Previously, when Saarthi needed to understand your project, you'd see multiple requests like:\n- \"Can I read `src/app.js`?\" → You approve\n- \"Now can I read `src/utils.js`?\" → You approve\n- \"And can I read `src/config.json`?\" → You approve\n\n**With concurrent file reads**: Saarthi asks once to read all related files together, getting the full picture immediately and providing better assistance faster.\n\n## Key Benefits\n\n- **Fewer interruptions** - One approval dialog instead of many\n- **Faster assistance** - Saarthi understands your project context immediately\n- **Better analysis** - Saarthi can see relationships between files from the start\n- **Smoother workflow** - Less waiting, more productive conversations\n\n## Enabling Concurrent File Reads\n\nTo enable this experimental feature:\n\n1. Open Saarthi settings ( icon in the top right corner)\n2. Navigate to **Advanced Settings** → **Experimental Features**\n3. Check **\"Enable concurrent file reads\"**\n\nOnce enabled, you can configure the maximum number of concurrent files (2-100) using the slider that appears below the toggle.\n\n## Configuration Options\n\n### Maximum Concurrent Files\n\n- **Default**: 15 files when enabled, 1 file when disabled\n- **Range**: 2-100 files (when experimental feature is enabled)\n- **Location**: Settings → Advanced Settings → Context Management → \"Concurrent file reads limit\"\n\nThe setting automatically adjusts when you toggle the experimental feature:\n- Enabling sets the limit to 15 files\n- Disabling resets the limit to 1 file (standard behavior)\n\n## User Experience\n\n### Batch Approval Interface\n\nWhen Saarthi requests to read multiple files, you'll see a new batch approval interface that displays:\n\n- List of all files to be read\n- File paths with line range indicators (if specified)\n- Clickable file headers to open files in your editor\n- **Approve All** and **Deny All** buttons for quick decisions\n- Individual file permission controls\n\n### Enhanced Tool Behavior\n\nThe [`read_file`](/advanced-usage/available-tools/read-file) tool automatically adapts based on your settings:\n\n- **Single-file mode** (disabled): Uses traditional one-file-at-a-time approach\n- **Multi-file mode** (enabled): Accepts multiple files in a single request using XML format\n\n## How It Works Behind the Scenes\n\nWhen you enable concurrent file reads, Saarthi can request multiple files at once instead of one at a time. You don't need to understand the technical details - it just works more efficiently.\n\n### What You Control\n\n- **Enable/disable the feature** - Simple checkbox in settings\n- **Set file limits** - Choose how many files Saarthi can request at once (2-100, default 15)\n- **Individual file approval** - Still approve or deny specific files if needed\n\n### What Happens Automatically\n\n- Saarthi groups related files together in single requests\n- Files are processed in parallel for faster results\n- If some files are blocked or denied, Saarthi still gets the approved ones\n- Everything falls back gracefully if there are any issues\n\n## What You'll Experience\n\n### Before Concurrent File Reads\nWhen working on a feature that spans multiple files, you might see:\n1. Saarthi: \"I need to understand your authentication system. Can I read `auth.js`?\"\n2. You: *Approve*\n3. Saarthi: \"Now I need to see the user model. Can I read `user.js`?\"\n4. You: *Approve*\n5. Saarthi: \"And the database config. Can I read `database.js`?\"\n6. You: *Approve*\n7. Finally, Saarthi can help with your request\n\n### With Concurrent File Reads\n1. Saarthi: \"I need to understand your authentication system. Can I read these files: `auth.js`, `user.js`, `database.js`?\"\n2. You: *Approve all at once*\n3. Saarthi immediately provides comprehensive help\n\n### Real-World Impact\n\n- **Faster debugging** - Saarthi can see the full picture of related files instantly\n- **Better code reviews** - Understanding entire features at once, not piece by piece\n- **Smoother refactoring** - Analyzing dependencies across multiple files simultaneously\n- **Less clicking** - One approval instead of many interruptions\n\n## Backward Compatibility\n\nThe feature maintains full backward compatibility:\n\n- Existing single-file requests continue to work unchanged\n- Legacy `path` parameter format is still supported\n- No breaking changes for current workflows\n- Smooth transition between single and multi-file modes\n\n## Common Questions\n\n**\"I enabled the feature but don't see any difference\"**\n- Restart VS Code after enabling the experimental feature\n- The feature only activates when Saarthi needs to read multiple files\n\n**\"Saarthi is asking for too many files at once\"**\n- Lower the concurrent file limit in settings (try 5-10 instead of 15)\n- You can still approve or deny individual files in the batch dialog\n\n**\"Some files were denied but others approved\"**\n- This is normal - Saarthi gets the files you approve and works with those\n- Files might be blocked by your `.saarthiignore` settings\n\n## Need Help?\n\nIf you run into issues:\n1. Check the [FAQ section](/faq) for common solutions\n2. Report problems on [GitHub Issues](https://github.com/godspeedsystems/Saarthi/issues)\n3. Include what you were trying to do and any error messages\n\n## What's Next\n\nThis experimental feature is actively being improved based on user feedback. We're working on making the approval interface even smoother and helping Saarthi make smarter decisions about which files to request together.\n\nYour experience matters - let us know how this feature works for your workflow!\n\nFor more about experimental features, see [Experimental Features Overview](/features/experimental/experimental-features).\n" }, "docs/saarthi/features/experimental/experimental-features.md": { "content": "import Codicon from '@site/src/components/Codicon';\n\n# Experimental Features\n\nSaarthi includes experimental features that are still under development. These features may be unstable, change significantly, or be removed in future versions. Use them with caution and be aware that they may not work as expected.\n\n**Warning:** Experimental features may have unexpected behavior, including potential data loss or security vulnerabilities. Enable them at your own risk.\n\n## Enabling Experimental Features\n\nTo enable or disable experimental features:\n\n1. Open the Saarthi settings ( icon in the top right corner).\n2. Go to the \"Advanced Settings\" section.\n3. Find the \"Experimental Features\" section.\n\n## Current Experimental Features\n\nThe following experimental features are currently available:\n\n- [Codebase Indexing](codebase-indexing) - Semantic search through AI-powered codebase indexing\n- [Concurrent File Reads](concurrent-file-reads) - Read multiple files simultaneously for improved efficiency\n- [Power Steering](power-steering)\n\n## Providing Feedback\n\nIf you encounter any issues with experimental features, or if you have suggestions for improvements, please report them on the [Saarthi GitHub Issues page](https://github.com/godspeedsystems/Saarthi/issues).\n\nYour feedback is valuable and helps us improve Saarthi!\n" }, "docs/saarthi/features/experimental/power-steering.md": { "content": "---\nsidebar_label: 'Power Steering'\n---\nimport Codicon from '@site/src/components/Codicon';\n\n# Power Steering (Experimental Feature)\n\nThe \"Power Steering\" experimental feature (`POWER_STEERING`) is designed to enhance the consistency of Saarthi's responses by more frequently reminding the underlying Large Language Model (LLM) about its current mode definition and any custom instructions.\n\n## How It Works\n\nWhen Power Steering is enabled, Saarthi constantly reinforces the LLM's understanding of its assigned role (e.g., \"You are a helpful coding assistant\") and any specific guidelines provided by the user (e.g., \"Always provide code examples in Python\").\n\nThis is achieved by explicitly including the `modeDetails.roleDefinition` and `modeDetails.customInstructions` within the information sent to the LLM with each interaction.\n\n**Goal:**\nThe primary goal is to ensure the LLM adheres more strictly to its defined persona and follows user-specific instructions more consistently. If you find Saarthi deviating from its role or overlooking custom rules, Power Steering can help maintain its focus.\n\n**Trade-off:**\nThese frequent reminders consume additional tokens in each message sent to the LLM. This means:\n* Increased token usage per message.\n* Potentially higher operational costs.\n* The context window may be filled more quickly.\n\nIt's a balance between stricter adherence to instructions and resource consumption.\n\n**Default Status:** Disabled.\n\n## Technical Details\n\n* **Experiment ID:** `powerSteering`\n* **Mechanism:**\n * The feature's status is checked by the `getEnvironmentDetails` function.\n * If enabled, the current mode's `roleDefinition` and `customInstructions` are added to the details sent to the LLM.\n * These details are wrapped in `` tags and become part of the context for each LLM interaction.\n* **Impact:** By frequently including the role definition and custom instructions, the LLM is steered to generate responses more aligned with these parameters.\n\n## Enabling This Feature\n\nPower Steering is managed within the \"Experimental Features\" section of Saarthi's Advanced Settings.\n\n1. Open Saarthi settings ( icon in the top right corner).\n2. Navigate to \"Advanced Settings\".\n3. Locate the \"Experimental Features\" area.\n4. Toggle the \"Power Steering\" option.\n5. Save your changes.\n![Settings for Intelligent Context Condensation and Power Steering](/img/power-steering/power-steering.png)\n\nFor general information on experimental features, see [Experimental Features Overview](/features/experimental/experimental-features).\n\n## Feedback\n\nPlease report any issues or suggestions regarding this feature on the [Saarthi GitHub Issues page](https://github.com/godspeedsystems/Saarthi/issues). Your feedback is crucial for improving Saarthi.\n" }, "docs/saarthi/features/fast-edits.md": { "content": "import Codicon from '@site/src/components/Codicon';\n\n# Diff/Fast Edits\n\n:::info Default Setting\nFast Edits (using the \"Enable editing through diffs\" setting) is enabled by default in Saarthi. You typically don't need to change these settings unless you encounter specific issues or want to experiment with different diff strategies.\n:::\n\nSaarthi offers an advanced setting to change how it edits files, using diffs (differences) instead of rewriting entire files. Enabling this feature provides significant benefits.\n\n:::note Per-Provider Setting\nDiff editing configuration is set per [API Configuration Profile](/features/api-configuration-profiles), allowing you to customize editing behavior for different providers and models.\n:::\n\n## Enable Editing Through Diffs\n\nOpen the Saarthi pane settings by clicking the gear icon . The `Providers` section will be visible. Select the specific [API Configuration Profile](/features/api-configuration-profiles) you want to configure.\n\nWhen **Enable editing through diffs** is checked:\n\n1. **Faster File Editing**: Saarthi modifies files more quickly by applying only the necessary changes.\n2. **Prevents Truncated Writes**: The system automatically detects and rejects attempts by the AI to write incomplete file content, which can happen with large files or complex instructions. This helps prevent corrupted files.\n\n:::note Disabling Fast Edits\nIf you uncheck **Enable editing through diffs**, Saarthi will revert to writing the entire file content for every edit using the [`write_to_file`](/advanced-usage/available-tools/write-to-file) tool, instead of applying targeted changes with [`apply_diff`](/advanced-usage/available-tools/apply-diff). This full-write approach is generally slower for modifying existing files and leads to higher token usage.\n:::\n\n## Match Precision\n\nThis slider controls how closely the code sections identified by the AI must match the actual code in your file before a change is applied.\n\n![Saarthi settings showing Enable editing through diffs checkbox and Match precision slider](/img/fast-edits/fast-edits-4.png)\n\n* **100% (Default)**: Requires an exact match. This is the safest option, minimizing the risk of incorrect changes.\n* **Lower Values (80%-99%)**: Allows for \"fuzzy\" matching. Saarthi can apply changes even if the code section has minor differences from what the AI expected. This can be useful if the file has been slightly modified, but **increases the risk** of applying changes in the wrong place.\n\n**Use values below 100% with extreme caution.** Lower precision might be necessary occasionally, but always review the proposed changes carefully.\n\nInternally, this setting adjusts a `fuzzyMatchThreshold` used with algorithms like Levenshtein distance to compare code similarity.\n" }, "docs/saarthi/features/footgun-prompting.md": { "content": "---\nsidebar_label: 'Footgun Prompting'\n---\nimport Codicon from '@site/src/components/Codicon';\n\n# Footgun Prompting: Override System Prompts\n\nFootgun Prompting (System Prompt Override) lets you replace the default system prompt for a specific Saarthi mode. This offers granular control but bypasses built-in safeguards. Use with caution.\n\n![Warning indicator for active system prompt override](/img/footgun-prompting/footgun-prompting-1.png)\n**Warning Indicator:** When a system prompt override is active for the current mode, Saarthi will display a warning icon in the chat input area to remind you that the default behavior has been modified.\n\n\n:::info **footgun** _(noun)_\n\n1. _(programming slang, humorous, derogatory)_ Any feature likely to lead to the programmer shooting themself in the foot.\n\n> The System Prompt Override is considered a footgun because modifying the core instructions without a deep understanding can lead to unexpected or broken behavior, especially regarding tool usage and response consistency.\n\n:::\n\n## How It Works\n\n1. **Override File:** Create a file named `.saarthi/system-prompt-{mode-slug}` in your workspace root (e.g., `.saarthi/system-prompt-code` for the Code mode).\n2. **Content:** The content of this file becomes the new system prompt for that specific mode.\n3. **Activation:** Saarthi automatically detects this file. When present, it replaces most of the standard system prompt sections.\n4. **Preserved Sections:** Only the core `roleDefinition` and any `customInstructions` you've set for the mode are kept alongside your override content. Standard sections like tool descriptions, rules, and capabilities are bypassed.\n5. **Construction:** The final prompt sent to the model looks like this:\n\n ```\n ${roleDefinition}\n\n ${content_of_your_override_file}\n\n ${customInstructions}\n ```\n\n## Accessing the Feature\n\nFind the option and instructions in the Saarthi UI:\n\n1. Click the icon in the Saarthi top menu bar.\n2. Expand the **\"Advanced: Override System Prompt\"** section.\n3. Clicking the file path link within the explanation will open or create the correct override file for the currently selected mode in VS Code.\n\n![UI showing the Advanced: Override System Prompt section](/img/footgun-prompting/footgun-prompting.png)\n\n## Using Context Variables\n\nWhen creating your custom system prompt file, you can use special variables (placeholders) that Saarthi will automatically replace with relevant information about the current environment. This allows you to make your prompts more dynamic and context-aware.\n\nHere are the available variables:\n\n- `{{mode}}`: The slug (short name) of the current Saarthi mode being used (e.g., `code`, `chat-mode`).\n- `{{language}}`: The display language configured in VS Code (e.g., `en`, `es`).\n- `{{shell}}`: The default terminal shell configured in VS Code (e.g., `/bin/bash`, `powershell.exe`).\n- `{{operatingSystem}}`: The type of operating system your computer is running (e.g., `Linux`, `Darwin` for macOS, `Windows_NT`).\n- `{{workspace}}`: The file path to the root of your current project workspace.\n\n**Example Usage:**\n\nYou can include these variables directly in your prompt file content like this:\n\n```\nYou are assisting a user in the '{{mode}}' mode.\nTheir operating system is {{operatingSystem}} and their default shell is {{shell}}.\nThe project is located at: {{workspace}}.\nPlease respond in {{language}}.\n```\n\nSaarthi substitutes these placeholders before sending the prompt to the model.\n\n## Key Considerations & Warnings\n\n- **Intended Audience:** Best suited for users deeply familiar with Saarthi's prompting system and the implications of modifying core instructions.\n- **Impact on Functionality:** Custom prompts override standard instructions, including those for tool usage and response consistency. This can cause unexpected behavior or errors if not managed carefully.\n- **Mode-Specific:** Each override file applies only to the mode specified in its filename (`{mode-slug}`).\n- **No File, No Override:** If the `.saarthi/system-prompt-{mode-slug}` file doesn't exist, Saarthi uses the standard system prompt generation process for that mode.\n- **Blank Files Ignored:** If the override file exists but is empty (blank), it will be ignored and the default system prompt will be used.\n- **Directory Creation:** Saarthi ensures the `.saarthi` directory exists before attempting to read or create the override file.\nUse this feature cautiously. Incorrect implementation can significantly degrade Saarthi's performance and reliability for the affected mode.\n" }, "docs/saarthi/features/intelligent-context-condensing.mdx": { "content": "---\nsidebar_label: 'Intelligent Context Condensing'\n---\nimport Codicon from '@site/src/components/Codicon';\n\n# Intelligent Context Condensing\n\n## How It Works\n\nAs your conversation with Saarthi grows, it might approach the context window limit of the underlying AI model. When this happens, older messages would typically be removed to make space. Intelligent Context Condensing aims to prevent this abrupt loss by:\n\n1. **Summarizing:** Using an AI model, it condenses earlier parts of the conversation.\n2. **Retaining Essentials:** The goal is to reduce the overall token count while keeping the key information from the summarized messages.\n3. **Maintaining Flow:** This allows the AI to have a more coherent understanding of the entire conversation, even very long ones.\n\n**Important Considerations:**\n* **Summarization Impact:** While original messages are preserved if you use [Checkpoints](/features/checkpoints) to rewind, the summarized version is what's used in ongoing LLM calls to keep the context manageable.\n* **Cost:** The AI call to perform the summarization incurs a cost. This cost is included in the context condensing metrics displayed in the UI.\n\n## Configuration\n\nIntelligent Context Condensing is **enabled by default** and offers several configuration options:\n\n1. Open Saarthi settings ( icon in the top right corner of the Saarthi panel).\n2. Navigate to the \"Context\" settings section.\n3. Configure the available options:\n - **Automatically trigger intelligent context condensing**: Enabled by default, this controls whether condensing happens automatically\n - **Threshold to trigger intelligent context condensing**: A percentage slider (default 100%) that determines when condensing activates based on context window usage\n - **API Configuration for Context Condensing**: Choose which API configuration to use for condensing operations (defaults to your current active configuration)\n - **Custom Context Condensing Prompt**: Customize the system prompt used for context condensing operations\n\n![Settings for Intelligent Context Condensing](/img/intelligent-context-condensing/intelligent-context-condensing.png)\n*Intelligent Context Condensing configuration options: automatic triggering toggle, threshold slider, API configuration selection, and custom prompt customization.*\n## Controlling and Understanding Context Condensing\n\nSaarthi provides several ways to control and understand the Intelligent Context Condensing feature:\n\n### Controlling Context Condensing\n* **Automatic Threshold:** The threshold slider in \"Context\" settings allows you to define a percentage (e.g., 80%) of context window usage. Saarthi will attempt to condense the context automatically when the conversation reaches this level of capacity.\n* **API Configuration:** Select which API configuration to use for context condensing operations. This allows you to use a different provider or model specifically for condensing if desired.\n* **Custom Prompts:** Modify the system prompt used for condensing to better suit your workflow or to emphasize certain aspects of conversation summarization.\n* **Manual Trigger:** A **Condense Context** button is available at the top of the task, positioned to the right of the context bar. This allows you to initiate the context condensing process at any time.\n\n![Manual Condense Context button in expanded task view](/img/intelligent-context-condensing/intelligent-context-condensing-1.png)\n *The Manual Condense Context button (highlighted with a yellow arrow) is easily accessible for manual control.*\n\n### Understanding Context Condensing Activity\n* **Context Condensing Metrics:** When context condensing occurs, Saarthi displays:\n * The context token counts before and after context condensing.\n * The cost associated with the context condensing AI call.\n * An expandable summary detailing what was condensed (this information is part of the `ContextCondenseRow` component visible in the chat history).\n\n\"Context\n*After context condensing, a message indicates the context has been condensed, showing token changes and cost.*\n\n* **Visual Indicators:**\n * A progress indicator (\"Condensing context...\") is shown in the chat interface while context condensing is active.\n\n![Condensing context progress indicator in chat](/img/intelligent-context-condensation/intelligent-context-condensation-3.png)\n*The \"Condensing context...\" indicator appears in the chat during the process.*\n\n * The task header also displays the current context condensing status.\n * The `ContextWindowProgress` bar offers a visual representation of token distribution, including current usage, space reserved for the AI's output, available space, and raw token numbers.\n* **Interface Clarity:** The \"Condense Context\" button includes a tooltip explaining its function, available in all supported languages.\n\n## Technical Implementation\n\n### Token Counting\nSaarthi uses a sophisticated token counting system that:\n- Employs native token counting endpoints when available (e.g., Anthropic's API)\n- Falls back to tiktoken estimation if API calls fail\n- Provides accurate counting for different content types:\n - Text content: Uses word-based estimation with punctuation and newline overhead\n - Image content: Uses a conservative estimate of 300 tokens per image\n - System prompts: Includes additional overhead for structural elements\n\n### Context Window Management\n- By default, 30% of the context window is reserved (20% for model output and 10% as a safety buffer), leaving 70% available for conversation history.\n- This reservation can be overridden by model-specific settings\n- The system automatically calculates available space while maintaining this reservation\n" }, "docs/saarthi/features/keyboard-shortcuts.md": { "content": "---\nsidebar_label: Keyboard Shortcuts\n---\n\n# Keyboard Shortcuts\n\nThe Saarthi interface supports keyboard shortcuts to streamline your workflow and reduce dependence on mouse interactions.\n\n## Available Keyboard Commands\n\nSaarthi offers keyboard commands to enhance your workflow. This page focuses on the `saarthi.acceptInput` command, but here's a quick reference to all keyboard commands:\n\n| Command | Description | Default Shortcut |\n|---------|-------------|-----------------|\n| `saarthi.acceptInput` | Submit text or accept the primary suggestion | None (configurable) |\n| `saarthi.focus` | Focus the Saarthi input box | None (configurable) |\n\n### Key Benefits of Keyboard Commands\n\n* **Keyboard-Driven Interface**: Submit text or select the primary suggestion button without mouse interaction\n* **Improved Accessibility**: Essential for users with mobility limitations or those who experience discomfort with mouse usage\n* **Vim/Neovim Compatibility**: Supports seamless transitions for developers coming from keyboard-centric environments\n* **Workflow Efficiency**: Reduces context switching between keyboard and mouse during development tasks\n\n## saarthi.acceptInput Command\n\nThe `saarthi.acceptInput` command lets you submit text or accept suggestions with keyboard shortcuts instead of clicking buttons or pressing Enter in the input area.\n\n### What It Does\n\nThe `saarthi.acceptInput` command is a general-purpose input submission command. When triggered, it:\n\n- Submits your current text or image input when in the text input area (equivalent to pressing Enter)\n- Clicks the primary (first) button when action buttons are visible (such as confirm/cancel buttons or any other action buttons)\n\n### Detailed Setup Guide\n\n#### Method 1: Using the VS Code UI\n\n1. Open the Command Palette (`Ctrl+Shift+P` or `Cmd+Shift+P` on Mac)\n2. Type \"Preferences: Open Keyboard Shortcuts\"\n3. In the search box, type \"saarthi.acceptInput\"\n4. Locate \"Saarthi: Accept Input/Suggestion\" in the results\n5. Click the + icon to the left of the command\n6. Press your desired key combination (e.g., `Ctrl+Enter` or `Alt+Enter`)\n7. Press Enter to confirm\n\n\n#### Method 2: Editing keybindings.json directly\n\n1. Open the Command Palette (`Ctrl+Shift+P` or `Cmd+Shift+P` on Mac)\n2. Type \"Preferences: Open Keyboard Shortcuts (JSON)\"\n3. Add the following entry to the JSON array:\n\n```json\n{\n \"key\": \"ctrl+enter\", // or your preferred key combination\n \"command\": \"saarthi.acceptInput\",\n \"when\": \"rooViewFocused\" // This is a context condition that ensures the command only works when Saarthi is focused\n}\n```\n\nYou can also use a more specific condition:\n```json\n{\n \"key\": \"ctrl+enter\",\n \"command\": \"saarthi.acceptInput\",\n \"when\": \"webviewViewFocus && webviewViewId == 'saarthi-cline.SidebarProvider'\"\n}\n```\n\n#### Recommended Key Combinations\n\nChoose a key combination that doesn't conflict with existing VS Code shortcuts:\n\n- `Alt+Enter` - Easy to press while typing\n- `Ctrl+Space` - Familiar for those who use autocomplete\n- `Ctrl+Enter` - Intuitive for command execution\n- `Alt+A` - Mnemonic for \"Accept\"\n\n### Practical Use Cases\n\n#### Quick Development Workflows\n\n- **Text Submission**: Send messages to Saarthi without moving your hands from the keyboard\n- **Action Confirmations**: Accept operations like saving files, running commands, or applying diffs\n- **Multi-Step Processes**: Move quickly through steps that require confirmation or input\n- **Consecutive Tasks**: Chain multiple tasks together with minimal interruption\n\n#### Keyboard-Centric Development\n\n- **Vim/Neovim Workflows**: If you're coming from a Vim/Neovim background, maintain your keyboard-focused workflow\n- **IDE Integration**: Use alongside other VS Code keyboard shortcuts for a seamless experience\n- **Code Reviews**: Quickly accept suggestions when reviewing code with Saarthi\n- **Documentation Writing**: Submit text and accept formatting suggestions when generating documentation\n\n#### Accessibility Use Cases\n\n- **Hand Mobility Limitations**: Essential for users who have difficulty using a mouse\n- **Repetitive Strain Prevention**: Reduce mouse usage to prevent or manage repetitive strain injuries\n- **Screen Reader Integration**: Works well with screen readers for visually impaired users\n- **Voice Control Compatibility**: Can be triggered via voice commands when using voice control software\n\n### Accessibility Benefits\n\nThe `saarthi.acceptInput` command was designed with accessibility in mind:\n\n- **Reduced Mouse Dependence**: Complete entire workflows without reaching for the mouse\n- **Reduced Physical Strain**: Helps users who experience discomfort or pain from mouse usage\n- **Alternative Input Method**: Supports users with mobility impairments who rely on keyboard navigation\n- **Workflow Optimization**: Particularly valuable for users coming from keyboard-centric environments like Vim/Neovim\n\n### Keyboard-Centric Workflows\n\nHere are some complete workflow examples showing how to effectively use keyboard shortcuts with Saarthi:\n\n#### Development Workflow Example\n\n1. Open VS Code and navigate to your project\n2. Open Saarthi via the sidebar\n3. Type your request: \"Create a REST API endpoint for user registration\"\n4. When Saarthi asks for framework preferences, use your `saarthi.acceptInput` shortcut to select the first suggestion\n5. Continue using the shortcut to accept code generation suggestions\n6. When Saarthi offers to save the file, use the shortcut again to confirm\n7. Use VS Code's built-in shortcuts to navigate through the created files\n\n#### Code Review Workflow\n\n1. Select code you want to review and use VS Code's \"Copy\" command\n2. Ask Saarthi to review it: \"Review this code for security issues\"\n3. As Saarthi asks clarifying questions about the code context, use your shortcut to accept suggestions\n4. When Saarthi provides improvement recommendations, use the shortcut again to accept implementation suggestions\n\n### Troubleshooting\n\n| Issue | Solution |\n|-------|----------|\n| Shortcut doesn't work | Ensure Saarthi is focused (click in the Saarthi panel first) |\n| Wrong suggestion selected | The command always selects the first (primary) button; use mouse if you need a different option |\n| Conflicts with existing shortcuts | Try a different key combination in VS Code keyboard settings |\n| No visual feedback when used | This is normal - the command silently activates the function without visual confirmation |\n| Shortcut works inconsistently | Make sure the `when` clause is properly configured in your keybindings.json (either `rooViewFocused` or the webview-specific condition) |\n\n### Technical Implementation\n\nThe `saarthi.acceptInput` command is implemented as follows:\n\n- Command registered as `saarthi.acceptInput` with display title \"Saarthi: Accept Input/Suggestion\" in the command palette\n- When triggered, it sends an \"acceptInput\" message to the active Saarthi webview\n- The webview determines the appropriate action based on the current UI state:\n - Clicks the primary action button if action buttons are visible and enabled\n - Sends the message if the text area is enabled and contains text/images\n- No default key binding - users assign their preferred shortcut\n\n### Limitations\n\n- Works only when the Saarthi interface is active\n- Has no effect if no inputs or suggestions are currently available\n- Prioritizes the primary (first) button when multiple options are shown" }, "docs/saarthi/features/mcp/mcp-vs-api.md": { "content": "---\ntitle: MCP vs API\nsidebar_label: MCP vs API\n---\n\n# MCP vs REST APIs: A Fundamental Distinction\n\nComparing REST APIs to the Model Context Protocol (MCP) is a category error. They operate at different layers of abstraction and serve fundamentally different purposes in AI systems.\n\n## Architectural Differences\n\n| Feature | MCP | REST APIs |\n|---------|-----|-----------|\n| State Management | **Stateful** - maintains context across interactions | **Stateless** - each request is independent |\n| Connection Type | Persistent, bidirectional connections | One-way request/response |\n| Communication Style | JSON-RPC based with ongoing sessions | HTTP-based with discrete requests |\n| Context Handling | Context is intrinsic to the protocol | Context must be manually managed |\n| Tool Discovery | Runtime discovery of available tools | Design-time integration requiring prior knowledge |\n| Integration Approach | Runtime integration with dynamic capabilities | Design-time integration requiring code changes |\n\n## Different Layers, Different Purposes\n\nREST APIs and MCP serve different tiers in the technology stack:\n\n1. **REST**: Low-level web communication pattern that exposes operations on resources\n2. **MCP**: High-level AI protocol that orchestrates tool usage and maintains context\n\nMCP often uses REST APIs internally, but abstracts them away for the AI. Think of MCP as middleware that turns discrete web services into a cohesive environment the AI can operate within.\n\n## Context Preservation: Critical for AI Workflows\n\nMCP's stateful design solves a key limitation of REST in AI applications:\n\n- **REST Approach**: Each call is isolated, requiring manual context passing between steps\n- **MCP Approach**: One conversation context persists across multiple tool uses\n\nFor example, an AI debugging a codebase can open a file, run tests, and identify errors without losing context between steps. The MCP session maintains awareness of previous actions and results.\n\n## Dynamic Tool Discovery\n\nMCP enables an AI to discover and use tools at runtime:\n\n```json\n// AI discovers available tools\n{\n \"tools\": [\n {\n \"name\": \"readFile\",\n \"description\": \"Reads content from a file\",\n \"parameters\": {\n \"path\": { \"type\": \"string\", \"description\": \"File path\" }\n }\n },\n {\n \"name\": \"createTicket\",\n \"description\": \"Creates a ticket in issue tracker\",\n \"parameters\": {\n \"title\": { \"type\": \"string\" },\n \"description\": { \"type\": \"string\" }\n }\n }\n ]\n}\n```\n\nThis \"plug-and-play\" capability allows new tools to be added without redeploying or modifying the AI itself.\n\n## Real-World Example: Multi-Tool Workflow\n\nConsider a task requiring multiple services: \"Check recent commits, create a JIRA ticket for the bug fix, and post to Slack.\"\n\n**REST-based approach**:\n- Requires separate integrations for Git, JIRA, and Slack APIs\n- Needs custom code to manage context between calls\n- Breaks if any service changes its API\n\n**MCP-based approach**:\n- One unified protocol for all tools\n- Maintains context across the entire workflow\n- New tools can be swapped in without code changes\n\n## Why Saarthi Uses MCP\n\nSaarthi leverages MCP to provide:\n\n1. **Extensibility**: Add unlimited custom tools without waiting for official integration\n2. **Contextual awareness**: Tools can access conversation history and project context\n3. **Simplified integration**: One standard protocol rather than numerous API patterns\n4. **Runtime flexibility**: Discover and use new capabilities on-the-fly\n\nMCP creates a universal connector between Saarthi and external services, with REST APIs often powering those services behind the scenes.\n\n## Conclusion: Complementary, Not Competing Technologies\n\nMCP doesn't replace REST APIs - it builds upon them. REST excels at providing discrete services, while MCP excels at orchestrating those services for AI agents.\n\nThe critical distinction is that MCP is AI-native: it treats the model as a first-class user, providing the contextual, stateful interaction layer that AI agents need to function effectively in complex environments." }, "docs/saarthi/features/mcp/overview.md": { "content": "---\ntitle: MCP Overview\nsidebar_label: MCP Overview\n---\n\n# Model Context Protocol (MCP)\n\nThe Model Context Protocol (MCP) is a standard for extending Saarthi's capabilities by connecting to external tools and services. MCP servers provide additional tools and resources that help Saarthi accomplish tasks beyond its built-in capabilities, such as accessing databases, custom APIs, and specialized functionality.\n\n## MCP Documentation\n\nThis documentation is organized into several sections:\n\n* [**Using MCP in Saarthi**](/saarthi/features/mcp/using-mcp-in-saarthi) - Comprehensive guide to configuring, enabling, and managing MCP servers with Saarthi. Includes server settings, tool approval, and troubleshooting.\n\n* [**What is MCP?**](/saarthi/features/mcp/what-is-mcp) - Clear explanation of the Model Context Protocol, its client-server architecture, and how it enables AI systems to interact with external tools.\n\n* [**STDIO, Streamable HTTP & SSE Transports**](/saarthi/features/mcp/server-transports) - Detailed comparison of local (STDIO) and remote (Streamable HTTP & legacy SSE) transport mechanisms with deployment considerations for each approach.\n\n* [**MCP vs API**](/saarthi/features/mcp/mcp-vs-api) - Analysis of the fundamental distinction between MCP and REST APIs, explaining how they operate at different layers of abstraction for AI systems.\n\n* [**Recommended MCP Servers**](/saarthi/features/mcp/recommended-mcp-servers) - Curated list of tested and recommended MCP servers for Saarthi, including a setup guide for Context7.\n" }, "docs/saarthi/features/mcp/recommended-mcp-servers.md": { "content": "---\ntitle: Recommended MCP Servers\nsidebar_label: Recommended MCP Servers\n---\nimport Codicon from '@site/src/components/Codicon';\n\n# Recommended MCP Servers\n\nWhile Saarthi can connect to any Model Context Protocol (MCP) server that follows the specification, the community has already built several high-quality servers that work out-of-the-box. This page curates the servers we **actively recommend** and provides step-by-step setup instructions so you can get productive in minutes.\n\n> We'll keep this list up-to-date. If you maintain a server you'd like us to consider, please open a pull-request.\n\n---\n\n## Context7\n\n`Context7` is our first-choice general-purpose MCP server. It ships a collection of highly-requested tools, installs with a single command, and has excellent support across every major editor that speaks MCP.\n\n### Why we recommend Context7\n\n* **One-command install** – everything is bundled, no local build step.\n* **Cross-platform** – runs on macOS, Windows, Linux, or inside Docker.\n* **Actively maintained** – frequent updates from the Upstash team.\n* **Rich toolset** – database access, web-search, text utilities, and more.\n* **Open source** – released under the MIT licence.\n\n---\n\n## Installing Context7 in Saarthi\n\nThere are two common ways to register the server:\n\n1. **Global configuration** – available in every workspace.\n2. **Project-level configuration** – checked into version control alongside your code.\n\nWe'll cover both below.\n\n### 1. Global configuration\n\n1. Open the Saarthi **MCP settings** panel by clicking the icon.\n2. Click **Edit Global MCP**.\n3. Paste the JSON below inside the `mcpServers` object and save.\n\n```json\n{\n \"mcpServers\": {\n \"context7\": {\n \"command\": \"npx\",\n \"args\": [\"-y\", \"@upstash/context7-mcp@latest\"]\n }\n }\n}\n```\n\n**Windows (cmd.exe) variant**\n\n```json\n{\n \"mcpServers\": {\n \"context7\": {\n \"type\": \"stdio\",\n \"command\": \"cmd\",\n \"args\": [\"/c\", \"npx\", \"-y\", \"@upstash/context7-mcp@latest\"]\n }\n }\n}\n```\n\nAlso on **Windows (cmd)** you may need to invoke `npx` through `cmd.exe`:\n\n![Adding Context7 to the global MCP settings](/img/recommended-mcp-servers/context7-global-setup-fixed.png)\n\n### 2. Project-level configuration\n\nIf you prefer to commit the configuration to your repository, create a file called `.saarthi/mcp.json` at the project root and add the same snippet:\n\n```json\n{\n \"mcpServers\": {\n \"context7\": {\n \"command\": \"npx\",\n \"args\": [\"-y\", \"@upstash/context7-mcp@latest\"]\n }\n }\n}\n```\n\n**Windows (cmd.exe) variant**\n\n```json\n{\n \"mcpServers\": {\n \"context7\": {\n \"type\": \"stdio\",\n \"command\": \"cmd\",\n \"args\": [\"/c\", \"npx\", \"-y\", \"@upstash/context7-mcp@latest\"]\n }\n }\n}\n```\n\n![Adding Context7 to a project-level MCP file](/img/recommended-mcp-servers/context7-project-setup-fixed.png)\n\n> When both global and project files define a server with the same name, **the project configuration wins**.\n\n---\n\n## Verifying the installation\n\n1. Make sure **Enable MCP Servers** is turned on in the MCP settings panel.\n2. You should now see **Context7** listed. Click the toggle to start it if it isn't already running.\n3. Saarthi will prompt you the first time a Context7 tool is invoked. Approve the request to continue.\n\n![Context7 running in Saarthi](/img/recommended-mcp-servers/context7-running-fixed.png)\n\n---\n\n## Next steps\n\n* Browse the list of tools shipped with Context7 in the server pane.\n* Configure **Always allow** for the tools you use most to streamline your workflow.\n* Want to expose your own APIs? Check out the [MCP server creation guide](../../features/mcp/using-mcp-in-saarthi#enabling-or-disabling-mcp-server-creation).\n\nLooking for other servers? Watch this page – we'll add more recommendations soon!\n" }, "docs/saarthi/features/mcp/server-transports.md": { "content": "---\ntitle: MCP Server Transports\nsidebar_label: STDIO, Streamable HTTP & SSE Transports\n---\n\n# MCP Server Transports: STDIO, Streamable HTTP & SSE\n\nModel Context Protocol (MCP) supports three primary transport mechanisms for communication between Saarthi and MCP servers: Standard Input/Output (STDIO), Streamable HTTP (the modern standard), and Server-Sent Events (SSE) (for legacy use). Each has distinct characteristics, advantages, and use cases.\n\n## STDIO Transport\n\nSTDIO transport runs locally on your machine and communicates via standard input/output streams.\n\n### How STDIO Transport Works\n\n1. The client (Saarthi) spawns an MCP server as a child process\n2. Communication happens through process streams: client writes to server's STDIN, server responds to STDOUT\n3. Each message is delimited by a newline character\n4. Messages are formatted as JSON-RPC 2.0\n\n```\nClient Server\n | |\n |---- JSON message ------>| (via STDIN)\n | | (processes request)\n |<---- JSON message ------| (via STDOUT)\n | |\n```\n\n### STDIO Characteristics\n\n* **Locality**: Runs on the same machine as Saarthi\n* **Performance**: Very low latency and overhead (no network stack involved)\n* **Simplicity**: Direct process communication without network configuration\n* **Relationship**: One-to-one relationship between client and server\n* **Security**: Inherently more secure as no network exposure\n\n### When to Use STDIO\n\nSTDIO transport is ideal for:\n\n* Local integrations and tools running on the same machine\n* Security-sensitive operations\n* Low-latency requirements\n* Single-client scenarios (one Saarthi instance per server)\n* Command-line tools or IDE extensions\n\n### STDIO Implementation Example\n\n```typescript\nimport { Server } from '@modelcontextprotocol/sdk/server/index.js';\nimport { StdioServerTransport } from '@modelcontextprotocol/sdk/server/stdio.js';\n\nconst server = new Server({name: 'local-server', version: '1.0.0'});\n// Register tools...\n\n// Use STDIO transport\nconst transport = new StdioServerTransport(server);\ntransport.listen();\n```\n## Streamable HTTP Transport\n\nStreamable HTTP transport is the modern standard for remote MCP server communication, replacing the older HTTP+SSE transport. It operates over HTTP/HTTPS and allows for more flexible server implementations.\n\n### How Streamable HTTP Transport Works\n\n1. The server provides a single HTTP endpoint (MCP endpoint) that supports both POST and GET methods.\n2. The client (Saarthi) sends requests to this MCP endpoint using HTTP POST.\n3. The server processes the request and sends back a response.\n4. Optionally, the server can use Server-Sent Events (SSE) over the same connection to stream multiple messages or notifications to the client. This allows for basic request-response interactions as well as more advanced streaming and server-initiated communication.\n\n```\nClient Server\n | |\n |---- HTTP POST /mcp_endpoint ---->| (client request)\n | | (processes request)\n |<--- HTTP Response / SSE Stream --| (server response / stream)\n | |\n```\n\n### Streamable HTTP Characteristics\n\n* **Modern Standard**: Preferred method for new remote MCP server implementations.\n* **Remote Access**: Can be hosted on a different machine from Saarthi.\n* **Scalability**: Can handle multiple client connections concurrently.\n* **Protocol**: Works over standard HTTP/HTTPS.\n* **Flexibility**: Supports simple request-response and advanced streaming.\n* **Single Endpoint**: Uses a single URL path for all MCP communication.\n* **Authentication**: Can use standard HTTP authentication mechanisms.\n* **Backwards Compatibility**: Servers can maintain compatibility with older HTTP+SSE clients.\n\n### When to Use Streamable HTTP\n\nStreamable HTTP transport is ideal for:\n\n* All new remote MCP server developments.\n* Servers requiring robust, scalable, and flexible communication.\n* Integrations that might involve streaming data or server-sent notifications.\n* Public services or centralized tools.\n* Replacing legacy SSE transport implementations.\n\n### Streamable HTTP Implementation Example\n\nConfiguration in `settings.json`:\n```json\n\"mcp.servers\": {\n \"StreamableHTTPMCPName\": {\n \"type\": \"streamable-http\",\n \"url\": \"http://localhost:8080/mcp\"\n }\n}\n```\n\nFor server-side implementation, refer to the MCP SDK documentation for `StreamableHTTPClientTransport`.\n\n### Backwards Compatibility with HTTP+SSE\n\nClients and servers can maintain backwards compatibility with the deprecated HTTP+SSE transport (from protocol version 2024-11-05).\n\nServers wanting to support older clients should:\n* Continue to host both the SSE (`/events`) and POST (`/message`) endpoints of the old transport, alongside the new “MCP endpoint” defined for the Streamable HTTP transport.\n\n## SSE Transport (Legacy)\n\nServer-Sent Events (SSE) transport is a legacy method for remote server communication over HTTP/HTTPS. For new implementations, **Streamable HTTP transport is recommended.** SSE remains available for compatibility with older MCP servers.\n\n### How SSE Transport Works\n\n1. The client (Saarthi) connects to the server's SSE endpoint via HTTP GET request\n2. This establishes a persistent connection where the server can push events to the client\n3. For client-to-server communication, the client makes HTTP POST requests to a separate endpoint\n4. Communication happens over two channels:\n * Event Stream (GET): Server-to-client updates\n * Message Endpoint (POST): Client-to-server requests\n\n```\nClient Server\n | |\n |---- HTTP GET /events ----------->| (establish SSE connection)\n |<---- SSE event stream -----------| (persistent connection)\n | |\n |---- HTTP POST /message --------->| (client request)\n |<---- SSE event with response ----| (server response)\n | |\n```\n\n### SSE Characteristics\n\n* **Remote Access**: Can be hosted on a different machine from Saarthi\n* **Scalability**: Can handle multiple client connections concurrently\n* **Protocol**: Works over standard HTTP (no special protocols needed)\n* **Persistence**: Maintains a persistent connection for server-to-client messages\n* **Authentication**: Can use standard HTTP authentication mechanisms\n\n### When to Use SSE\n\nSSE transport is better for:\n\n* Remote access across networks\n* Multi-client scenarios\n* Public services\n* Centralized tools that many users need to access\n* Integration with web services\n\n### SSE Implementation Example\n\n```typescript\nimport { Server } from '@modelcontextprotocol/sdk/server/index.js';\nimport { SSEServerTransport } from '@modelcontextprotocol/sdk/server/sse.js';\nimport express from 'express';\n\nconst app = express();\nconst server = new Server({name: 'remote-server', version: '1.0.0'});\n// Register tools...\n\n// Use SSE transport\nconst transport = new SSEServerTransport(server);\napp.use('/mcp', transport.requestHandler());\napp.listen(3000, () => {\n console.log('MCP server listening on port 3000');\n});\n```\n## Local vs. Hosted: Deployment Aspects\n\nThe choice between STDIO and SSE transports directly impacts how you'll deploy and manage your MCP servers.\n\n### STDIO: Local Deployment Model\n\nSTDIO servers run locally on the same machine as Saarthi, which has several important implications:\n\n* **Installation**: The server executable must be installed on each user's machine\n* **Distribution**: You need to provide installation packages for different operating systems\n* **Updates**: Each instance must be updated separately\n* **Resources**: Uses the local machine's CPU, memory, and disk\n* **Access Control**: Relies on the local machine's filesystem permissions\n* **Integration**: Easy integration with local system resources (files, processes)\n* **Execution**: Starts and stops with Saarthi (child process lifecycle)\n* **Dependencies**: Any dependencies must be installed on the user's machine\n\n#### Practical Example\n\nA local file search tool using STDIO would:\n* Run on the user's machine\n* Have direct access to the local filesystem\n* Start when needed by Saarthi\n* Not require network configuration\n* Need to be installed alongside Saarthi or via a package manager\n\n### Streamable HTTP / SSE (Legacy): Hosted Deployment Model\n\nStreamable HTTP (recommended) and legacy SSE servers can be deployed to remote servers and accessed over the network:\n\n* **Installation**: Installed once on a server, accessed by many users\n* **Distribution**: Single deployment serves multiple clients\n* **Updates**: Centralized updates affect all users immediately\n* **Resources**: Uses server resources, not local machine resources\n* **Access Control**: Managed through authentication and authorization systems\n* **Integration**: More complex integration with user-specific resources\n* **Execution**: Runs as an independent service (often continuously)\n* **Dependencies**: Managed on the server, not on user machines\n\n#### Practical Example\n\nA database query tool using SSE would:\n* Run on a central server\n* Connect to databases with server-side credentials\n* Be continuously available for multiple users\n* Require proper network security configuration\n* Be deployed using container or cloud technologies\n\n### Hybrid Approaches\n\nSome scenarios benefit from a hybrid approach:\n\n1. **STDIO with Network Access**: A local STDIO server that acts as a proxy to remote services\n2. **SSE with Local Commands**: A remote SSE server that can trigger operations on the client machine through callbacks\n3. **Gateway Pattern**: STDIO servers for local operations that connect to SSE servers for specialized functions\n\n## Choosing Between Transports\n\n| Consideration | STDIO | Streamable HTTP | SSE (Legacy) |\n|---------------|-------|-----------------|--------------|\n| **Location** | Local machine only | Local or remote | Local or remote |\n| **Clients** | Single client | Multiple clients | Multiple clients |\n| **Performance** | Lower latency | Higher latency (network overhead) | Higher latency (network overhead) |\n| **Setup Complexity** | Simpler | More complex (requires HTTP server) | More complex (requires HTTP server, potentially two endpoints) |\n| **Security** | Inherently secure | Requires explicit security measures | Requires explicit security measures |\n| **Network Access** | Not needed | Required | Required |\n| **Scalability** | Limited to local machine | Can distribute across network | Can distribute across network |\n| **Deployment** | Per-user installation | Centralized installation | Centralized installation |\n| **Updates** | Distributed updates | Centralized updates | Centralized updates |\n| **Resource Usage** | Uses client resources | Uses server resources | Uses server resources |\n| **Dependencies** | Client-side dependencies | Server-side dependencies | Server-side dependencies |\n| **Recommendation** | Ideal for local, secure, single-client tools | **Modern standard for all new remote servers** | Legacy, for existing older servers |\n\n## Configuring Transports in Saarthi\n\nFor detailed information on configuring STDIO, Streamable HTTP, and SSE (Legacy) transports in Saarthi, including example configurations, see the [Understanding Transport Types](/saarthi/features/mcp/using-mcp-in-saarthi#understanding-transport-types) section in the Using MCP in Saarthi guide." }, "docs/saarthi/features/mcp/what-is-mcp.md": { "content": "---\ntitle: What is MCP?\nsidebar_label: What is MCP?\n---\n\n# What is MCP?\n\nMCP (Model Context Protocol) is a standardized communication protocol for LLM systems to interact with external tools and services. It functions as a universal adapter between AI assistants and various data sources or applications.\n\n## How It Works\n\nMCP uses a client-server architecture:\n\n1. The AI assistant (client) connects to MCP servers\n2. Each server provides specific capabilities (file access, database queries, API integrations)\n3. The AI uses these capabilities through a standardized interface\n4. Communication occurs via JSON-RPC 2.0 messages\n\nThink of MCP as similar to a USB-C port in the sense that any compatible LLM can connect to any MCP server to access its functionality. This standardization eliminates the need to build custom integrations for each tool and service.\n\nFor example, an AI using MCP can perform tasks like \"search our company database and generate a report\" without requiring specialized code for each database system.\n\n## Common Questions\n\n- **Is MCP a cloud service?** MCP servers can run locally on your computer or remotely as cloud services, depending on the use case and security requirements.\n\n- **Does MCP replace other integration methods?** No. MCP complements existing tools like API plugins and retrieval-augmented generation. It provides a standardized protocol for tool interaction but doesn't replace specialized integration approaches.\n\n- **How is security handled?** Users control which MCP servers they connect to and what permissions those servers have. As with any tool that accesses data or services, use trusted sources and configure appropriate access controls.\n\n## MCP in Saarthi\n\nSaarthi implements the Model Context Protocol to:\n\n- Connect to both local and remote MCP servers\n- Provide a consistent interface for accessing tools\n- Extend functionality without core modifications\n- Enable specialized capabilities on demand\n\nMCP provides a standardized way for AI systems to interact with external tools and services, making complex integrations more accessible and consistent.\n\n## Learn More About MCP\n\nReady to dig deeper? Check out these guides:\n\n- [MCP Overview](/saarthi/features/mcp/overview) - A quick glance at the MCP documentation structure\n- [Using MCP in Saarthi](/saarthi/features/mcp/using-mcp-in-saarthi) - Get started with MCP in Saarthi, including creating simple servers\n- [MCP vs API](/saarthi/features/mcp/mcp-vs-api) - Technical advantages compared to traditional APIs\n- [STDIO & Streamable HTTP & SSE Transports](/saarthi/features/mcp/server-transports) - Local vs. hosted deployment models" }, "docs/saarthi/features/model-temperature.md": { "content": "\nimport Codicon from '@site/src/components/Codicon';\n\n# Model Temperature\n\nTemperature controls the randomness of AI model outputs. Adjusting this setting optimizes results for different tasks - from precise code generation to creative brainstorming. Temperature is one of the most powerful parameters for controlling AI behavior. A well-tuned temperature setting can dramatically improve the quality and appropriateness of responses for specific tasks.\n\n## What is Temperature?\n\nTemperature is a setting (usually between 0.0 and 2.0) that controls how random or predictable the AI's output is. Finding the right balance is key: lower values make the output more focused and consistent, while higher values encourage more creativity and variation. For many coding tasks, a moderate temperature (around 0.3 to 0.7) often works well, but the best setting depends on what you're trying to achieve.\n\n:::info Temperature and Code: Common Misconceptions\nTemperature controls output randomness, not code quality or accuracy directly. Key points:\n\n* **Low Temperature (near 0.0):** Produces predictable, consistent code. Good for simple tasks, but can be repetitive and lack creativity. It doesn't guarantee *better* code.\n* **High Temperature:** Increases randomness, potentially leading to creative solutions but also more errors or nonsensical code. It doesn't guarantee *higher-quality* code.\n* **Accuracy:** Code accuracy depends on the model's training and prompt clarity, not temperature.\n* **Temperature 0.0:** Useful for consistency, but limits exploration needed for complex problems.\n:::\n\n## Default Values in Saarthi\n\nSaarthi uses a default temperature of 0.0 for most models, optimizing for maximum determinism and precision in code generation. This applies to OpenAI models, Anthropic models (non-thinking variants), LM Studio models, and most other providers.\n\nSome models use higher default temperatures - DeepSeek R1 models and certain reasoning-focused models default to 0.6, providing a balance between determinism and creative exploration.\n\nModels with thinking capabilities (where the AI shows its reasoning process) require a fixed temperature of 1.0 which cannot be changed, as this setting ensures optimal performance of the thinking mechanism. This applies to any model with the \":thinking\" flag enabled.\n\nSome specialized models don't support temperature adjustments at all, in which case Saarthi respects these limitations automatically.\n\n## When to Adjust Temperature\n\nHere are some examples of temperature settings that might work well for different tasks:\n\n* **Code Mode (0.0-0.3):** For writing precise, correct code with consistent, deterministic results\n* **Architect Mode (0.4-0.7):** For brainstorming architecture or design solutions with balanced creativity and structure\n* **Ask Mode (0.7-1.0):** For explanations or open-ended questions requiring diverse and insightful responses\n* **Debug Mode (0.0-0.3):** For troubleshooting bugs with consistent precision\n\nThese are starting points – it's important to [experiment with different settings](#experimentation) to find what works best for your specific needs and preferences.\n\n## How to Adjust Temperature\n\n1. **Open the Saarthi Panel:** Click the Saarthi icon in the VS Code Activity Bar\n2. **Open Settings:** Click the icon in the top right corner\n3. **Find Temperature Control:** Navigate to the Providers section\n4. **Enable Custom Temperature:** Check the \"Use custom temperature\" box\n5. **Set Your Value:** Adjust the slider to your preferred value\n\n![Temperature setting in Saarthi settings panel](/img/model-temperature/model-temperature.png)\n *Temperature slider in Saarthi settings panel*\n\n## Using API Configuration Profiles for Temperature\n\nCreate multiple [API configuration profiles](/features/api-configuration-profiles) with different temperature settings:\n\n**How to set up task-specific temperature profiles:**\n\n1. Create specialized profiles like \"Code - Low Temp\" (0.1) and \"Ask - High Temp\" (0.8)\n2. Configure each profile with appropriate temperature settings\n3. Switch between profiles using the dropdown in settings or chat interface\n4. Set different profiles as defaults for each mode for automatic switching when changing modes\n\nThis approach optimizes model behavior for specific tasks without manual adjustments.\n\n## Technical Implementation\n\nSaarthi implements temperature handling with these considerations:\n\n* User-defined settings take priority over defaults\n* Provider-specific behaviors are respected\n* Model-specific limitations are enforced:\n * Thinking-enabled models require a fixed temperature of 1.0\n * Some models don't support temperature adjustments\n\n## Experimentation\n\nExperimenting with different temperature settings is the most effective way to discover what works best for your specific needs:\n\n### Effective Temperature Testing\n\n1. **Start with defaults** - Begin with Saarthi's preset values (0.0 for most tasks) as your baseline\n2. **Make incremental adjustments** - Change values in small steps (±0.1) to observe subtle differences\n3. **Test consistently** - Use the same prompt across different temperature settings for valid comparisons\n4. **Document results** - Note which values produce the best outcomes for specific types of tasks\n5. **Create profiles** - Save effective settings as [API configuration profiles](/features/api-configuration-profiles) for quick access\n\nRemember that different models may respond differently to the same temperature values, and thinking-enabled models always use a fixed temperature of 1.0 regardless of your settings.\n\n## Related Features\n\n- Works with all [API providers](/providers/openai) supported by Saarthi\n- Complements [custom instructions](/features/custom-instructions) for fine-tuning responses\n- Works alongside [custom modes](/features/custom-modes) you create\n" }, "docs/saarthi/features/more-features.md": { "content": "---\nsidebar_label: Additional Features\n---\n\n\n# Additional Features\n\nThis page describes additional features in Saarthi that enhance your development workflow.\n\n## Suggested Responses\n\nSaarthi provides suggested responses to questions, saving you time typing. These suggestions appear as buttons below the chat input box after you ask a question. Click a suggestion to quickly use it as your next prompt.\n\nThis feature aims to streamline your workflow by anticipating your potential follow-up questions and providing one-click access to relevant prompts.\n\n## Text to Speech\n\nSaarthi includes a Text-to-Speech (TTS) feature that reads out the AI responses, allowing you to listen to the information instead of reading it. This can be helpful for accessibility, learning, or simply for a change of pace.\n\nTo use Text-to-Speech, simply enable it in the Saarthi settings. Once enabled, a speaker icon will appear next to each AI response in the chat. Click the icon to start listening.\n\n## Global Language Support\n\nSaarthi supports 14 languages, making it accessible to a wider range of users globally. You can now use Saarthi in:\n\n- Simplified Chinese\n- Traditional Chinese\n- Spanish\n- Hindi\n- French\n- Portuguese\n- German\n- Japanese\n- Korean\n- Italian\n- Turkish\n- Vietnamese\n- Polish\n- Catalan\n\nTo change your language, go to **Advanced Settings > Language** in the Saarthi settings panel.\n\nThis global update ensures a smoother and more inclusive coding experience for users around the world.\n" }, "docs/saarthi/features/saarthiignore.md": { "content": "---\nsidebar_label: .saarthiignore\n---\n\n# Using .saarthiignore to Control File Access\n\nThe `.saarthiignore` file is a key feature for managing Saarthi's interaction with your project files. It allows you to specify files and directories that Saarthi should not access or modify, similar to how `.gitignore` works for Git.\n\n## What is `.saarthiignore`?\n\n* **Purpose**: To protect sensitive information, prevent accidental changes to build artifacts or large assets, and generally define Saarthi's operational scope within your workspace.\n* **How to Use**: Create a file named `.saarthiignore` in the root directory of your VS Code workspace. List patterns in this file to tell Saarthi which files and directories to ignore.\n* **Scope**: `.saarthiignore` affects both Saarthi's tools and context mentions (like `@directory` attachments).\n\nSaarthi actively monitors the `.saarthiignore` file. Any changes you make are reloaded automatically, ensuring Saarthi always uses the most current rules. The `.saarthiignore` file itself is always implicitly ignored, so Saarthi cannot change its own access rules.\n\n## Pattern Syntax\n\nThe syntax for `.saarthiignore` is identical to `.gitignore`. Here are common examples:\n\n* `node_modules/`: Ignores the entire `node_modules` directory.\n* `*.log`: Ignores all files ending in `.log`.\n* `config/secrets.json`: Ignores a specific file.\n* `!important.log`: An exception; Saarthi will *not* ignore this specific file, even if a broader pattern like `*.log` exists.\n* `build/`: Ignores the `build` directory.\n* `docs/**/*.md`: Ignores all Markdown files in the `docs` directory and its subdirectories.\n\nFor a comprehensive guide on syntax, refer to the [official Git documentation on .gitignore](https://git-scm.com/docs/gitignore).\n\n## How Saarthi Tools Interact with `.saarthiignore`\n\n`.saarthiignore` rules are enforced across various Saarthi tools:\n\n### Strict Enforcement (Reads & Writes)\n\nThese tools directly check `.saarthiignore` before any file operation. If a file is ignored, the operation is blocked:\n\n* [`read_file`](/advanced-usage/available-tools/read-file): Will not read ignored files.\n* [`write_to_file`](/advanced-usage/available-tools/write-to-file): Will not write to or create new ignored files.\n* [`apply_diff`](/advanced-usage/available-tools/apply-diff): Will not apply diffs to ignored files.\n* [`insert_content`](/advanced-usage/available-tools/insert-content): Will not write to ignored files.\n* [`search_and_replace`](/advanced-usage/available-tools/search-and-replace): Will not search and replace within ignored files.\n* [`list_code_definition_names`](/advanced-usage/available-tools/list-code-definition-names): Will not parse ignored files for code symbols.\n\n### File Discovery and Listing\n\n* **[`list_files`](/advanced-usage/available-tools/list-files) Tool & `@directory` Attachments**: When Saarthi lists files or when you use `@directory` attachments, ignored files are omitted or marked with a 🔒 symbol (see \"User Experience\" below). Both use identical filtering logic.\n* **Environment Details**: Information about your workspace (like open tabs and project structure) provided to Saarthi is filtered to exclude or mark ignored items.\n\n### Context Mentions\n\n* **`@directory` Attachments**: Directory contents respect `.saarthiignore` patterns. Ignored files are filtered out or marked with `[🔒]` prefix depending on the `showRooIgnoredFiles` setting.\n* **Single File Mentions**: Ignored files return \"(File is ignored by .saarthiignore)\" instead of content.\n\n### Command Execution\n\n* **[`execute_command`](/advanced-usage/available-tools/execute-command) Tool**: This tool checks if a command (from a predefined list like `cat` or `grep`) targets an ignored file. If so, execution is blocked.\n\n## Key Limitations and Scope\n\n* **Workspace-Centric**: `.saarthiignore` rules apply **only to files and directories within the current VS Code workspace root**. Files outside this scope are not affected.\n* **[`execute_command`](/advanced-usage/available-tools/execute-command) Specificity**: Protection for `execute_command` is limited to a predefined list of file-reading commands. Custom scripts or uncommon utilities might not be caught.\n* **Not a Full Sandbox**: `.saarthiignore` is a powerful tool for controlling Saarthi's file access via its tools, but it does not create a system-level sandbox.\n\n## User Experience and Notifications\n\n* **Visual Cue (🔒)**: In file listings and `@directory` attachments, files ignored by `.saarthiignore` may be marked with a lock symbol (🔒), depending on the `showRooIgnoredFiles` setting (defaults to `true`).\n* **Ignore Messages**: Single file mentions return \"(File is ignored by .saarthiignore)\" instead of content.\n* **Error Messages**: If a tool operation is blocked, Saarthi receives an error: `\"Access to [file_path] is blocked by the .saarthiignore file settings. You must try to continue in the task without using this file, or ask the user to update the .saarthiignore file.\"`\n* **Chat Notifications**: You will typically see a notification in the Saarthi chat interface when an action is blocked due to `.saarthiignore`.\n\nThis guide helps you understand the `.saarthiignore` feature, its capabilities, and its current limitations, so you can effectively manage Saarthi's interaction with your codebase.\n" }, "docs/saarthi/features/settings-management.md": { "content": "---\ntitle: Import, Export, and Reset Settings\nsidebar_label: Import/Export/Reset Settings\ndescription: Manage your Saarthi settings by exporting, importing, or resetting them to defaults.\n---\n\nimport Codicon from '@site/src/components/Codicon';\n\n# Import, Export, and Reset Settings\n\nSaarthi allows you to manage your configuration settings effectively through export, import, and reset options. These features are useful for backing up your setup, sharing configurations with others, or restoring default settings if needed.\n\nYou can find these options at the bottom of the Saarthi settings page, accessible via the gear icon () in the Saarthi chat view.\n\n![Export, Import, and Reset buttons in Saarthi settings](/img/settings-management/settings-management.png)\n*Image: Export, Import, and Reset buttons.*\n\n## Export Settings\n\nClicking the **Export** button saves your current Saarthi settings to a JSON file.\n\n* **What's Exported:** The file includes your configured API Provider Profiles and Global Settings (UI preferences, mode configurations, context settings, etc.).\n* **Security Warning:** The exported JSON file contains **all** your configured API Provider Profiles and Global Settings. Crucially, this includes **API keys in plaintext**. Treat this file as highly sensitive. Do not share it publicly or with untrusted individuals, as it grants access to your API accounts.\n* **Process:**\n 1. Click **Export**.\n 2. A file save dialog appears, suggesting `saarthi-code-settings.json` as the filename (usually in your `~/Documents` folder).\n 3. Choose a location and save the file.\n\nThis creates a backup of your configuration or a file you can share.\n\n## Import Settings\n\nClicking the **Import** button allows you to load settings from a previously exported JSON file.\n\n* **Process:**\n 1. Click **Import**.\n 2. A file open dialog appears. Select the `saarthi-code-settings.json` file (or similarly named file) you want to import.\n 3. Saarthi reads the file, validates its contents against the expected schema, and applies the settings.\n* **Merging:** Importing settings **merges** the configurations. It adds new API profiles and updates existing ones and global settings based on the file content. It does **not** delete configurations present in your current setup but missing from the imported file.\n* **Validation:** Only valid settings matching the internal schema can be imported, preventing configuration errors. A success notification appears upon completion.\n\n## Reset Settings\n\nClicking the **Reset** button completely clears all Saarthi configuration data and returns the extension to its default state. This is a destructive action intended for troubleshooting or starting fresh.\n\n* **Warning:** This action is **irreversible**. It permanently deletes all API configurations (including keys stored in secret storage), custom modes, global settings, and task history.\n\n* **Process:**\n 1. Click the red **Reset** button.\n 2. A confirmation dialog appears, warning that the action cannot be undone.\n 3. Click \"Yes\" to confirm.\n\n* **What is Reset:**\n * **API Provider Profiles:** All configurations are deleted from settings and secret storage.\n * **Global Settings:** All preferences (UI, modes, approvals, browser, etc.) are reset to defaults.\n * **Custom Modes:** All user-defined modes are deleted.\n * **Secret Storage:** All API keys and other secrets managed by Saarthi are cleared.\n * **Task History:** The current task stack is cleared.\n\n* **Result:** Saarthi returns to its initial state, as if freshly installed, with default settings and no user configurations.\n\nUse this option only if you are certain you want to remove all Saarthi data or if instructed during troubleshooting. Consider exporting your settings first if you might want to restore them later.\n" }, "docs/saarthi/features/shell-integration.md": { "content": "import Codicon from '@site/src/components/Codicon';\n\n# Terminal Shell Integration\n\nTerminal Shell Integration is a key feature that enables Saarthi to execute commands in your terminal and intelligently process their output. This bidirectional communication between the AI and your development environment unlocks powerful automation capabilities.\n\n## What is Shell Integration?\n\nShell integration is automatically enabled in Saarthi and connects directly to your terminal's command execution lifecycle without requiring any setup from you. This built-in feature allows Saarthi to:\n\n- Execute commands on your behalf through the [`execute_command`](/advanced-usage/available-tools/execute-command) tool\n- Read command output in real-time without manual copy-pasting\n- Automatically detect and fix errors in running applications\n- Observe command exit codes to determine success or failure\n- Track working directory changes as you navigate your project\n- React intelligently to terminal output without user intervention\n- Stop running commands directly from the chat interface using the stop button that appears next to the command execution message.\n\n![Stop Command Button in Chat UI](/img/v3.15/v3.15.png)\n\nWhen you ask Saarthi to perform tasks like installing dependencies, starting a development server, or analyzing build errors, shell integration works behind the scenes to make these interactions smooth and effective.\n\n## Troubleshooting Shell Integration\n\nShell integration is built into Saarthi and works automatically in most cases. If you see \"Shell Integration Unavailable\" messages or experience issues with command execution, try these solutions:\n\n1. **Update VSCode/Cursor** to the latest version (VSCode 1.93+ required)\n2. **Ensure a compatible shell is selected**: Command Palette (`Ctrl+Shift+P` or `Cmd+Shift+P`) → \"Terminal: Select Default Profile\" → Choose bash, zsh, PowerShell, or fish\n3. **Windows PowerShell users**: Run `Set-ExecutionPolicy RemoteSigned -Scope CurrentUser` then restart VSCode\n4. **WSL users**: Add `. \"$(code --locate-shell-integration-path bash)\"` to your `~/.bashrc`\n\n## Command Execution Fallback\n\nSaarthi has a fallback mechanism for executing commands. This is most relevant if you have chosen to use VS Code's terminal integration (by UNCHECKING the [`Disable terminal shell integration`](#disable-terminal-shell-integration) setting) and that integration then fails.\n\n- **How it works**: If Saarthi is configured to use VS Code's terminal integration but cannot connect or encounters issues, it may automatically attempt to execute the command directly using a background process. This is a fallback to ensure the command still attempts to run.\n- **Notification**: You might see a notification in the chat if this fallback is used, indicating that the command is running without the full features of either Saarthi's inline terminal or VS Code's shell integration (e.g., real-time output streaming or precise exit code detection might be limited).\n![Command execution fallback notification example](/img/v3.15.0/v3.15.0.png)\n\n- **Resolution**: If you encounter this fallback, it typically indicates an issue with your VS Code shell integration setup. Review the troubleshooting steps in this document or consider using Saarthi's recommended inline terminal by ensuring the [`Disable terminal shell integration`](#disable-terminal-shell-integration) setting is CHECKED.\n\n![Saarthi's recommended inline terminal in action](/img/shell-integration/shell-integration-12.png)\n*Example of Saarthi's recommended inline terminal.*\n\n\n## Terminal Integration Settings\n\nSaarthi provides settings to fine-tune how it interacts with terminals. To access these settings:\n1. Click the icon in the top-right corner of the Saarthi sidebar.\n2. In the settings pane that opens, select the \"Terminal\" group from the left-hand menu.\n\n### Basic Settings\n\n#### Terminal Output Limit\n![Terminal output limit slider set to 500](/img/shell-integration/shell-integration.png)\nThis setting controls how much output Saarthi captures from your commands. Consider lowering it if you're concerned about token usage or if Saarthi seems slow processing very long outputs (you'll still get the beginning and end). Consider increasing it if you frequently need more middle content from long commands directly in Saarthi's context, but be mindful of potential token costs. Default: 500 lines.\n\n#### Compress progress bar output\n![Compress progress bar output checkbox](/img/shell-integration/shell-integration-10.png)\nKeep this enabled (default) for cleaner output and token savings. It makes Saarthi process dynamic output like progress bars or spinners more like a real terminal, showing only the final state. Disable this only in rare cases where you specifically need to debug the intermediate, raw output of a progress bar or similar dynamic display.\n\n### Advanced Settings\n\n:::info Important\n**Terminal restart required for these settings**\n\nChanges to advanced terminal settings only take effect after restarting your terminals. To restart a terminal:\n\n1. Click the trash icon in the terminal panel to close the current terminal\n2. Open a new terminal with Terminal → New Terminal or Ctrl+` (backtick)\n\nAlways restart all open terminals after changing any of these settings.\n:::\n\n#### Inherit environment variables\n![Inherit environment variables checkbox](/img/shell-integration/shell-integration-11.png)\nThis setting controls whether Saarthi's terminal sessions use the same environment variables (like `PATH`, API keys, etc.) as your main VSCode/Cursor environment. It directly mirrors the VSCode global setting [`terminal.integrated.inheritEnv`](https://code.visualstudio.com/docs/editor/integrated-terminal#_inherit-environment-variables). Keep this enabled (default for VSCode) if you want Saarthi commands to operate with the same context and tools available in your regular VSCode terminal. Consider disabling it only if you need a completely clean, isolated environment for Saarthi's terminal tasks or are troubleshooting complex environment variable conflicts.\n\n### Runtime Environment\nOn macOS (and possibly other operating systems) the environment provided to VSCode, and consequently Saarthi, can differ depending on how VSCode is launched. \nIf launched from the command line `vscode` command, VSCode and Saarthi will inherit the environment from the shell that launched it, and all will (usually) be well.\nIf launched from the Finder, Dock, or Spotlight, environment exported from `.zshrc`, or `.zprofile` will likely be missing. If you have environment variables set in one of those files, and find they are missing when running VSCode, move them to .zshenv, and log out and back in again, so the window manager will pick up the new environment settings.\n\n#### Disable terminal shell integration\n![Disable terminal shell integration checkbox](/img/shell-integration/shell-integration-9.png)\nThis setting determines how Saarthi executes terminal commands.\n- **Keep this checkbox CHECKED (recommended):** Saarthi will execute commands using its built-in inline terminal, displaying output directly within the chat interface. This method is generally robust, provides clear output, and is the preferred way for most users to interact with terminal commands through Saarthi. It ensures commands run in a consistent environment managed by Saarthi.\n\n ![Saarthi's inline terminal with 'Disable terminal shell integration' CHECKED](/img/shell-integration/shell-integration-12.png)\n *Saarthi's inline terminal, active when \"Disable terminal shell integration\" is CHECKED.*\n\n- **UNCHECK this checkbox (to use VS Code's terminal integration):** Saarthi will attempt to run commands directly within your active VS Code terminal panel. This alternative method might be useful for specific edge cases where you explicitly need commands to run within your fully customized VS Code shell environment or require interaction with the VS Code terminal's specific features for a command. However, this can sometimes be less reliable depending on your shell setup and VS Code version.\n\nThe following settings are advanced options that apply **only if you have UNCHECKED 'Disable terminal shell integration'** (choosing to use VS Code's terminal integration instead of Saarthi's recommended inline terminal):\n\n##### Terminal shell integration timeout\n![Terminal shell integration timeout slider set to 15s](/img/shell-integration/shell-integration-1.png)\nIf shell integration is enabled but you still see 'Shell Integration Unavailable,' especially with complex shell setups (e.g., Zsh with many plugins, or a slow-loading corporate environment), your shell might be taking too long to initialize. Increase this value to give your shell more time to signal its readiness to Saarthi. Try increments of 5-10 seconds. Default: 15s (as shown in UI).\n\n##### Terminal command delay\n![Terminal command delay slider set to 0ms](/img/shell-integration/shell-integration-2.png)\nIf command output appears incomplete or Saarthi seems to miss the end of a command's output (even with shell integration enabled), a small delay might help. Introduce a small delay (e.g., 50ms or 100ms). This gives the terminal more time to flush all output before Saarthi considers the command complete. This is a workaround for potential timing issues in VSCode's terminal or certain shells (see VSCode bug [#237208](https://github.com/microsoft/vscode/issues/237208)). Default: 0ms.\n\n##### Enable PowerShell counter workaround\n![Enable PowerShell counter workaround checkbox](/img/shell-integration/shell-integration-3.png)\nSpecific to PowerShell users. Enable this if you find Saarthi struggles to run the *exact same PowerShell command multiple times in a row*, or if output capture from PowerShell commands is unreliable. This adds a unique counter to commands to help PowerShell differentiate them.\n\n##### Clear ZSH EOL mark\n![Clear ZSH EOL mark checkbox](/img/shell-integration/shell-integration-4.png)\nSpecific to Zsh users. Zsh sometimes adds a special character (often `%`) at the end of a line if it doesn't end with a newline. Enable this if Saarthi seems to misinterpret or get confused by the output of Zsh commands, particularly if the last line of output appears to include an unexpected character. This attempts to remove that marker (`PROMPT_EOL_MARK=''`).\n\n##### Enable Oh My Zsh integration\n![Enable Oh My Zsh integration checkbox](/img/shell-integration/shell-integration-5.png)\nFor users of the popular Oh My Zsh framework for Zsh. Enable this if you use Oh My Zsh and experience general issues with terminal command execution or output rendering that aren't solved by other settings. This helps Saarthi align with Oh My Zsh's specific shell integration mechanisms by setting `ITERM_SHELL_INTEGRATION_INSTALLED=Yes`. Restarting the IDE might be necessary.\n\n##### Enable Powerlevel10k integration\n![Enable Powerlevel10k integration checkbox](/img/shell-integration/shell-integration-6.png)\nFor users of the Powerlevel10k theme for Zsh. Enable this if your Powerlevel10k prompt (which can be quite complex) seems to interfere with Saarthi's ability to correctly detect command boundaries, parse output, or track the current working directory. This sets `POWERLEVEL9K_TERM_SHELL_INTEGRATION=true`.\n\n##### Enable ZDOTDIR handling\n![Enable ZDOTDIR handling checkbox](/img/shell-integration/shell-integration-7.png)\nAn advanced option for Zsh users with customized Zsh startup file locations. Enable this if you use `ZDOTDIR` to specify a custom directory for your Zsh configuration files (like `.zshrc`). This setting helps Saarthi work correctly with such setups by creating an isolated, temporary `ZDOTDIR` for its own integration scripts, preventing conflicts with your personal Zsh environment.\n\n![Fish Cygwin Integration Example](/img/shell-integration/shell-integration-8.png)\n\n## Troubleshooting Resources\n\n### Checking Debug Logs\nWhen shell integration issues occur, check the debug logs:\n1. Open Help → Toggle Developer Tools → Console\n2. Set \"Show All Levels\" to see all log messages\n3. Look for messages containing `[Terminal Process]`\n4. Check `preOutput` content in error messages:\n - Empty preOutput (`''`) means VSCode sent no data\n - This indicates a potential VSCode shell integration issue, or an upstream bug that is out of our control\n - The absence of shell integration markers may require adjusting settings to work around possible upstream bugs or local workstation configuration issues related to shell initialization and VSCode's loading of special shell integration hooks\n\n### Using the VSCode Terminal Integration Test Extension\nThe [VSCode Terminal Integration Test Extension](https://github.com/KJ7LNW/vsce-test-terminal-integration) helps diagnose shell integration issues by testing different settings combinations:\n\n\n1. **When Commands Stall**:\n - If you see \"command already running\" warnings, click \"Reset Stats\" to reset the terminal state\n - These warnings indicate shell integration is not working\n - Try different settings combinations until you find one that works\n - If it really gets stuck, restart the extension by closing the window and pressing F5\n\n2. **Testing Settings**:\n - Systematically try different combinations of:\n * Terminal Command Delay\n * Shell Integration settings\n - Document which combinations succeed or fail\n - This helps identify patterns in shell integration issues\n\n3. **Reporting Issues**:\n - Once you find a problematic configuration\n - Document the exact settings combination\n - Note your environment (OS, VSCode version, shell, and any shell prompt customization)\n - Open an issue with these details to help improve shell integration\n\n### Additional Resources\n\n- [VSCode Terminal Output Issue #237208](https://github.com/microsoft/vscode/issues/237208)\n- [VSCode Terminal Integration Test Repository](https://github.com/KJ7LNW/vsce-test-terminal-integration)\n\n\n## Support\n\nIf you're still having issues:\n\n1. Check [Saarthi GitHub Issues](https://github.com/godspeedsystems/Saarthi/issues)\n2. Create a new issue with:\n - OS and VSCode version\n - Shell type\n - Steps to reproduce\n - Error messages\n\n\n" }, "docs/saarthi/features/suggested-responses.md": { "content": "---\nsidebar_label: Suggested Responses\n---\n\nimport Codicon from '@site/src/components/Codicon';\n\n# Suggested Responses\n\nWhen Saarthi needs more information to complete a task, it uses the [`ask_followup_question` tool](/advanced-usage/available-tools/ask-followup-question). To make responding easier and faster, Saarthi often provides suggested answers alongside the question.\n\n## Overview\n\nSuggested Responses appear as clickable buttons directly below Saarthi's question in the chat interface. They offer pre-formulated answers relevant to the question, helping you provide input quickly.\n\n![Example of Saarthi asking a question with suggested response buttons below it](/img/suggested-responses/suggested-responses.png)\n\n## How It Works\n\n1. **Question Appears**: Saarthi asks a question using the `ask_followup_question` tool.\n2. **Suggestions Displayed**: If suggestions are provided by Saarthi, they appear as buttons below the question.\n3. **Interaction**: You can interact with these suggestions in two ways.\n\n## Interacting with Suggestions\n\nYou have three options for using suggested responses:\n\n1. **Direct Selection**:\n * **Action**: Simply click the button containing the answer you want to provide.\n * **Result**: The selected answer is immediately sent back to Saarthi as your response. This is the quickest way to reply if one of the suggestions perfectly matches your intent.\n\n2. **Keyboard Shortcut**:\n * **Action**: Use the `saarthi.acceptInput` command with your configured keyboard shortcut.\n * **Result**: The primary (first) suggestion button is automatically selected.\n * **Note**: For setup details, see [Keyboard Shortcuts](/features/keyboard-shortcuts).\n\n3. **Edit Before Sending**:\n * **Action**:\n * Hold down `Shift` and click the suggestion button.\n * *Alternatively*, hover over the suggestion button and click the pencil icon () that appears.\n * **Result**: The text of the suggestion is copied into the chat input box. You can then modify the text as needed before pressing Enter to send your customized response. This is useful when a suggestion is close but needs minor adjustments.\n\n![Chat input box showing text copied from a suggested response, ready for editing](/img/suggested-responses/suggested-responses-1.png)\n\n## Benefits\n\n* **Speed**: Quickly respond without typing full answers.\n* **Clarity**: Suggestions often clarify the type of information Saarthi needs.\n* **Flexibility**: Edit suggestions to provide precise, customized answers when needed.\n\nThis feature streamlines the interaction when Saarthi requires clarification, allowing you to guide the task effectively with minimal effort.\n" }, "docs/saarthi/getting-started/your-first-task.md": { "content": "---\nsidebar_label: Your First Task\n---\n\n\n# Starting Your First Task with Saarthi\n\nNow that you've [configured your AI provider and model](connecting-api-provider.md), you're ready to start using Saarthi! This guide walks you through your first interaction.\n\n## Step 1: Open the Saarthi Panel\n\nClick the Saarthi icon in the VS Code Activity Bar (vertical bar on the side of the window) to open the chat interface. If you don't see the icon, verify the extension is installed and enabled.\n\n*The Saarthi icon in the Activity Bar opens the chat interface.*\n\n## Step 2: Type Your Task\n\nType a clear, concise description of what you want Saarthi to do in the chat box at the bottom of the panel. Examples of effective tasks:\n\n* \"Create a file named `hello.txt` containing 'Hello, world!'.\"\n* \"Write a Python function that adds two numbers.\"\n* \"Create an HTML file for a simple website with the title 'Saarthi test'\"\n\nNo special commands or syntax needed—just use plain English.\n\n*Enter your task in natural language - no special syntax required.*\n\n## Step 3: Send Your Task\n\nPress Enter or click the Send icon to the right of the input box.\n\n## Step 4: Review and Approve Actions\n\nSaarthi analyzes your request and proposes specific actions. These may include:\n\n* **Reading files:** Shows file contents it needs to access\n* **Writing to files:** Displays a diff with proposed changes (added lines in green, removed in red)\n* **Executing commands:** Shows the exact command to run in your terminal\n* **Using the Browser:** Outlines browser actions (click, type, etc.)\n* **Asking questions:** Requests clarification when needed to proceed\n\n*Saarthi shows exactly what action it wants to perform and waits for your approval.*\n\n**Each action requires your explicit approval** (unless auto-approval is enabled):\n\n* **Approve:** Click the \"Approve\" button to execute the proposed action\n* **Reject:** Click the \"Reject\" button and provide feedback if needed\n\n## Step 5: Iterate\n\nSaarthi works iteratively. After each action, it waits for your feedback before proposing the next step. Continue this review-approve cycle until your task is complete.\n\n*After completing the task, Saarthi shows the final result and awaits your next instruction.*\n\n## Conclusion\n\nYou've now completed your first task with Saarthi! Through this process, you've learned:\n\n* How to interact with Saarthi using natural language\n* The approval-based workflow that keeps you in control\n* The iterative approach Saarthi uses to solve problems step-by-step\n\nThis iterative, approval-based workflow is at the core of how Saarthi works—letting AI handle the tedious parts of coding while you maintain complete oversight. Now that you understand the basics, you're ready to tackle more complex tasks, explore different [modes](../basic-usage/using-modes) for specialized workflows, or try the [auto-approval feature](../features/auto-approving-actions) to speed up repetitive tasks.\n" }, "docs/saarthi/providers/anthropic.md": { "content": "---\nsidebar_label: Anthropic\n---\n\nimport Codicon from '@site/src/components/Codicon';\n\n# Using Anthropic With Saarthi\n\nAnthropic is an AI safety and research company that builds reliable, interpretable, and steerable AI systems. Their Claude models are known for their strong reasoning abilities, helpfulness, and honesty.\n\n**Website:** [https://www.anthropic.com/](https://www.anthropic.com/)\n\n## Getting an API Key\n\n1. **Sign Up/Sign In:** Go to the [Anthropic Console](https://console.anthropic.com/). Create an account or sign in.\n2. **Navigate to API Keys:** Go to the [API keys](https://console.anthropic.com/settings/keys) section.\n3. **Create a Key:** Click \"Create Key\". Give your key a descriptive name (e.g., \"Saarthi\").\n4. **Copy the Key:** **Important:** Copy the API key *immediately*. You will not be able to see it again. Store it securely.\n\n## Supported Models\n\nSaarthi supports the following Anthropic Claude models:\n\n* `claude-opus-4-20250514`\n* `claude-opus-4-20250514:thinking` (Extended Thinking variant)\n* `claude-sonnet-4-20250514` (Recommended)\n* `claude-sonnet-4-20250514:thinking` (Extended Thinking variant)\n* `claude-3-7-sonnet-20250219`\n* `claude-3-7-sonnet-20250219:thinking` (Extended Thinking variant)\n* `claude-3-5-sonnet-20241022`\n* `claude-3-5-haiku-20241022`\n* `claude-3-opus-20240229`\n* `claude-3-haiku-20240307`\n\nSee [Anthropic's Model Documentation](https://docs.anthropic.com/en/docs/about-claude/models) for more details on each model's capabilities.\n\n## Configuration in Saarthi\n\n1. **Open Saarthi Settings:** Click the gear icon () in the Saarthi panel.\n2. **Select Provider:** Choose \"Anthropic\" from the \"API Provider\" dropdown.\n3. **Enter API Key:** Paste your Anthropic API key into the \"Anthropic API Key\" field.\n4. **Select Model:** Choose your desired Claude model from the \"Model\" dropdown.\n5. **(Optional) Custom Base URL:** If you need to use a custom base URL for the Anthropic API, check \"Use custom base URL\" and enter the URL. Most people won't need to adjust this.\n\n## Tips and Notes\n\n* **Prompt Caching:** Claude 3 models support [prompt caching](https://docs.anthropic.com/en/docs/build-with-claude/prompt-caching), which can significantly reduce costs and latency for repeated prompts.\n* **Context Window:** Claude models have large context windows (200,000 tokens), allowing you to include a significant amount of code and context in your prompts.\n* **Pricing:** Refer to the [Anthropic Pricing](https://www.anthropic.com/pricing) page for the latest pricing information.\n* **Rate Limits:** Anthropic has strict rate limits based on [usage tiers](https://docs.anthropic.com/en/api/rate-limits#requirements-to-advance-tier). If you're repeatedly hitting rate limits, consider contacting Anthropic sales or accessing Claude through a different provider like [OpenRouter](/providers/openrouter) or [Requesty](/providers/requesty).\n" }, "docs/saarthi/providers/bedrock.md": { "content": "---\nsidebar_label: AWS Bedrock\n---\nimport Codicon from '@site/src/components/Codicon';\n\n# Using AWS Bedrock With Saarthi\n\nSaarthi supports accessing models through Amazon Bedrock, a fully managed service that makes a selection of high-performing foundation models (FMs) from leading AI companies available via a single API.\n\n**Website:** [https://aws.amazon.com/bedrock/](https://aws.amazon.com/bedrock/)\n\n## Prerequisites\n\n* **AWS Account:** You need an active AWS account.\n* **Bedrock Access:** You must request and be granted access to Amazon Bedrock. See the [AWS Bedrock documentation](https://docs.aws.amazon.com/bedrock/latest/userguide/getting-started.html) for details on requesting access.\n* **Model Access:** Within Bedrock, you need to request access to the specific models you want to use (e.g., Anthropic Claude).\n* **Install AWS CLI:** Use AWS CLI to configure your account for authentication\n ```bash\n aws configure\n ```\n\n## Getting Credentials\n\nYou have two main options for configuring AWS credentials:\n\n1. **AWS Access Keys (Recommended for Development):**\n * Create an IAM user with the necessary permissions (at least `bedrock:InvokeModel`).\n * Generate an access key ID and secret access key for that user.\n * *(Optional)* Create a session token if required by your IAM configuration.\n2. **AWS Profile:**\n * Configure an AWS profile using the AWS CLI or by manually editing your AWS credentials file. See the [AWS CLI documentation](https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-profiles.html) for details.\n\n## Supported Models\n\nSaarthi supports the following models through Bedrock (based on source code):\n\n* **Amazon:**\n * `amazon.nova-pro-v1:0`\n * `amazon.nova-pro-latency-optimized-v1:0`\n * `amazon.nova-lite-v1:0`\n * `amazon.nova-micro-v1:0`\n * `amazon.titan-text-lite-v1:0`\n * `amazon.titan-text-express-v1:0`\n * `amazon.titan-text-embeddings-v1:0`\n * `amazon.titan-text-embeddings-v2:0`\n* **Anthropic:**\n * `anthropic.claude-opus-4-20250514-v1:0`\n * `anthropic.claude-sonnet-4-20250514-v1:0`\n * `anthropic.claude-3-7-sonnet-20250219-v1:0`\n * `anthropic.claude-3-5-sonnet-20241022-v2:0`\n * `anthropic.claude-3-5-haiku-20241022-v1:0`\n * `anthropic.claude-3-5-sonnet-20240620-v1:0`\n * `anthropic.claude-3-opus-20240229-v1:0`\n * `anthropic.claude-3-sonnet-20240229-v1:0`\n * `anthropic.claude-3-haiku-20240307-v1:0`\n * `anthropic.claude-2-1-v1:0`\n * `anthropic.claude-2-0-v1:0`\n * `anthropic.claude-instant-v1:0`\n* **DeepSeek:**\n * `deepseek.r1-v1:0`\n* **Meta:**\n * `meta.llama3-3-70b-instruct-v1:0`\n * `meta.llama3-2-90b-instruct-v1:0`\n * `meta.llama3-2-11b-instruct-v1:0`\n * `meta.llama3-2-3b-instruct-v1:0`\n * `meta.llama3-2-1b-instruct-v1:0`\n * `meta.llama3-1-405b-instruct-v1:0`\n * `meta.llama3-1-70b-instruct-v1:0`\n * `meta.llama3-1-70b-instruct-latency-optimized-v1:0`\n * `meta.llama3-1-8b-instruct-v1:0`\n * `meta.llama3-70b-instruct-v1:0`\n * `meta.llama3-8b-instruct-v1:0`\n\nRefer to the [Amazon Bedrock documentation](https://docs.aws.amazon.com/bedrock/latest/userguide/models-supported.html) for the most up-to-date list of available models and their IDs. Make sure to use the *model ID* when configuring Saarthi, not the model name.\n\n## Configuration in Saarthi\n\n1. **Open Saarthi Settings:** Click the gear icon () in the Saarthi panel.\n2. **Select Provider:** Choose \"Bedrock\" from the \"API Provider\" dropdown.\n3. **Select Authentication Method:**\n * **AWS Credentials:**\n * Enter your \"AWS Access Key\" and \"AWS Secret Key.\"\n * (Optional) Enter your \"AWS Session Token\" if you're using temporary credentials.\n * **AWS Profile:**\n * Enter your \"AWS Profile\" name (e.g., \"default\").\n4. **Select Region:** Choose the AWS region where your Bedrock service is available (e.g., \"us-east-1\").\n5. **(Optional) Cross-Region Inference:** Check \"Use cross-region inference\" if you want to access models in a region different from your configured AWS region.\n6. **(Optional) VPC Endpoint:** For enterprise environments:\n * Check \"Use VPC Endpoint\" to route all Bedrock API calls through your VPC endpoint\n * Enter your VPC endpoint URL in the text field that appears\n * This ensures all LLM transactions remain within your corporate network\n7. **Select Model:** Choose your desired model from the \"Model\" dropdown.\n\n## Tips and Notes\n\n* **Permissions:** Ensure your IAM user or role has the necessary permissions to invoke Bedrock models. The `bedrock:InvokeModel` permission is required.\n* **Pricing:** Refer to the [Amazon Bedrock pricing](https://aws.amazon.com/bedrock/pricing/) page for details on model costs.\n* **Cross-Region Inference:** Using cross-region inference may result in higher latency.\n* **VPC Endpoints:** When using VPC endpoints, ensure your endpoint is properly configured to handle Bedrock API calls. This feature is particularly useful for organizations with strict security requirements that mandate keeping all API traffic within their private network.\n" }, "docs/saarthi/providers/chutes.md": { "content": "---\nsidebar_label: Chutes AI\n---\n\nimport Codicon from '@site/src/components/Codicon';\n\n# Using Chutes AI With Saarthi\n\nChutes.ai offers free API access to several large language models (LLMs), allowing developers to integrate and experiment with these models without immediate financial commitment. They provide access to a curated set of open-source and proprietary language models, often with a focus on specific capabilities or regional language support.\n\n**Website:** [https://chutes.ai/](https://chutes.ai/)\n\n## Getting an API Key\n\nTo use Chutes AI with Saarthi, obtain an API key from the [Chutes AI platform](https://chutes.ai/). After signing up or logging in, you should find an option to generate or retrieve your API key within your account dashboard or settings.\n\n## Supported Models\n\nSaarthi will attempt to fetch the list of available models from the Chutes AI API. The specific models available will depend on Chutes AI's current offerings.\n\nAlways refer to the official Chutes AI documentation or your dashboard for the most up-to-date list of supported models.\n\n## Configuration in Saarthi\n\n1. **Open Saarthi Settings:** Click the gear icon () in the Saarthi panel.\n2. **Select Provider:** Choose \"Chutes AI\" from the \"API Provider\" dropdown.\n3. **Enter API Key:** Paste your Chutes AI API key into the \"Chutes AI API Key\" field.\n4. **Select Model:** Choose your desired model from the \"Model\" dropdown.\n" }, "docs/saarthi/providers/deepseek.md": { "content": "---\nsidebar_label: DeepSeek\n---\nimport Codicon from '@site/src/components/Codicon';\n\n\n# Using DeepSeek With Saarthi\n\nSaarthi supports accessing models through the DeepSeek API, including `deepseek-chat` and `deepseek-reasoner`.\n\n**Website:** [https://platform.deepseek.com/](https://platform.deepseek.com/)\n\n## Getting an API Key\n\n1. **Sign Up/Sign In:** Go to the [DeepSeek Platform](https://platform.deepseek.com/). Create an account or sign in.\n2. **Navigate to API Keys:** Find your API keys in the [API keys](https://platform.deepseek.com/api_keys) section of the platform.\n3. **Create a Key:** Click \"Create new API key\". Give your key a descriptive name (e.g., \"Saarthi\").\n4. **Copy the Key:** **Important:** Copy the API key *immediately*. You will not be able to see it again. Store it securely.\n\n## Supported Models\n\nSaarthi supports the following DeepSeek models:\n\n* `deepseek-chat` (Recommended for coding tasks)\n*\t `deepseek-reasoner` (Recommended for reasoning tasks)\n\n## Configuration in Saarthi\n\n1. **Open Saarthi Settings:** Click the gear icon () in the Saarthi panel.\n2. **Select Provider:** Choose \"DeepSeek\" from the \"API Provider\" dropdown.\n3. **Enter API Key:** Paste your DeepSeek API key into the \"DeepSeek API Key\" field.\n4. **Select Model:** Choose your desired model from the \"Model\" dropdown.\n\n## Tips and Notes\n* **Pricing:** Refer to the [DeepSeek Pricing](https://api-docs.deepseek.com/quick_start/pricing/) page for details on model costs.\n" }, "docs/saarthi/providers/gemini.md": { "content": "---\nsidebar_label: Google Gemini\n---\nimport Codicon from '@site/src/components/Codicon';\n\n\n# Using Google Gemini With Saarthi\n\nSaarthi supports Google's Gemini family of models through the Google AI Gemini API.\n\n**Website:** [https://ai.google.dev/](https://ai.google.dev/)\n\n## Getting an API Key\n\n1. **Go to Google AI Studio:** Navigate to [https://ai.google.dev/](https://ai.google.dev/).\n2. **Sign In:** Sign in with your Google account.\n3. **Create API Key:** Click on \"Create API key\" in the left-hand menu.\n4. **Copy API Key:** Copy the generated API key.\n\n## Supported Models\n\nSaarthi supports the following Gemini models:\n\n* `gemini-2.5-flash-preview-05-20`\n* `gemini-2.5-pro-exp-03-25`\n* `gemini-2.0-flash-001`\n* `gemini-2.0-flash-lite-preview-02-05`\n* `gemini-2.0-pro-exp-02-05`\n* `gemini-2.0-flash-thinking-exp-01-21`\n* `gemini-2.0-flash-thinking-exp-1219`\n* `gemini-2.0-flash-exp`\n* `gemini-1.5-flash-002`\n* `gemini-1.5-flash-exp-0827`\n* `gemini-1.5-flash-8b-exp-0827`\n* `gemini-1.5-pro-002`\n* `gemini-1.5-pro-exp-0827`\n* `gemini-exp-1206`\n\nRefer to the [Gemini documentation](https://ai.google.dev/models/gemini) for more details on each model.\n\n## Configuration in Saarthi\n\n1. **Open Saarthi Settings:** Click the gear icon () in the Saarthi panel.\n2. **Select Provider:** Choose \"Google Gemini\" from the \"API Provider\" dropdown.\n3. **Enter API Key:** Paste your Gemini API key into the \"Gemini API Key\" field.\n4. **Select Model:** Choose your desired Gemini model from the \"Model\" dropdown.\n\n## Tips and Notes\n\n* **Pricing:** Gemini API usage is priced based on input and output tokens. Refer to the [Gemini pricing page](https://ai.google.dev/pricing) for detailed information.\n" }, "docs/saarthi/providers/glama.md": { "content": "---\nsidebar_label: Glama\n---\n\nimport Codicon from '@site/src/components/Codicon';\n\n# Using Glama With Saarthi\n\nGlama provides access to a variety of language models through a unified API, including models from Anthropic, OpenAI, and others. It offers features like prompt caching and cost tracking.\n\n**Website:** [https://glama.ai/](https://glama.ai/)\n\n## Getting an API Key\n\n1. **Sign Up/Sign In:** Go to the [Glama sign-up page](https://glama.ai/sign-up). Sign up using your Google account or name/email/password.\n2. **Get API Key:** After signing up, navigate to the [API Keys](https://glama.ai/settings/gateway/api-keys) page to get an API key.\n3. **Copy the Key:** Copy the displayed API key.\n\n## Supported Models\n\nSaarthi will automatically try to fetch a list of available models from the Glama API. Some models that are commonly available through Glama include:\n\n* **Anthropic Claude models:** (e.g., `anthropic/claude-3-5-sonnet`) These are generally recommended for best performance with Saarthi.\n* **OpenAI models:** (e.g., `openai/o3-mini-high`)\n* **Other providers and open-source models**\n \nRefer to the [Glama documentation](https://glama.ai/models) for the most up-to-date list of supported models.\n\n## Configuration in Saarthi\n\n1. **Open Saarthi Settings:** Click the gear icon () in the Saarthi panel.\n2. **Select Provider:** Choose \"Glama\" from the \"API Provider\" dropdown.\n3. **Enter API Key:** Paste your Glama API key into the \"Glama API Key\" field.\n4. **Select Model:** Choose your desired model from the \"Model\" dropdown.\n\n## Tips and Notes\n\n* **Pricing:** Glama operates on a pay-per-use basis. Pricing varies depending on the model you choose.\n* **Prompt Caching:** Glama supports prompt caching, which can significantly reduce costs and improve performance for repeated prompts.\n" }, "docs/saarthi/providers/groq.md": { "content": "---\nsidebar_label: Groq\n---\n\nimport Codicon from '@site/src/components/Codicon';\n\n# Using Groq With Saarthi\n\nGroq specializes in providing very high-speed inference for large language models, utilizing their custom-built Language Processing Units (LPUs). This can result in significantly faster response times for supported models.\n\n**Website:** [https://groq.com/](https://groq.com/)\n\n## Getting an API Key\n\nTo use Groq with Saarthi, you'll need an API key from the [GroqCloud Console](https://console.groq.com/). After signing up or logging in, navigate to the API Keys section of your dashboard to create and copy your key.\n\n## Supported Models\n\nSaarthi will attempt to fetch the list of available models from the Groq API. Common models available via Groq include:\n\n* `llama3-8b-8192`\n* `llama3-70b-8192`\n* `mixtral-8x7b-32768`\n* `gemma-7b-it`\n\nRefer to the [Groq Documentation](https://console.groq.com/docs/models) for the most up-to-date list of supported models and their capabilities.\n\n## Configuration in Saarthi\n\n1. **Open Saarthi Settings:** Click the gear icon () in the Saarthi panel.\n2. **Select Provider:** Choose \"Groq\" from the \"API Provider\" dropdown.\n3. **Enter API Key:** Paste your Groq API key into the \"Groq API Key\" field.\n4. **Select Model:** Choose your desired model from the \"Model\" dropdown.\n" }, "docs/saarthi/providers/human-relay.md": { "content": "import Codicon from '@site/src/components/Codicon';\n\n\n# Human Relay Provider\n\nThe Human Relay provider allows you to use Saarthi with web-based AI models like ChatGPT or Claude without needing an API key. Instead, it relies on you to manually relay messages between Saarthi and the AI's web interface.\n\n## How it Works\n\n1. **Select Human Relay**: Choose \"Human Relay\" as your API provider in Saarthi's settings. No API key is required.\n2. **Initiate a Request**: Start a chat or task with Saarthi as usual.\n3. **Dialog Prompt**: A dialog box will appear in VS Code. Your message to the AI is automatically copied to your clipboard.\n4. **Paste to Web AI**: Go to the web interface of your chosen AI (e.g., chat.openai.com, claude.ai) and paste the message from your clipboard into the chat input.\n5. **Copy AI Response**: Once the AI responds, copy its complete response text.\n6. **Paste Back to Saarthi**: Return to the dialog box in VS Code, paste the AI's response into the designated field, and click \"Confirm\".\n7. **Continue**: Saarthi will process the response as if it came directly from an API.\n\n## Use Cases\n\nThis provider is useful if:\n\n* You want to use models that don't offer direct API access.\n* You prefer not to manage API keys.\n* You need to leverage the specific capabilities or context available only in the web UI of certain AI models.\n\n## Limitations\n\n* **Manual Effort**: Requires constant copy-pasting between VS Code and your browser.\n* **Slower Interaction**: The back-and-forth process is significantly slower than direct API integration.\n* **Potential for Errors**: Manual copying and pasting can introduce errors or omissions.\n\nChoose this provider when the benefits of using a specific web AI outweigh the inconvenience of the manual relay process." }, "docs/saarthi/providers/litellm.md": { "content": "---\nsidebar_label: LiteLLM\n---\n\nimport Codicon from '@site/src/components/Codicon';\n\n# Using LiteLLM With Saarthi\n\nLiteLLM is a versatile tool that provides a unified interface to over 100 Large Language Models (LLMs) by offering an OpenAI-compatible API. This allows you to run a local server that can proxy requests to various model providers or serve local models, all accessible through a consistent API endpoint.\n\n**Website:** [https://litellm.ai/](https://litellm.ai/) (Main project) & [https://docs.litellm.ai/](https://docs.litellm.ai/) (Documentation)\n\n## Key Benefits\n\n* **Unified API:** Access a wide range of LLMs (from OpenAI, Anthropic, Cohere, HuggingFace, etc.) through a single, OpenAI-compatible API.\n* **Local Deployment:** Run your own LiteLLM server locally, giving you more control over model access and potentially reducing latency.\n* **Simplified Configuration:** Manage credentials and model configurations in one place (your LiteLLM server) and let Saarthi connect to it.\n* **Cost Management:** LiteLLM offers features for tracking costs across different models and providers.\n\n## Setting Up Your LiteLLM Server\n\nTo use LiteLLM with Saarthi, you first need to set up and run a LiteLLM server.\n\n1. **Installation:** Follow the official [LiteLLM installation guide](https://docs.litellm.ai/docs/proxy_server) to install LiteLLM and its dependencies.\n2. **Configuration:** Configure your LiteLLM server with the models you want to use. This typically involves setting API keys for the underlying providers (e.g., OpenAI, Anthropic) in your LiteLLM server's configuration.\n3. **Start the Server:** Run your LiteLLM server. By default, it usually starts on `http://localhost:4000`.\n * You can also configure an API key for your LiteLLM server itself for added security.\n\nRefer to the [LiteLLM documentation](https://docs.litellm.ai/docs/) for detailed instructions on server setup, model configuration, and advanced features.\n\n## Configuration in Saarthi\n\nOnce your LiteLLM server is running:\n\n1. **Open Saarthi Settings:** Click the gear icon () in the Saarthi panel.\n2. **Select Provider:** Choose \"LiteLLM\" from the \"API Provider\" dropdown.\n3. **Enter Base URL:**\n * Input the URL of your LiteLLM server.\n * Defaults to `http://localhost:4000` if left blank.\n4. **Enter API Key (Optional):**\n * If you've configured an API key for your LiteLLM server, enter it here.\n * If your LiteLLM server doesn't require an API key, Saarthi will use a default dummy key (`\"dummy-key\"`), which should work fine.\n5. **Select Model:**\n * Saarthi will attempt to fetch the list of available models from your LiteLLM server by querying the `${baseUrl}/v1/model/info` endpoint.\n * The models displayed in the dropdown are sourced from this endpoint.\n * Use the refresh button to update the model list if you've added new models to your LiteLLM server.\n * If no model is selected, Saarthi defaults to `anthropic/claude-3-7-sonnet-20250219` (this is `litellmDefaultModelId`). Ensure this model (or your desired default) is configured and available on your LiteLLM server.\n\n\"Saarthi\n\n## How Saarthi Fetches and Interprets Model Information\n\nWhen you configure the LiteLLM provider, Saarthi interacts with your LiteLLM server to get details about the available models:\n\n* **Model Discovery:** Saarthi makes a GET request to `${baseUrl}/v1/model/info` on your LiteLLM server. If an API key is provided in Saarthi's settings, it's included in the `Authorization: Bearer ${apiKey}` header.\n* **Model Properties:** For each model reported by your LiteLLM server, Saarthi extracts and interprets the following:\n * `model_name`: The identifier for the model.\n * `maxTokens`: Maximum output tokens. Defaults to `8192` if not specified by LiteLLM.\n * `contextWindow`: Maximum context tokens. Defaults to `200000` if not specified by LiteLLM.\n * `supportsImages`: Determined from `model_info.supports_vision` provided by LiteLLM.\n * `supportsPromptCache`: Determined from `model_info.supports_prompt_caching` provided by LiteLLM.\n * `inputPrice` / `outputPrice`: Calculated from `model_info.input_cost_per_token` and `model_info.output_cost_per_token` from LiteLLM.\n * `supportsComputerUse`: This flag is set to `true` if the underlying model identifier (from `litellm_params.model`, e.g., `openrouter/anthropic/claude-3.5-sonnet`) matches one of the Anthropic models predefined in Saarthi as suitable for \"computer use\" (see `COMPUTER_USE_MODELS` in technical details).\n\nSaarthi uses default values for some of these properties if they are not explicitly provided by your LiteLLM server's `/model/info` endpoint for a given model. The defaults are:\n* `maxTokens`: 8192\n* `contextWindow`: 200,000\n* `supportsImages`: `true`\n* `supportsComputerUse`: `true` (for the default model ID)\n* `supportsPromptCache`: `true`\n* `inputPrice`: 3.0 (µUSD per 1k tokens)\n* `outputPrice`: 15.0 (µUSD per 1k tokens)\n\n## Tips and Notes\n\n* **LiteLLM Server is Key:** The primary configuration for models, API keys for downstream providers (like OpenAI, Anthropic), and other advanced features are managed on your LiteLLM server. Saarthi acts as a client to this server.\n* **Model Availability:** The models available in Saarthi's \"Model\" dropdown depend entirely on what your LiteLLM server exposes through its `/v1/model/info` endpoint.\n* **Network Accessibility:** Ensure your LiteLLM server is running and accessible from the machine where VS Code and Saarthi are running (e.g., check firewall rules if not on `localhost`).\n* **Troubleshooting:** If models aren't appearing or requests fail:\n * Verify your LiteLLM server is running and configured correctly.\n * Check the LiteLLM server logs for errors.\n * Ensure the Base URL in Saarthi settings matches your LiteLLM server's address.\n * Confirm any API key required by your LiteLLM server is correctly entered in Saarthi.\n* **Computer Use Models:** The `supportsComputerUse` flag in Saarthi is primarily relevant for certain Anthropic models known to perform well with tool-use and function-calling tasks. If you are routing other models through LiteLLM, this flag might not be automatically set unless the underlying model ID matches the specific Anthropic ones Saarthi recognizes.\n\nBy leveraging LiteLLM, you can significantly expand the range of models accessible to Saarthi while centralizing their management." }, "docs/saarthi/providers/lmstudio.md": { "content": "---\nsidebar_label: LM Studio\n---\nimport Codicon from '@site/src/components/Codicon';\n\n\n# Using LM Studio With Saarthi\n\nSaarthi supports running models locally using LM Studio. LM Studio provides a user-friendly interface for downloading, configuring, and running local language models. It also includes a built-in local inference server that emulates the OpenAI API, making it easy to integrate with Saarthi.\n\n**Website:** [https://lmstudio.ai/](https://lmstudio.ai/)\n\n## Setting Up LM Studio\n\n1. **Download and Install LM Studio:** Download LM Studio from the [LM Studio website](https://lmstudio.ai/).\n2. **Download a Model:** Use the LM Studio interface to search for and download a model. Some recommended models include:\n * CodeLlama models (e.g., `codellama:7b-code`, `codellama:13b-code`, `codellama:34b-code`)\n * Mistral models (e.g., `mistralai/Mistral-7B-Instruct-v0.1`)\n * DeepSeek Coder models (e.g., `deepseek-coder:6.7b-base`)\n * Any other model that is supported by Saarthi, or for which you can set the context window.\n\n Look for models in the GGUF format. LM Studio provides a search interface to find and download models.\n3. **Start the Local Server:**\n * Open LM Studio.\n * Click the **\"Local Server\"** tab (the icon looks like `<->`).\n * Select the model you downloaded.\n * Click **\"Start Server\"**.\n\n## Configuration in Saarthi\n\n1. **Open Saarthi Settings:** Click the gear icon () in the Saarthi panel.\n2. **Select Provider:** Choose \"LM Studio\" from the \"API Provider\" dropdown.\n3. **Enter Model ID:** Enter the *file name* of the model you loaded in LM Studio (e.g., `codellama-7b.Q4_0.gguf`). You can find this in the LM Studio \"Local Server\" tab.\n4. **(Optional) Base URL:** By default, Saarthi will connect to LM Studio at `http://localhost:1234`. If you've configured LM Studio to use a different address or port, enter the full URL here.\n\n## Tips and Notes\n\n* **Resource Requirements:** Running large language models locally can be resource-intensive. Make sure your computer meets the minimum requirements for the model you choose.\n* **Model Selection:** LM Studio provides a wide range of models. Experiment to find the one that best suits your needs.\n* **Local Server:** The LM Studio local server must be running for Saarthi to connect to it.\n* **LM Studio Documentation:** Refer to the [LM Studio documentation](https://lmstudio.ai/docs) for more information.\n* **Troubleshooting:** If you see a \"Please check the LM Studio developer logs to debug what went wrong\" error, you may need to adjust the context length settings in LM Studio.\n* **Token Tracking:** Saarthi tracks token usage for models run via LM Studio, helping you monitor consumption.\n* **Reasoning Support:** For models that support it, Saarthi can parse \"think\" tags or similar reasoning indicators in LM Studio responses, offering more insight into the model's process.\n" }, "docs/saarthi/providers/mistral.md": { "content": "---\nsidebar_label: Mistral AI\n---\n\nimport Codicon from '@site/src/components/Codicon';\n\n# Using Mistral AI With Saarthi\n\nSaarthi supports accessing models through the Mistral AI API, including both standard Mistral models and the code-specialized Codestral model.\n\n**Website:** [https://mistral.ai/](https://mistral.ai/)\n\n## Getting an API Key\n\n1. **Sign Up/Sign In:** Go to the [Mistral Platform](https://console.mistral.ai/). Create an account or sign in. You may need to go through a verification process.\n2. **Create an API Key:** \n - [La Plateforme API Key](https://console.mistral.ai/api-keys/) and/or \n - [Codestral API Key](https://console.mistral.ai/codestral)\n\n## Supported Models\n\nSaarthi supports the following Mistral models:\n\n| Model ID | Model Default Temperature | Function Calling | Vision / Image support |\n|------------------------|-------------------------|------------------|--------|\n| codestral-latest | 0.3 | ✅ | ❌ |\n| mistral-large-latest | 0.7 | ✅ | ❌ |\n| ministral-8b-latest | 0.3 | ✅ | ❌ |\n| ministral-3b-latest | 0.3 | ✅ | ❌ |\n| mistral-small-latest | 0.3 | ✅ | ❌ |\n| pixtral-large-latest | 0.7 | ✅ | ✅ |\nThe default model temperature in Saarthi is 0.0, so you should consider experimenting with [temperature adjustments](/features/model-temperature)!\n\n**Note:** Model availability and specifications may change.\nRefer to the [Mistral AI documentation](https://docs.mistral.ai/api/) and [Mistral Model Overview](https://docs.mistral.ai/getting-started/models/models_overview/) for the latest information.\n\n## Configuration in Saarthi\n\n1. **Open Saarthi Settings:** Click the gear icon () in the Saarthi panel.\n2. **Select Provider:** Choose \"Mistral\" from the \"API Provider\" dropdown.\n3. **Enter API Key:** Paste your Mistral API key into the \"Mistral API Key\" field if you're using a `mistral` model. If you intend to use `codestral-latest`, see the \"Codestral\" section below.\n4. **Select Model:** Choose your desired model from the \"Model\" dropdown. \n\n## Using Codestral\n\n[Codestral](https://docs.mistral.ai/capabilities/code_generation/) is a model specifically designed for code generation and interaction. \nOnly for Codestral you could use different endpoints (Default: codestral.mistral.ai). \nFor the La Platforme API Key change the **Codestral Base Url** to: https://api.mistral.ai \n\nTo use Codestral:\n\n1. **Select \"Mistral\" as the API Provider.**\n2. **Select a Codestral Model**\n3. **Enter your Codestral (codestral.mistral.ai) or La Plateforme (api.mistral.ai) API Key.** \n" }, "docs/saarthi/providers/ollama.md": { "content": "---\nsidebar_label: Ollama\n---\n\nimport Codicon from '@site/src/components/Codicon'; \n\n# Using Ollama With Saarthi\n\nSaarthi supports running models locally using Ollama. This provides privacy, offline access, and potentially lower costs, but requires more setup and a powerful computer.\n\n**Website:** [https://ollama.com/](https://ollama.com/)\n\n## Setting up Ollama\n\n1. **Download and Install Ollama:** Download the Ollama installer for your operating system from the [Ollama website](https://ollama.com/). Follow the installation instructions. Make sure Ollama is running\n\n ```bash\n ollama serve\n ```\n\n2. **Download a Model:** Ollama supports many different models. You can find a list of available models on the [Ollama website](https://ollama.com/library). Some recommended models for coding tasks include:\n\n * `codellama:7b-code` (good starting point, smaller)\n * `codellama:13b-code` (better quality, larger)\n * `codellama:34b-code` (even better quality, very large)\n * `qwen2.5-coder:32b`\n * `mistralai/Mistral-7B-Instruct-v0.1` (good general-purpose model)\n * `deepseek-coder:6.7b-base` (good for coding tasks)\n * `llama3:8b-instruct-q5_1` (good for general tasks)\n\n To download a model, open your terminal and run:\n\n ```bash\n ollama pull \n ```\n\n For example:\n\n ```bash\n ollama pull qwen2.5-coder:32b\n ```\n\n3. **Configure the Model:** by default, Ollama uses a context window size of 2048 tokens, which is too small for Saarthi requests. You need to have at least 12k to get decent results, ideally - 32k. To configure a model, you actually need to set its parameters and save a copy of it.\n\n Load the model (we will use `qwen2.5-coder:32b` as an example):\n \n ```bash\n ollama run qwen2.5-coder:32b\n ```\n\n Change context size parameter:\n\n ```bash\n /set parameter num_ctx 32768\n ```\n\n Save the model with a new name:\n\n ```bash\n /save your_model_name\n ```\n\n4. **Configure Saarthi:**\n * Open the Saarthi sidebar icon.\n \n * Click the settings gear icon ().\n * Select \"ollama\" as the API Provider.\n * Enter the Model name from the previous step (e.g., `your_model_name`).\n * (Optional) You can configure the base URL if you're running Ollama on a different machine. The default is `http://localhost:11434`.\n * (Optional) Configure Model context size in Advanced settings, so Saarthi knows how to manage its sliding window.\n\n## Tips and Notes\n\n* **Resource Requirements:** Running large language models locally can be resource-intensive. Make sure your computer meets the minimum requirements for the model you choose.\n* **Model Selection:** Experiment with different models to find the one that best suits your needs.\n* **Offline Use:** Once you've downloaded a model, you can use Saarthi offline with that model.\n* **Token Tracking:** Saarthi tracks token usage for models run via Ollama, helping you monitor consumption.\n* **Ollama Documentation:** Refer to the [Ollama documentation](https://ollama.com/docs) for more information on installing, configuring, and using Ollama.\n" }, "docs/saarthi/providers/openai-compatible.md": { "content": "---\nsidebar_label: OpenAI Compatible\n---\nimport Codicon from '@site/src/components/Codicon';\n\n\n# Using OpenAI Compatible Providers With Saarthi\n\nSaarthi supports a wide range of AI model providers that offer APIs compatible with the OpenAI API standard. This means you can use models from providers *other than* OpenAI, while still using a familiar API interface. This includes providers like:\n\n* **Local models** running through tools like Ollama and LM Studio (covered in separate sections).\n* **Cloud providers** like Perplexity, Together AI, Anyscale, and others.\n* **Any other provider** offering an OpenAI-compatible API endpoint.\n\nThis document focuses on setting up providers *other than* the official OpenAI API (which has its own [dedicated configuration page](/providers/openai)).\n\n## General Configuration\n\nThe key to using an OpenAI-compatible provider is to configure two main settings:\n\n1. **Base URL:** This is the API endpoint for the provider. It will *not* be `https://api.openai.com/v1` (that's for the official OpenAI API).\n2. **API Key:** This is the secret key you obtain from the provider.\n3. **Model ID:** This is the model name of the specific model.\n\nYou'll find these settings in the Saarthi settings panel (click the icon):\n\n* **API Provider:** Select \"OpenAI Compatible\".\n* **Base URL:** Enter the base URL provided by your chosen provider. **This is crucial.**\n* **API Key:** Enter your API key.\n* **Model:** Choose a model.\n* **Model Configuration:** This lets you customize advanced configuration for the model\n - Max Output Tokens\n - Context Window\n - Image Support\n - Computer Use\n - Input Price\n - Output Price\n\n## Supported Models (for OpenAI Native Endpoint)\n\nWhile this provider type allows connecting to various endpoints, if you are connecting directly to the official OpenAI API (or an endpoint mirroring it exactly), Saarthi recognizes the following model IDs based on the `openAiNativeModels` definition in its source code:\n\n* `o3-mini`\n* `o3-mini-high`\n* `o3-mini-low`\n* `o1`\n* `o1-preview`\n* `o1-mini`\n* `gpt-4.5-preview`\n* `gpt-4o`\n* `gpt-4o-mini`\n\n**Note:** If you are using a different OpenAI-compatible provider (like Together AI, Anyscale, etc.), the available model IDs will vary. Always refer to your specific provider's documentation for their supported model names.\n\n## Troubleshooting\n\n* **\"Invalid API Key\":** Double-check that you've entered the API key correctly.\n* **\"Model Not Found\":** Make sure you're using a valid model ID for your chosen provider.\n* **Connection Errors:** Verify the Base URL is correct and that your provider's API is accessible.\n* **Unexpected Results:** If you're getting unexpected results, try a different model.\n\nBy using an OpenAI-compatible provider, you can leverage the flexibility of Saarthi with a wider range of AI models. Remember to always consult your provider's documentation for the most accurate and up-to-date information.\n" }, "docs/saarthi/providers/openai.md": { "content": "---\nsidebar_label: OpenAI\n---\n\nimport Codicon from '@site/src/components/Codicon';\n\n# Using OpenAI With Saarthi\n\nSaarthi supports accessing models directly through the official OpenAI API.\n\n**Website:** [https://openai.com/](https://openai.com/)\n\n## Getting an API Key\n\n1. **Sign Up/Sign In:** Go to the [OpenAI Platform](https://platform.openai.com/). Create an account or sign in.\n2. **Navigate to API Keys:** Go to the [API keys](https://platform.openai.com/api-keys) page.\n3. **Create a Key:** Click \"Create new secret key\". Give your key a descriptive name (e.g., \"Saarthi\").\n4. **Copy the Key:** **Important:** Copy the API key *immediately*. You will not be able to see it again. Store it securely.\n\n## Supported Models\n\nSaarthi supports a variety of OpenAI models, including:\n\n*\t`o3-mini` (medium reasoning effort)\n*\t`o3-mini-high` (high reasoning effort)\n* `o3-mini-low` (low reasoning effort)\n* `o1`\n* `o1-preview`\n*\t`o1-mini`\n* `gpt-4.5-preview`\n* `gpt-4o`\n* `gpt-4o-mini`\n\nRefer to the [OpenAI Models documentation](https://platform.openai.com/docs/models) for the most up-to-date list of models and capabilities.\n\n## Configuration in Saarthi\n\n1. **Open Saarthi Settings:** Click the gear icon () in the Saarthi panel.\n2. **Select Provider:** Choose \"OpenAI\" from the \"API Provider\" dropdown.\n3. **Enter API Key:** Paste your OpenAI API key into the \"OpenAI API Key\" field.\n4. **Select Model:** Choose your desired model from the \"Model\" dropdown.\n5. **(Optional) Base URL:** If you need to use a custom base URL, enter the URL. Most people won't need to adjust this.\n\n## Tips and Notes\n\n* **Pricing:** Refer to the [OpenAI Pricing](https://openai.com/pricing) page for details on model costs.\n* **Azure OpenAI Service:** If you'd like to use the Azure OpenAI service, please see our section on [OpenAI-compatible](/providers/openai-compatible) providers.\n" }, "docs/saarthi/providers/openrouter.md": { "content": "---\nsidebar_label: OpenRouter\n---\nimport Codicon from '@site/src/components/Codicon';\n\n\n# Using OpenRouter With Saarthi\n\nOpenRouter is an AI platform that provides access to a wide variety of language models from different providers, all through a single API. This can simplify setup and allow you to easily experiment with different models.\n\n**Website:** [https://openrouter.ai/](https://openrouter.ai/)\n\n## Getting an API Key\n\n1. **Sign Up/Sign In:** Go to the [OpenRouter website](https://openrouter.ai/). Sign in with your Google or GitHub account.\n2. **Get an API Key:** Go to the [keys page](https://openrouter.ai/keys). You should see an API key listed. If not, create a new key.\n3. **Copy the Key:** Copy the API key.\n\n## Supported Models\n\nOpenRouter supports a large and growing number of models. Saarthi automatically fetches the list of available models. Refer to the [OpenRouter Models page](https://openrouter.ai/models) for the complete and up-to-date list.\n\n## Configuration in Saarthi\n\n1. **Open Saarthi Settings:** Click the gear icon () in the Saarthi panel.\n2. **Select Provider:** Choose \"OpenRouter\" from the \"API Provider\" dropdown.\n3. **Enter API Key:** Paste your OpenRouter API key into the \"OpenRouter API Key\" field.\n4. **Select Model:** Choose your desired model from the \"Model\" dropdown.\n5. **(Optional) Custom Base URL:** If you need to use a custom base URL for the OpenRouter API, check \"Use custom base URL\" and enter the URL. Leave this blank for most users.\n\n## Supported Transforms\n\nOpenRouter provides an [optional \"middle-out\" message transform](https://openrouter.ai/docs/features/message-transforms) to help with prompts that exceed the maximum context size of a model. You can enable it by checking the \"Compress prompts and message chains to the context size\" box.\n\n## Tips and Notes\n\n* **Model Selection:** OpenRouter offers a wide range of models. Experiment to find the best one for your needs.\n* **Pricing:** OpenRouter charges based on the underlying model's pricing. See the [OpenRouter Models page](https://openrouter.ai/models) for details.\n* **Prompt Caching:**\n * OpenRouter passes caching requests to underlying models that support it. Check the [OpenRouter Models page](https://openrouter.ai/models) to see which models offer caching.\n * For most models, caching should activate automatically if supported by the model itself (similar to how Requesty works).\n * **Exception for Gemini Models via OpenRouter:** Due to potential response delays sometimes observed with Google's caching mechanism when accessed via OpenRouter, a manual activation step is required *specifically for Gemini models*.\n * If using a **Gemini model** via OpenRouter, you **must manually check** the \"Enable Prompt Caching\" box in the provider settings to activate caching for that model. This checkbox serves as a temporary workaround. For non-Gemini models on OpenRouter, this checkbox is not necessary for caching.\n" }, "docs/saarthi/providers/requesty.md": { "content": "---\nsidebar_label: Requesty\n---\n\nimport Codicon from '@site/src/components/Codicon';\n\n# Using Requesty With Saarthi\n\nSaarthi supports accessing models through the [Requesty](https://www.requesty.ai/) AI platform. Requesty provides an easy and optimized API for interacting with 150+ large language models (LLMs).\n\n**Website:** [https://www.requesty.ai/](https://www.requesty.ai/)\n\n## Getting an API Key\n\n1. **Sign Up/Sign In:** Go to the [Requesty website](https://www.requesty.ai/) and create an account or sign in.\n2. **Get API Key:** You can get an API key from the [API Management](https://app.requesty.ai/manage-api) section of your Requesty dashboard.\n\n## Supported Models\n\nRequesty provides access to a wide range of models. Saarthi will automatically fetch the latest list of available models. You can see the full list of available models on the [Model List](https://app.requesty.ai/router/list) page.\n\n## Configuration in Saarthi\n\n1. **Open Saarthi Settings:** Click the gear icon () in the Saarthi panel.\n2. **Select Provider:** Choose \"Requesty\" from the \"API Provider\" dropdown.\n3. **Enter API Key:** Paste your Requesty API key into the \"Requesty API Key\" field.\n4. **Select Model:** Choose your desired model from the \"Model\" dropdown.\n\n## Tips and Notes\n\n- **Optimizations**: Requesty offers range of in-flight cost optimizations to lower your costs.\n- **Unified and simplified billing**: Unrestricted access to all providers and models, automatic balance top ups and more via a single [API key](https://app.requesty.ai/manage-api).\n- **Cost tracking**: Track cost per model, coding language, changed file, and more via the [Cost dashboard](https://app.requesty.ai/cost-management) or the [Requesty VS.code extension](https://marketplace.visualstudio.com/items?itemName=Requesty.requesty).\n- **Stats and logs**: See your [coding stats dashboard](https://app.requesty.ai/usage-stats) or go through your [LLM interaction logs](https://app.requesty.ai/logs).\n- **Fallback policies**: Keep your LLM working for you with fallback policies when providers are down.\n* **Prompt Caching:** Some providers support prompt caching. [Search models with caching](https://app.requesty.ai/router/list).\n\n" }, "docs/saarthi/providers/unbound.md": { "content": "---\nsidebar_label: Unbound\n---\n\nimport Codicon from '@site/src/components/Codicon';\n\n# Using Unbound With Saarthi\n\nSaarthi supports accessing models through [Unbound](https://getunbound.ai/), a platform that focuses on providing secure and reliable access to a variety of large language models (LLMs). Unbound acts as a gateway, allowing you to use models from providers like Anthropic and OpenAI without needing to manage multiple API keys and configurations directly. They emphasize security and compliance features for enterprise use.\n\n**Website:** [https://getunbound.ai/](https://getunbound.ai/)\n\n## Creating an Account\n\n1. **Sign Up/Sign In:** Go to the [Unbound gateway](https://gateway.getunbound.ai). Create an account or sign in.\n2. **Create an Application:** Go to the [Applications](https://gateway.getunbound.ai/ai-gateway-applications) page and hit the \"Create Application\" button.\n3. **Copy the API Key:** Copy the API key to your clipboard.\n\n## Supported Models\n\nUnbound allows you configure a list of supported models in your application, and Saarthi will automatically fetch the list of available models from the Unbound API.\n\n## Configuration in Saarthi\n\n1. **Open Saarthi Settings:** Click the gear icon () in the Saarthi panel.\n2. **Select Provider:** Choose \"Unbound\" from the \"API Provider\" dropdown.\n3. **Enter API Key:** Paste your Unbound API key into the \"Unbound API Key\" field.\n4. **Select Model:** Choose your desired model from the \"Model\" dropdown.\n\n## Tips and Notes\n\n* **Security Focus:** Unbound emphasizes security features for enterprise use. If your organization has strict security requirements for AI usage, Unbound might be a good option.\n* **Model List Refresh:** Saarthi includes a refresh button specifically for the Unbound provider in the settings. This allows you to easily update the list of available models from your Unbound application and get immediate feedback on your API key's validity.\n" }, "docs/saarthi/providers/vertex.md": { "content": "---\nsidebar_label: GCP Vertex AI\n---\nimport Codicon from '@site/src/components/Codicon';\n\n\n# Using GCP Vertex AI With Saarthi\n\nSaarthi supports accessing models through Google Cloud Platform's Vertex AI, a managed machine learning platform that provides access to various foundation models, including Anthropic's Claude family.\n\n**Website:** [https://cloud.google.com/vertex-ai](https://cloud.google.com/vertex-ai)\n\n## Prerequisites\n\n* **Google Cloud Account:** You need an active Google Cloud Platform (GCP) account.\n* **Project:** You need a GCP project with the Vertex AI API enabled.\n* **Model Access:** You must request and be granted access to the specific Claude models on Vertex AI you want to use. See the [Google Cloud documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/partner-models/use-claude#before_you_begin) for instructions.\n* **Application Default Credentials (ADC):** Saarthi uses Application Default Credentials to authenticate with Vertex AI. The easiest way to set this up is to:\n 1. Install the Google Cloud CLI: [https://cloud.google.com/sdk/docs/install](https://cloud.google.com/sdk/docs/install)\n 2. Authenticate using: `gcloud auth application-default login`\n* **Service Account Key (Alternative):** Alternatively, you can authenticate using a Google Cloud Service Account key file. You'll need to generate this key in your GCP project. See the [Google Cloud documentation on creating service account keys](https://cloud.google.com/iam/docs/creating-managing-service-account-keys).\n\n## Supported Models\n\nSaarthi supports the following models through Vertex AI (based on source code):\n\n* **Google Gemini Models:**\n * `gemini-2.5-flash-preview-05-20`\n * `gemini-2.0-flash-001`\n * `gemini-2.5-pro-exp-03-25`\n * `gemini-2.0-pro-exp-02-05`\n * `gemini-2.0-flash-lite-001`\n * `gemini-2.0-flash-thinking-exp-01-21`\n * `gemini-1.5-flash-002`\n * `gemini-1.5-pro-002`\n* **Anthropic Claude Models:**\n * `claude-opus-4@20250514:thinking`\n * `claude-opus-4@20250514`\n * `claude-sonnet-4@20250514:thinking`\n * `claude-sonnet-4@20250514`\n * `claude-3-7-sonnet@20250219:thinking`\n * `claude-3-7-sonnet@20250219`\n * `claude-3-5-sonnet-v2@20241022`\n * `claude-3-5-sonnet@20240620`\n * `claude-3-5-haiku@20241022`\n * `claude-3-opus@20240229`\n * `claude-3-haiku@20240307`\n\nRefer to the [Google Cloud documentation on Vertex AI Models](https://cloud.google.com/vertex-ai/generative-ai/docs/learn/models) for the most up-to-date list of available models and their IDs.\n\n## Configuration in Saarthi\n\n1. **Open Saarthi Settings:** Click the gear icon () in the Saarthi panel.\n2. **Select Provider:** Choose \"GCP Vertex AI\" from the \"API Provider\" dropdown.\n3. **Configure Authentication:**\n * **If using Application Default Credentials (ADC):** No further action is needed here. ADC will be used automatically if configured correctly (see Prerequisites).\n * **If *not* using ADC (Service Account Key):**\n * **Option A: Paste JSON Content:** Paste the entire content of your Service Account JSON key file into the **Google Cloud Credentials** field.\n * **Option B: Provide File Path:** Enter the absolute path to your downloaded Service Account JSON key file in the **Google Cloud Key File Path** field.\n4. **Enter Project ID:** Enter your Google Cloud Project ID.\n5. **Select Region:** Choose the region where your Vertex AI resources are located (e.g., `us-east5`).\n6. **Select Model:** Choose your desired model from the \"Model\" dropdown.\n## Tips and Notes\n\n* **Permissions:** Ensure your Google Cloud account has the necessary permissions to access Vertex AI and the specific models you want to use.\n* **Pricing:** Refer to the [Vertex AI pricing](https://cloud.google.com/vertex-ai/pricing) page for details.\n" }, "docs/saarthi/providers/vscode-lm.md": { "content": "---\nsidebar_label: VS Code Language Model API\n---\nimport Codicon from '@site/src/components/Codicon';\n\n\n# Using VS Code Language Model API With Saarthi\n\nSaarthi includes *experimental* support for the [VS Code Language Model API](https://code.visualstudio.com/api/language-extensions/language-model-access). This API allows extensions to provide access to language models directly within VS Code. This means you can potentially use models from:\n\n* **GitHub Copilot:** If you have a Copilot subscription and the extension installed.\n* **Other VS Code Extensions:** Any extension that implements the Language Model API.\n\n**Important:** This integration is highly experimental and may not work as expected. It is dependent on other extensions correctly implementing the VS Code Language Model API.\n\n## Prerequisites\n\n* **VS Code:** The Language Model API is available through VS Code (and is not currently supported by Cursor).\n* **A Language Model Provider Extension:** You need an extension that provides a language model. Examples include:\n * **GitHub Copilot:** If you have a Copilot subscription, the GitHub Copilot and GitHub Copilot Chat extensions can provide models.\n * **Other Extensions:** Search the VS Code Marketplace for extensions that mention \"Language Model API\" or \"lm\". There may be other experimental extensions available.\n\n## Configuration\n\n1. **Open Saarthi Settings:** Click the gear icon () in the Saarthi panel.\n2. **Select Provider:** Choose \"VS Code LM API\" from the \"API Provider\" dropdown.\n3. **Select Model:** The \"Language Model\" dropdown will (eventually) list available models. The format is `vendor/family`. For example, if you have Copilot, you might see options like:\n * `copilot - claude-3.5-sonnet`\n * `copilot - o3-mini`\n * `copilot - o1-ga`\n * `copilot - gemini-2.0-flash`\n\n## Limitations\n\n* **Experimental API:** The VS Code Language Model API is still under development. Expect changes and potential instability.\n* **Extension Dependent:** This feature relies entirely on other extensions providing models. Saarthi cannot directly control which models are available.\n* **Limited Functionality:** The VS Code Language Model API may not support all the features of other API providers (e.g., image input, streaming, detailed usage information).\n* **No Direct Cost Control:** You are subject to the pricing and terms of the extension providing the model. Saarthi cannot directly track or limit costs.\n* **GitHub Copilot Rate Limits:** When using the VS Code LM API with GitHub Copilot, be aware that GitHub may impose rate limits on Copilot usage. These limits are controlled by GitHub, not Saarthi.\n\n\n## Troubleshooting\n\n* **No Models Appear:**\n * Ensure you have VS Code installed.\n * Ensure you have a language model provider extension installed and enabled (e.g., GitHub Copilot, GitHub Copilot Chat).\n * If using Copilot, make sure that you have sent a Copilot Chat message using the model you would like to use.\n* **Unexpected Behavior:** If you encounter unexpected behavior, it's likely an issue with the underlying Language Model API or the provider extension. Consider reporting the issue to the provider extension's developers.\n" }, "docs/saarthi/providers/xai.md": { "content": "---\nsidebar_label: xAI (Grok)\n---\n\nimport Codicon from '@site/src/components/Codicon';\n\n# Using xAI (Grok) With Saarthi\n\nxAI is the company behind Grok, a large language model known for its conversational abilities and large context window. Grok models are designed to provide helpful, informative, and contextually relevant responses.\n\n**Website:** [https://x.ai/](https://x.ai/)\n\n## Getting an API Key\n\n1. **Sign Up/Sign In:** Go to the [xAI Console](https://console.x.ai/). Create an account or sign in.\n2. **Navigate to API Keys:** Go to the API keys section in your dashboard.\n3. **Create a Key:** Click to create a new API key. Give your key a descriptive name (e.g., \"Saarthi\").\n4. **Copy the Key:** **Important:** Copy the API key *immediately*. You will not be able to see it again. Store it securely.\n\n## Supported Models\n\nSaarthi supports the following xAI Grok models:\n\n### Grok-3 Models\n* `grok-3-beta` (Default) - xAI's Grok-3 beta model with 131K context window\n* `grok-3-fast-beta` - xAI's Grok-3 fast beta model with 131K context window\n* `grok-3-mini-beta` - xAI's Grok-3 mini beta model with 131K context window\n* `grok-3-mini-fast-beta` - xAI's Grok-3 mini fast beta model with 131K context window\n\n### Grok-2 Models\n* `grok-2-latest` - xAI's Grok-2 model - latest version with 131K context window\n* `grok-2` - xAI's Grok-2 model with 131K context window\n* `grok-2-1212` - xAI's Grok-2 model (version 1212) with 131K context window\n\n### Grok Vision Models\n* `grok-2-vision-latest` - xAI's Grok-2 Vision model - latest version with image support and 32K context window\n* `grok-2-vision` - xAI's Grok-2 Vision model with image support and 32K context window\n* `grok-2-vision-1212` - xAI's Grok-2 Vision model (version 1212) with image support and 32K context window\n* `grok-vision-beta` - xAI's Grok Vision Beta model with image support and 8K context window\n\n### Legacy Models\n* `grok-beta` - xAI's Grok Beta model (legacy) with 131K context window\n\n## Configuration in Saarthi\n\n1. **Open Saarthi Settings:** Click the gear icon () in the Saarthi panel.\n2. **Select Provider:** Choose \"xAI\" from the \"API Provider\" dropdown.\n3. **Enter API Key:** Paste your xAI API key into the \"xAI API Key\" field.\n4. **Select Model:** Choose your desired Grok model from the \"Model\" dropdown.\n\n## Reasoning Capabilities\n\nGrok 3 Mini models feature specialized reasoning capabilities, allowing them to \"think before responding\" - particularly useful for complex problem-solving tasks.\n\n### Reasoning-Enabled Models\n\nReasoning is only supported by:\n* `grok-3-mini-beta`\n* `grok-3-mini-fast-beta`\n\nThe Grok 3 models `grok-3-beta` and `grok-3-fast-beta` do not support reasoning.\n\n### Controlling Reasoning Effort\n\nWhen using reasoning-enabled models, you can control how hard the model thinks with the `reasoning_effort` parameter:\n\n* `low`: Minimal thinking time, using fewer tokens for quick responses\n* `high`: Maximum thinking time, leveraging more tokens for complex problems\n\nChoose `low` for simple queries that should complete quickly, and `high` for harder problems where response latency is less important.\n\n### Key Features\n\n* **Step-by-Step Problem Solving**: The model thinks through problems methodically before delivering an answer\n* **Math & Quantitative Strength**: Excels at numerical challenges and logic puzzles\n* **Reasoning Trace Access**: The model's thinking process is available via the `reasoning_content` field in the response completion object\n\n## Tips and Notes\n\n* **Context Window:** Most Grok models feature large context windows (up to 131K tokens), allowing you to include substantial amounts of code and context in your prompts.\n* **Vision Capabilities:** Select vision-enabled models (`grok-2-vision-latest`, `grok-2-vision`, etc.) when you need to process or analyze images.\n* **Pricing:** Pricing varies by model, with input costs ranging from $0.3 to $5.0 per million tokens and output costs from $0.5 to $25.0 per million tokens. Refer to the xAI documentation for the most current pricing information.\n* **Performance Tradeoffs:** \"Fast\" variants typically offer quicker response times but may have higher costs, while \"mini\" variants are more economical but may have reduced capabilities." }, "docs/saarthi/tips-and-tricks.md": { "content": "import Codicon from '@site/src/components/Codicon';\n\n# Tips & Tricks\n\nA collection of quick tips to help you get the most out of Saarthi.\n\n- Drag Saarthi to the [Secondary Sidebar](https://code.visualstudio.com/api/ux-guidelines/sidebars#secondary-sidebar) so you can see the Explorer, Search, Source Control, etc.\n\n- Once you have Saarthi in a separate sidebar from the file explorer, you can drag files from the explorer into the chat window (and even multiple at once). Just make sure to hold down the shift key after you start dragging the files.\n- If you're not using [MCP](./features/mcp/overview), turn it off in the MCP Servers tab to significantly cut down the size of the system prompt.\n- To keep your [custom modes](./features/custom-modes) on track, limit the types of files that they're allowed to edit.\n- If you hit the dreaded `input length and max tokens exceed context limit` error, you can recover by deleting a message, rolling back to a previous checkpoint, or switching over to a model with a long context window like Gemini for a message.\n- In general, be thoughtful about your `Max Tokens` setting for thinking models. Every token you allocate to that takes away from space available to store conversation history. Consider only using high `Max Tokens` / `Max Thinking Tokens` settings with modes like Architect and Debug, and keeping Code mode at 16k max tokens or less.\n- If there's a real world job posting for something you want a custom mode to do, try asking Code mode to `Create a custom mode based on the job posting at @[url]`\n- If you want to really accelerate, check out multiple copies of your repository and run Saarthi on all of them in parallel (using git to resolve any conflicts, same as with human devs).\n- When using Debug mode, ask saarthi to \"start a new task in Debug mode with all of the necessary context needed to figure out X\" so that the debugging process uses its own context window and doesn't pollute the main task\n- Add your own tips by clicking \"Edit this page\" below!\n- To manage large files and reduce context/resource usage, adjust the `File read auto-truncate threshold` setting. This setting controls the number of lines read from a file in one batch. Lower values can improve performance when working with very large files, but may require more read operations. You can find this setting in the Saarthi settings under 'Advanced Settings'.\n- Set up a keyboard shortcut for the [`saarthi.acceptInput` command](./features/keyboard-shortcuts) to accept suggestions or submit text input without using the mouse. Perfect for keyboard-focused workflows and reducing hand strain.\n- Use **Sticky Models** to assign specialized AI models to different modes (reasoning model for planning, non-reasoning model for coding). Saarthi automatically switches to each mode's last-used model without manual selection.\n" }, "docs/welcome.md": { "content": "---\ntitle: Welcome to Godspeed\ndescription: Introduction to Godspeed's 4th-gen SDLC platform and its capabilities.\nslug: /\n---\n\n# Welcome to Godspeed\n\n> **From Idea to Scale — 10x Faster & Better**\n\n**Godspeed** is the first **agentic SDLC platform** that brings order, speed, and consistency to the software development lifecycle. Whether you're an individual developer or a scaling team, Godspeed gives you the tools to ship clean, compliant code faster — with fewer bugs and less chaos.\n\n\n## What is Godspeed?\n\nGodspeed combines three powerful innovations into one cohesive developer experience:\n\n- **[Saarthi: Agentic SDLC Team](/docs/saarthi/index.md)** \n AI teammates for product planning, development, testing, DevOps, and monitoring.\n \n- **[4th-Generation Framework Stack](/docs/microservices-framework/guide/get-started.md)** \n Build full-stack, event-driven, distributed systems with fewer lines of code and [built-in best practices](/docs/microservices-framework/introduction/guard-rails.md)\n\n- **[Godspeed Studio](https://studio.godspeed.systems)** — a powerful visual editor for managing and building your Godspeed projects through an intuitive UI.\n\n> Even entry-level engineers — or LLMs — can ship production-grade software using Godspeed.\n\n---\n\n## What Can You Build?\n\n| Category | Examples |\n|------------------------------------- |---------------------------------------------------------------|\n| **Microservices** | Loosely coupled, event-driven services |\n| **API-First Backends** | REST & GraphQL-ready systems |\n| **Distributed Systems** | Event-sourced, scalable infrastructure |\n| **LLM-Powered Apps** | AI-integrated workflows and copilots |\n| **Internal Platforms & SaaS Tools** | Dashboards, portals, automation platforms |\n| **DevOps Workflows & Infra-as-Code** | GitOps, CI/CD, Terraform-powered automation |\n\n---\n\n## Why Teams Choose Godspeed\n\n| With Godspeed | Without Godspeed |\n|---------------------------------------|-------------------------------------------|\n| Less code, fewer bugs | Boilerplate and glue code overload |\n| 10x faster delivery | Long MVP-to-scale cycles |\n| Greater control & visibility | Ops overwhelm and fragmented workflows |\n| No vendor lock-in | High dependency on specific tools |\n| Built-in architecture & security | Ad-hoc patterns, inconsistent practices |\n\n---\n\n## Key Benefits\n\n- **AI Agents for Full SDLC**\n- **Standardized Guardrails**\n- **Zero Boilerplate Setup**\n- **Smart Context & Token Management**\n- **Plug-and-Play with Your Stack**\n- **Battle-Tested by Builders at Scale**\n\n> With Godspeed Mode, reduce bugs and code clutter by 90%, and ship faster with confidence.\n\n\n\n## Free to Start\n\n- No cost to begin\n- Team upskilling & pairing support\n- Architectural guidance & reviews\n- Partnering opportunities & referrals\n\n[Start Building Now](/docs/microservices-framework/guide/get-started.md)\n" }, "helm-chart/templates/NOTES.txt": { "content": "1. Get the application URL by running these commands:\n{{- if .Values.ingress.enabled }}\n{{- range $host := .Values.ingress.hosts }}\n {{- range .paths }}\n http{{ if $.Values.ingress.tls }}s{{ end }}://{{ $host.host }}{{ .path }}\n {{- end }}\n{{- end }}\n{{- else if contains \"NodePort\" .Values.service.type }}\n export NODE_PORT=$(kubectl get --namespace {{ .Release.Namespace }} -o jsonpath=\"{.spec.ports[0].nodePort}\" services {{ include \"helm-chart.fullname\" . }})\n export NODE_IP=$(kubectl get nodes --namespace {{ .Release.Namespace }} -o jsonpath=\"{.items[0].status.addresses[0].address}\")\n echo http://$NODE_IP:$NODE_PORT\n{{- else if contains \"LoadBalancer\" .Values.service.type }}\n NOTE: It may take a few minutes for the LoadBalancer IP to be available.\n You can watch the status of by running 'kubectl get --namespace {{ .Release.Namespace }} svc -w {{ include \"helm-chart.fullname\" . }}'\n export SERVICE_IP=$(kubectl get svc --namespace {{ .Release.Namespace }} {{ include \"helm-chart.fullname\" . }} --template \"{{\"{{ range (index .status.loadBalancer.ingress 0) }}{{.}}{{ end }}\"}}\")\n echo http://$SERVICE_IP:{{ .Values.service.port }}\n{{- else if contains \"ClusterIP\" .Values.service.type }}\n export POD_NAME=$(kubectl get pods --namespace {{ .Release.Namespace }} -l \"app.kubernetes.io/name={{ include \"helm-chart.name\" . }},app.kubernetes.io/instance={{ .Release.Name }}\" -o jsonpath=\"{.items[0].metadata.name}\")\n export CONTAINER_PORT=$(kubectl get pod --namespace {{ .Release.Namespace }} $POD_NAME -o jsonpath=\"{.spec.containers[0].ports[0].containerPort}\")\n echo \"Visit http://127.0.0.1:8080 to use your application\"\n kubectl --namespace {{ .Release.Namespace }} port-forward $POD_NAME 8080:$CONTAINER_PORT\n{{- end }}\n" }, "idea.md": { "content": "# Godspeed Framework\n\n- Introduction\n1. Getting started\n2. Event Sources\n - Overview\n - Express as HTTP Event Source\n - Custom Event Source\n - DSL\n3. Workflows / Functions\n - Overview\n - Inbuilt workflows\n - Recipies/Examples for common use cases\n - Scripting in workflows\n - Retry\n4. Datasources\n - Overview\n - Prisma as datasource\n - Custom Data Source\n5. Authentication\n - Overview\n - Configuration\n - Error handler\n6. Authorization\n - Overview\n7. Config\n - Overview\n - Configuration\n7. Telemetry\n - Overview\n9. Caching\n - Overview\n9. Language Tools\n10. CLI" }, "src/intro.md": { "content": "Page one \n\n- name and tag line\n\n- we can replace the sample images with other images which will convey our vision more clearly (we have 3 image positions we can also add three tag lines related to that images)\n\n- contact info (linkedin,git etc)\n\n\n1. Introduction to Godspeed\n\n- what we do ?\n\n-- Establishment (when,why and who) => not mandatory\n-- what exactly we do \n\n\n- what a developer will get ?\n\n-- how we compressed the steps from a traditional way of writing backend with godspeed frame work\n\n-- how we decreased the huge work of writing backend and godspeed is always open take up new plugins of users wish\n\n\n- why to choose godspeed framework ?\n\n-- will talk about all the features of godspeed like developer friendly, no lock-in ,less time consumption etc\n\n-- Unique Selling Proposition (USP)\n\nExplain what sets your product apart from competitors. This could include unique features, cutting-edge technology, exceptional customer service, or a particular niche focus.\n\n-- we can talk about our happy clients and their feedbacks\n\n\n\n2. Introduction to godspeed framework\n\n- we can add the intro from the old godspeed documentation about the three main keys of godspeed framework like\n events,workflows,datasources etc\n\n-can add some points from the old documentaion preface section like frame work architecture,usecases,goals \n\n- how can developer contribute to godspeed\n\n- need to modify the datasources according to the new version add ons\n\n\n3. Getting started with godspeed\n\n- Godspeed installation process\n\n-- prerequisits\n-- framework installation steps\n\n\n- intro to godspeed project\n\n-- project structure\n-- configuration (intro about all the folders and files in the project)\n-- tests (generating test suite)\n\n- implementaion of godspeed framework with example\n-- creating and running a project from scartch\n" }, "docs/saarthi/basic-usage/using-modes.md": { "content": "# Agents and Modes\n\nAgents in Saarthi are specialized personas that tailor the assistant's behavior to your current task. Each mode offers different capabilities, expertise, and access levels to help you accomplish specific goals.\n\n:::info Sticky Models\nEach agent or remembers your last-used model. When switching modes, Saarthi automatically selects that model—no manual selection needed. Assign different models to different modes (e.g., Gemini 2.5 Preview for `🏗️ Architect` mode, Claude Sonnet 3.7 for `💻 Code` mode) and Saarthi will switch models automatically when you change modes.\n:::\n\n## Why Use Different Modes?\n\n- **Task specialization:** Get precisely the type of assistance you need for your current task\n- **Safety controls:** Prevent unintended file modifications when focusing on planning or learning\n- **Focused interactions:** Receive responses optimized for your current activity\n- **Workflow optimization:** Seamlessly transition between planning, implementing, debugging, and learning\n\n## Switching Between Modes\n\nFour ways to switch modes:\n\n1. **Dropdown menu:** Click the selector to the left of the chat input\n \n2. **Slash command:** Type `/architect`, `/ask`, `/debug`, `/code`, or `/orchestrator` in the chat input\n \n3. **Toggle command/Keyboard shortcut:** Use the keyboard shortcut below, applicable to your operating system. Each press cycles through the available modes in sequence, wrapping back to the first mode after reaching the end.\n \n | Operating System | Shortcut |\n |------------------|----------|\n | macOS | ⌘ + . |\n | Windows | Ctrl + . |\n | Linux | Ctrl + . |\n\n4. **Accept suggestions:** Click on mode switch suggestions that Saarthi offers when appropriate\n\n## Built-in Agents or Modes\n\n### Code (Default)\n\n| Aspect | Details |\n|--------|---------|\n| **Name** | `💻 Code` |\n| **Description** | A skilled software engineer with expertise in programming languages, design patterns, and best practices |\n| **Tool Access** | Full access to all tool groups: `read`, `edit`, `browser`, `command`, `mcp` |\n| **Ideal For** | Writing code, implementing features, debugging, and general development |\n| **Special Features** | No tool restrictions—full flexibility for all coding tasks |\n\n### Ask\n\n| Aspect | Details |\n|--------|---------|\n| **Name** | `❓ Ask` |\n| **Description** | A knowledgeable technical assistant focused on providing thorough and complete answers. It's less inclined to switch to implementing code unless explicitly requested and may use diagrams for clarification. |\n| **Tool Access** | Limited access: `read`, `browser`, `mcp` only (cannot edit files or run commands) |\n| **Ideal For** | Code explanation, concept exploration, and technical learning |\n| **Special Features** | Optimized for detailed, informative responses, often using diagrams for clarity, without modifying your project. |\n\n### Architect\n\n| Aspect | Details |\n|--------|---------|\n| **Name** | `🏗️ Architect` |\n| **Description** | An experienced technical leader and planner who helps design systems and create implementation plans |\n| **Tool Access** | Access to `read`, `browser`, `mcp`, and restricted `edit` (markdown files only) |\n| **Ideal For** | System design, high-level planning, and architecture discussions |\n| **Special Features** | Follows a structured approach from information gathering to detailed planning |\n\n### Debug\n\n| Aspect | Details |\n|--------|---------|\n| **Name** | `🪲 Debug` |\n| **Description** | An expert problem solver specializing in systematic troubleshooting and diagnostics |\n| **Tool Access** | Full access to all tool groups: `read`, `edit`, `browser`, `command`, `mcp` |\n| **Ideal For** | Tracking down bugs, diagnosing errors, and resolving complex issues |\n| **Special Features** | Uses a methodical approach of analyzing, narrowing possibilities, and fixing issues. Includes custom instructions to reflect, distill possibilities, add logs, and confirm before fixing. |\n\n### Orchestrator (aka Boomerang Mode)\n\n| Aspect | Details |\n|--------|---------|\n| **Name** | `🪃 Orchestrator` |\n| **Description** | A strategic workflow orchestrator (aka Boomerang Mode) that breaks down complex tasks and delegates them to specialized modes. Learn more about [Boomerang Tasks](../features/boomerang-tasks). |\n| **Tool Access** | Access to `read`, `browser`, `command`, `mcp`, and restricted `edit` (mode configuration files only: `.saarthimodes`, `custom_modes.json`) |\n| **Ideal For** | Managing multi-step projects, coordinating work across different modes, and automating complex workflows |\n| **Special Features** | Uses the [`new_task`](../advanced-usage/available-tools/new-task) tool to delegate subtasks to other modes. |\n\n### Godspeed\n\n| Aspect | Details |\n|--------|---------|\n| **Name** | `⚡ Godspeed` |\n| **Description** | A Godspeed meta-framework expert with deep knowledge of microservices architecture, distributed scalable enterprise grade systems and industry wide best practices |\n| **Tool Access** | Full access to all tool groups: `read`, `edit`, `browser`, `command`, `mcp` |\n| **Ideal For** | Developing and managing Godspeed applications, microservices architecture, and enterprise-grade systems |\n| **Special Features** | Optimized for Godspeed framework development, providing access to Godspeed-specific tools and knowledge **Context-Aware Intelligence via RAG Integration**- fetches relevant information from vector databases, allowing precise answers contextualized to the user's codebase and documentation. |\n\n---\n\n### Activate Godspeed Mode\n\n**Godspeed Mode** enables **retrieval-augmented generation (RAG)** powered by your codebase, docs, and the **Godspeed Systems framework** — delivering **production-grade microservices with embedded best practices and architectural guardrails**.\n\n### Connect to the RAG-node MCP Server\n\nTo use this mode:\n\n1. Open Saarthi’s **MCP Settings** (top-right of Saarthi pane).\n2. Scroll down and click:\n * **Edit Global MCP** → Opens global `mcp_settings.json`.\n3. Add your **Gemini API key** under the `env` block of the RAG-node server config:\n\n```json\n\"RAG-node\": {\n \"type\": \"stdio\",\n \"command\": \"npx\",\n \"args\": [\n \"-y\",\n \"@godspeedsystems/rag-node\"\n ],\n \"disabled\": false,\n \"cwd\": \"/path/to/home/directory\",\n \"alwaysAllow\": [\n \"*\"\n ],\n \"env\": {\n \"GOOGLE_API_KEY\": \"AIXXXXXXXXXXXXXXXXXX\" // Add your Gemini API Key here\n }\n}\n```\n🔗 [Get your Gemini API Key](https://makersuite.google.com/app/apikey)\n\n### What Happens Next?\n\nOnce connected:\n\n* Saarthi uses `mcp.handle-query` to forward your queries to the **RAG-node**.\n* The node uses **FAISS vector search** + **Gemini embeddings** to:\n * Retrieve precise, source-traceable information.\n * Auto-generate config, functions, events, and workflows.\n * Provide *real-time guidance* aligned with Godspeed's architecture.\n\n⚠️ *Currently supports only Google Gemini for embeddings.*\n\n### Code Review\n\n| Aspect | Details |\n|--------|---------|\n| **Name** | `🕵️ Code Review` |\n| **Description** | An AI Code Review Assistant—an expert system specialized in comprehensive, context-aware code analysis |\n| **Tool Access** | Full access to all tool groups: `read`, `edit`, `command`, `mcp` |\n| **Ideal For** | Performing code reviews, identifying potential issues, and ensuring code quality |\n| **Special Features** | Provides comprehensive code analysis, identifies potential issues, and suggests improvements |\n\nTo get review of your `Godspeed projects`, [Connect to the RAG-node MCP Server](/docs/saarthi/basic-usage/using-modes#connect-to-the-rag-node-mcp-server) first.\n\n### DevOps Mode\n\n| Aspect | Details |\n|--------|---------|\n| **Name** | `🛠️ DevOps` |\n| **Description** | The ultimate DevOps Mastermind. Streamlines deployment tasks for all types of projects. |\n| **Tool Access** | Full access to all tool groups: `read`, `edit`, `browser`, `command`, `mcp` |\n| **Ideal For** | Automating deployments, managing infrastructure, and streamlining development workflows |\n| **Special Features** | Local Docker Deployment: Automatically generates Dockerfile, docker-compose.yml, and .dockerignore. It builds and runs the project locally in Docker with minimal prompts. Render Deployment: Uses a new Render MCP tool (added to global MCPs). Just provide your Git repo URL and Render account details, and Saarthi handles the full Render deployment pipeline. |\n\n## Customizing Modes\n\nTailor Saarthi's behavior by customizing existing modes or creating new specialized assistants. Define tool access, file permissions, and behavior instructions to enforce team standards or create purpose-specific assistants. See [Custom Modes documentation](../features/custom-modes) for setup instructions.\n " }, "docs/saarthi/features/mcp/using-mcp-in-saarthi.mdx": { "content": "---\nid: using-mcp-in-saarthi\ntitle: Using MCP in Saarthi\nsidebar_label: Using MCP in Saarthi\n---\n\nimport Codicon from '@site/src/components/Codicon';\n\n# Using MCP in Saarthi\n\n:::info Confused about MCP Servers?\n\nAn MCP (Model Context Protocol) server acts as a bridge, giving Saarthi access to a wider range of **tools** and external services like databases, APIs, or custom scripts. It uses a standard communication method, allowing Saarthi to leverage these external capabilities.\n\nFor a deeper dive, check out [What is MCP?](/saarthi/features/mcp/what-is-mcp).\n:::\n\nModel Context Protocol (MCP) extends Saarthi's capabilities by connecting to external tools and services. This guide covers everything you need to know about using MCP with Saarthi.\n\n\n## Configuring MCP Servers\n\nMCP server configurations can be managed at two levels:\n\n1. **Global Configuration**: Stored in the `mcp_settings.json` file, accessible via VS Code settings (see below). These settings apply across all your workspaces unless overridden by a project-level configuration.\n2. **Project-level Configuration**: Defined in a `.saarthi/mcp.json` file within your project's root directory. This allows you to set up project-specific servers and share configurations with your team by committing the file to version control. Saarthi automatically detects and loads this file if it exists.\n\n**Precedence**: If a server name exists in both global and project configurations, the **project-level configuration takes precedence**.\n\n### Editing MCP Settings Files\n\nYou can edit both global and project-level MCP configuration files directly from the Saarthi MCP settings view:\n\n1. Click the icon in the top navigation of the Saarthi pane.\n2. Scroll to the bottom of the MCP settings view.\n3. Click the appropriate button:\n * **`Edit Global MCP`**: Opens the global `mcp_settings.json` file.\n * **`Edit Project MCP`**: Opens the project-specific `.saarthi/mcp.json` file. If this file doesn't exist, Saarthi will create it for you.\n\n![Edit Global MCP and Edit Project MCP buttons](/img/using-mcp-in-saarthi/using-mcp-in-saarthi-9.png)\n\nBoth files use a JSON format with a `mcpServers` object containing named server configurations:\n\n ```json\n {\n \"mcpServers\": {\n \"server1\": {\n \"command\": \"python\",\n \"args\": [\"/path/to/server.py\"],\n \"env\": {\n \"API_KEY\": \"your_api_key\"\n },\n \"alwaysAllow\": [\"tool1\", \"tool2\"],\n \"disabled\": false\n }\n }\n }\n ```\n *Example of MCP Server config in Saarthi (STDIO Transport)*\n\n ### Understanding Transport Types\n\n MCP supports three transport types for server communication: STDIO for local servers, Streamable HTTP (recommended for new remote servers), and SSE (for legacy remote servers).\n\n #### STDIO Transport\n\n Used for local servers running on your machine:\n\n * Communicates via standard input/output streams\n * Lower latency (no network overhead)\n * Better security (no network exposure)\n * Simpler setup (no HTTP server needed)\n * Runs as a child process on your machine\n\n For more in-depth information about how STDIO transport works, see [STDIO Transport](./server-transports#stdio-transport).\n\n STDIO configuration parameters:\n\n * `command` (required): The executable to run (e.g., `node`, `python`, `npx`, or an absolute path).\n * `args` (optional): An array of string arguments to pass to the command. You can reference system environment variables using `${env:VARIABLE_NAME}` syntax.\n * `cwd` (optional): The working directory from which to launch the server process. If omitted, defaults to the first workspace folder path or the main process's working directory. Useful if the server script relies on relative paths.\n * `env` (optional): An object containing environment variables to set for the server process.\n * `alwaysAllow` (optional): An array of tool names from this server to automatically approve.\n * `disabled` (optional): Set to `true` to disable this server configuration.\n\n STDIO configuration example:\n ```json\n {\n \"mcpServers\": {\n \"local-server\": {\n \"command\": \"node\",\n \"args\": [\"server.js\"],\n \"cwd\": \"/path/to/project/root\", // Optional: Specify working directory\n \"env\": {\n \"API_KEY\": \"your_api_key\"\n },\n \"alwaysAllow\": [\"tool1\", \"tool2\"],\n \"disabled\": false\n }\n }\n }\n ```\n\n #### Using System Environment Variables in Arguments\n\n You can reference system-level environment variables within the `args` array using the `${env:VARIABLE_NAME}` syntax. This allows you to pass sensitive information like API keys or tokens from your system environment without hardcoding them in your configuration:\n\n ```json\n {\n \"mcpServers\": {\n \"github\": {\n \"command\": \"docker\",\n \"args\": [\n \"run\",\n \"-i\",\n \"--rm\",\n \"-e\",\n \"GITHUB_PERSONAL_ACCESS_TOKEN=${env:GITHUB_PERSONAL_ACCESS_TOKEN}\",\n \"ghcr.io/github/github-mcp-server\"\n ],\n \"alwaysAllow\": [\n \"get_pull_request\"\n ]\n }\n }\n }\n ```\n\n In this example, `${env:GITHUB_PERSONAL_ACCESS_TOKEN}` will be replaced with the value of the `GITHUB_PERSONAL_ACCESS_TOKEN` environment variable from your system. This is particularly useful when:\n - Working with Docker containers that need environment variables passed through\n - Keeping sensitive credentials out of your configuration files\n - Using the same configuration across different environments with different credentials\n\n **Note:** The environment variable must exist in your system environment for this to work. You can set system environment variables through your operating system's settings or shell configuration files (e.g., `.bashrc`, `.zshrc`, or Windows Environment Variables).\n#### Streamable HTTP Transport\n\nThis is the **modern standard** for remote servers accessed over HTTP/HTTPS, offering more flexibility and replacing the legacy SSE transport for new implementations.\n\n* Communicates via HTTP POST/GET to a single MCP endpoint\n* Optionally uses Server-Sent Events (SSE) for streaming\n* Can be hosted on a different machine\n* Supports multiple client connections\n* Requires network access\n* Allows centralized deployment and management\n\nFor more in-depth information about how Streamable HTTP transport works, see [Streamable HTTP Transport](./server-transports#streamable-http-transport).\n\nStreamable HTTP configuration parameters:\n\n* `type` (required): Must be set to `\"streamable-http\"`.\n* `url` (required): The full URL of the remote MCP server's single endpoint (e.g., `https://your-server.com/mcp`).\n* `headers` (optional): An object containing custom HTTP headers to send with requests (e.g., for authentication tokens).\n* `alwaysAllow` (optional): An array of tool names from this server to automatically approve.\n* `disabled` (optional): Set to `true` to disable this server configuration.\n\nStreamable HTTP configuration example:\n```json\n{\n \"mcpServers\": {\n \"modern-remote-server\": {\n \"type\": \"streamable-http\",\n \"url\": \"https://your-modern-server.com/api/mcp-endpoint\",\n \"headers\": {\n \"X-API-Key\": \"your-secure-api-key\"\n },\n \"alwaysAllow\": [\"newToolA\", \"newToolB\"],\n \"disabled\": false\n }\n }\n}\n```\n\n #### SSE Transport (Legacy)\n\n Used for older remote servers accessed over HTTP/HTTPS. **For new remote server implementations, [Streamable HTTP Transport](#streamable-http-transport) is recommended.**\n\n * Communicates via Server-Sent Events protocol (typically requires separate endpoints for client-to-server and server-to-client communication)\n * Can be hosted on a different machine\n * Supports multiple client connections\n * Requires network access\n * Allows centralized deployment and management\n\n For more in-depth information about how legacy SSE transport works, see [SSE Transport (Legacy)](./server-transports#sse-transport-legacy).\n\n SSE (Legacy) configuration parameters:\n\n * `type` (optional, but recommended for clarity): Should be set to `\"sse\"` if providing a `url` for an SSE server, to distinguish from Streamable HTTP. If `url` is present and `type` is omitted, Saarthi might try to infer, but explicit declaration is safer.\n * `url` (required): The base URL for the remote MCP server. For legacy SSE, this usually implies separate paths like `/events` (for SSE stream) and `/message` (for POST requests) will be derived or expected by the server.\n * `headers` (optional): An object containing custom HTTP headers to send with requests (e.g., for authentication tokens).\n * `alwaysAllow` (optional): An array of tool names from this server to automatically approve.\n * `disabled` (optional): Set to `true` to disable this server configuration.\n\n SSE (Legacy) configuration example:\n ```json\n {\n \"mcpServers\": {\n \"legacy-remote-server\": {\n \"type\": \"sse\", // Explicitly define as SSE\n \"url\": \"https://your-legacy-server-url.com/mcp-base\", // Base URL\n \"headers\": {\n \"Authorization\": \"Bearer your-legacy-token\"\n },\n \"alwaysAllow\": [\"oldToolX\"],\n \"disabled\": false\n }\n }\n }\n ```\n\n ## Enabling or Disabling MCP Servers\n\nDisabling your MCP Servers here will remove all MCP related logic and definitions from your system prompt, reducing your token usage. This will prevent Saarthi from connecting to any MCP servers, and the `use_mcp_tool` and `access_mcp_resource` tools will not be available. Check this off if you don't intend to use MCP Servers. This is on by default.\n\n1. Click the icon in the top navigation of the Saarthi pane\n2. Check/Uncheck `Enable MCP Servers`\n\n## Enabling or Disabling MCP Server Creation\n\nDisabling your MCP Server Creation here will just remove the instructions from your system prompt that Saarthi uses to write MCP servers while not removing the context related to operating them. This reduces token usage. This is on by default.\n\n1. Click the icon in the top navigation of the Saarthi pane\n2. Check/Uncheck `Enable MCP Server Creation`\n\n## How to Use Saarthi to Create an MCP Server\n\nIf you need a specific tool or capability that isn't available through existing MCP servers, you can ask Saarthi to build a new one for you.\n\n**Prerequisite:** Ensure the **[Enable MCP Server Creation](#enabling-or-disabling-mcp-server-creation)** setting is checked ON in the MCP settings panel. If this is disabled, Saarthi will not have the necessary instructions to build a server.\n\n**How to Initiate:**\n\n1. **Make a Request:** Clearly ask Saarthi for the new tool or capability. For example:\n * \"Create an MCP tool that gets the current price of Bitcoin.\"\n * \"I need a tool that connects to my company's internal user database via its API.\"\n * \"Build an MCP server to interact with the GitHub Gist API.\"\n\n2. **Saarthi's Process (Simplified):** Once you make the request (and the setting is enabled), Saarthi will:\n * Fetch internal instructions for server creation.\n * Scaffold a basic server project (usually TypeScript) in the default MCP directory (e.g., `~/Documents/Saarthi/MCP` on macOS) unless you specify otherwise.\n * Write the code to implement the requested tool, including handling necessary API calls.\n * **Handle Secrets:** If the tool requires API keys or other credentials, Saarthi will ask you for them using the [`ask_followup_question`](/advanced-usage/available-tools/ask-followup-question) tool to ensure they are configured securely as environment variables for the server.\n * **Configure:** Automatically add the new server's configuration to your global `mcp_settings.json` or project `.Saarthi/mcp.json` file.\n * **Activate:** Attempt to connect to the newly configured server so its tools are immediately available.\n\n3. **Outcome:** If successful, Saarthi will confirm the creation, and the new server and its tools will appear in your MCP server list, ready for use.\n\nThis feature allows you to tailor Saarthi's capabilities by having it build the specific integrations you need directly from your requests. For a deeper look into the internal mechanics, see the [Tool Calling Mechanism](/advanced-usage/available-tools/tool-use-overview#tool-calling-mechanism).\n\n## Managing Individual MCP Servers\n\n ![Example of a configuration pane for a MCP Server](/img/using-mcp-in-saarthi/using-mcp-in-saarthi-8.png)\n\nEach MCP server has its own configuration panel where you can modify settings, manage tools, and control its operation. To access these settings:\n\n1. Click the icon in the top navigation of the Saarthi pane\n2. Locate the MCP server you want to manage in the list\n ![List of MCP Servers](/img/using-mcp-in-saarthi/using-mcp-in-saarthi-4.png)\n### Deleting a Server\n\n1. Press the next to the MCP server you would like to delete\n2. Press the `Delete` button on the confirmation box\n\n### Restarting a Server\n\n1. Press the button next to the MCP server you would like to restart\n\n### Enabling or Disabling a Server\n\n1. Press the toggle switch next to the MCP server to enable/disable it\n\n### Network Timeout\n\nTo set the maximum time to wait for a response after a tool call to the MCP server:\n\n1. Click the `Network Timeout` pulldown at the bottom of the individual MCP server's config box and change the time. Default is 1 minute but it can be set between 30 seconds and 5 minutes.\n\n### Auto Approve Tools\n\nMCP tool auto-approval works on a per-tool basis and is disabled by default. To configure auto-approval:\n\n1. First enable the global \"Use MCP servers\" auto-approval option in [auto-approving-actions](/features/auto-approving-actions)\n2. In the MCP server settings, locate the specific tool you want to auto-approve\n3. Check the `Always allow` checkbox next to the tool name\n\n![Always allow checkbox for MCP tools](/img/using-mcp-in-saarthi/using-mcp-in-saarthi-7.png)\n\nWhen enabled, Saarthi will automatically approve this specific tool without prompting. Note that the global \"Use MCP servers\" setting takes precedence - if it's disabled, no MCP tools will be auto-approved.\n\n## Finding and Installing MCP Servers\n\nSaarthi does not come with any pre-installed MCP servers. You'll need to find and install them separately.\n\n* **Community Repositories:** Check for community-maintained lists of MCP servers on GitHub\n* **Ask Saarthi:** You can ask Saarthi to help you find or even create MCP servers (when \"[Enable MCP Server Creation](#enabling-or-disabling-mcp-server-creation)\" is enabled)\n* **Build Your Own:** Create custom MCP servers using the SDK to extend Saarthi with your own tools\n\nFor full SDK documentation, visit the [MCP GitHub repository](https://github.com/modelcontextprotocol/).\n\n## Using MCP Tools in Your Workflow\n\nAfter configuring an MCP server, Saarthi automatically detects its available tools and resources. Effectively leveraging these tools involves understanding the core interaction steps and, crucially, how Saarthi interprets the tools you provide.\n\n### Core Workflow Steps\n\nYour interaction with MCP tools typically follows this sequence:\n\n#### 1. Initiate a Task\nBegin by typing your request in the Saarthi chat interface.\n\n#### 2. Tool Identification by Saarthi\nSaarthi analyzes your request to determine if an available MCP tool can assist. This stage is highly dependent on the quality of your MCP tool definitions.\n\n##### The Critical Role of Descriptions\nSaarthi's ability to:\n* Identify the *correct* tool for the job,\n* Understand how to structure the necessary parameters, and\n* Avoid misinterpreting a tool's capabilities,\nall hinge on clear, concise, and informative descriptions for both the tools themselves and their parameters. Vague or missing information, especially for parameters, can significantly hinder Saarthi's ability to select or use a tool effectively.\n\nFor instance, a request like \"Analyze the performance of my API\" might lead Saarthi to consider an MCP tool designed for API endpoint testing. Whether Saarthi successfully identifies and utilizes this tool as intended is directly influenced by the quality of its description.\n\n##### Best Practices for Defining MCP Tools\nTo ensure Saarthi can leverage your MCP tools efficiently, consider the following when defining them in your server:\n\n* **Tool Name:** Choose a descriptive and unambiguous name that clearly indicates the tool's primary function.\n* **Tool Description:** Provide a comprehensive summary of what the tool does, its purpose, and any important context or prerequisites for its use. Explain the outcome or result of using the tool.\n* **Parameter Descriptions:** This is critical. For each parameter:\n * Clearly state its purpose and what kind of data it expects (e.g., \"User ID for lookup,\" \"File path to process,\" \"Search query string\").\n * Specify any formatting requirements, constraints, or an example of a valid value if applicable.\n * Indicate if the parameter is optional or required (though the MCP schema usually handles this, a note can be helpful).\n* **Clarity for the AI:** Write descriptions as if you are explaining the tool to another developer (or an AI). The more context Saarthi has, the better it can integrate the tool into its problem-solving workflows. If a tool is intended to be used in a specific sequence or in conjunction with other tools, mentioning this can also be beneficial.\n* **Augment with Custom Instructions:** Beyond the descriptions embedded in the MCP server, you can further guide Saarthi's usage of specific MCP tools by providing [Custom Instructions](/features/custom-instructions). This allows you to define preferred approaches, outline complex workflows involving multiple tools, or specify when a particular MCP tool should be prioritized or avoided.\n\n#### 3. Tool Invocation\nIf Saarthi, guided by the tool descriptions, identifies a suitable tool, it will propose its use. You then approve this (unless [auto-approval](#auto-approve-tools) is configured for trusted tools).\n\n### Maximizing Synergy with MCP Servers\nBy investing effort in crafting detailed descriptions and potentially augmenting them with custom instructions, you significantly improve the synergy between Saarthi and your MCP servers. This unlocks their full potential for more reliable and efficient task completion.\n\n## Troubleshooting MCP Servers\n\nCommon issues and solutions:\n\n* **Server Not Responding:** Check if the server process is running and verify network connectivity\n* **Permission Errors:** Ensure proper API keys and credentials are configured in your `mcp_settings.json` (for global settings) or `.Saarthi/mcp.json` (for project settings).\n* **Tool Not Available:** Confirm the server is properly implementing the tool and it's not disabled in settings\n* **Slow Performance:** Try adjusting the network timeout value for the specific MCP server\n\n## Platform-Specific MCP Configuration Examples\n\n### Windows Configuration Example\n\nWhen setting up MCP servers on Windows, you'll need to use the Windows Command Prompt (`cmd`) to execute commands. Here's an example of configuring a Puppeteer MCP server on Windows:\n\n```json\n{\n \"mcpServers\": {\n \"puppeteer\": {\n \"command\": \"cmd\",\n \"args\": [\n \"/c\",\n \"npx\",\n \"-y\",\n \"@modelcontextprotocol/server-puppeteer\"\n ]\n }\n }\n}\n```\n\nThis Windows-specific configuration:\n- Uses the `cmd` command to access the Windows Command Prompt\n- Uses `/c` to tell cmd to execute the command and then terminate\n- Uses `npx` to run the package without installing it permanently\n- The `-y` flag automatically answers \"yes\" to any prompts during installation\n- Runs the `@modelcontextprotocol/server-puppeteer` package which provides browser automation capabilities\n\n### macOS and Linux Configuration Example\n\nWhen setting up MCP servers on macOS or Linux, you can use a simpler configuration since you don't need the Windows Command Prompt. Here's an example of configuring a Puppeteer MCP server on macOS or Linux:\n\n```json\n{\n \"mcpServers\": {\n \"puppeteer\": {\n \"command\": \"npx\",\n \"args\": [\n \"-y\",\n \"@modelcontextprotocol/server-puppeteer\"\n ]\n }\n }\n}\n```\n\nThis configuration:\n- Directly uses `npx` without needing a shell wrapper\n- Uses the `-y` flag to automatically answer \"yes\" to any prompts during installation\n- Runs the `@modelcontextprotocol/server-puppeteer` package which provides browser automation capabilities\n\nThe same approach can be used for other MCP servers on Windows, adjusting the package name as needed for different server types.\n\n## Runtime Version Manager Configuration\n\nWhen working with multiple versions of programming languages or runtimes, you may use version managers like [asdf](https://asdf-vm.com/) or [mise](https://mise.jdx.dev/) (formerly rtx). These tools help manage multiple runtime versions on a single system. Here's how to configure MCP servers to work with these version managers:\n\n### mise Configuration Example\n\n[mise](https://mise.jdx.dev/) is a fast, modern runtime version manager that can be used to specify which version of Node.js, Python, or other runtimes to use for your MCP server:\n\n```json\n{\n \"mcpServers\": {\n \"mcp-batchit\": {\n \"command\": \"mise\",\n \"args\": [\n \"x\",\n \"--\",\n \"node\",\n \"/Users/myself/workspace/mcp-batchit/build/index.js\"\n ],\n \"disabled\": false,\n \"alwaysAllow\": [\n \"search\",\n \"batch_execute\"\n ]\n }\n }\n}\n```\n\nThis configuration:\n- Uses the `mise` command to manage runtime versions\n- The `x` subcommand executes a command with the configured runtime version\n- The `--` separates mise arguments from the command to run\n- Runs `node` with the specific version configured in your mise settings\n- Points to the MCP server JavaScript file\n- Automatically allows the \"search\" and \"batch_execute\" tools\n\n### asdf Configuration Example\n\n[asdf](https://asdf-vm.com/) is a popular tool for managing multiple runtime versions. Here's how to configure an MCP server to use a specific Node.js version managed by asdf:\n\n```json\n{\n \"mcpServers\": {\n \"appsignal\": {\n \"command\": \"/Users/myself/.asdf/installs/nodejs/22.2.0/bin/node\",\n \"args\": [\n \"/Users/myself/Code/Personal/my-mcp/build/index.js\"\n ],\n \"env\": {\n \"ASDF_NODE_VERSION\": \"22.2.0\"\n },\n \"disabled\": false,\n \"alwaysAllow\": []\n }\n }\n}\n```\n\nThis configuration:\n- Directly references the Node.js executable from the asdf installations directory\n- Sets the `ASDF_NODE_VERSION` environment variable to ensure consistent version use\n- Points to the MCP server JavaScript file\n\nUsing version managers ensures that your MCP servers run with the correct runtime version, regardless of the system's default version, providing consistency across different environments and preventing version conflicts.\n\n" }, "docs/saarthi/getting-started/connecting-api-provider.md": { "content": "---\nsidebar_label: Connecting To AI Provider\n---\n\n\n# Connecting Your First AI Provider\n\nSaarthi requires an API key from an AI model provider to function. \n\n### Getting Your API Key from direct Providers\n\nFor direct access to specific models from their original providers, with full access to their features and capabilities:\n\n#### Anthropic\n\n1. Go to [console.anthropic.com](https://console.anthropic.com/)\n2. Sign up for an account or log in\n3. Navigate to the [API keys section](https://console.anthropic.com/settings/keys) and create a new key\n4. **Important:** Copy your API key immediately as it won't be displayed again\n\n*Anthropic console API Keys section with \"Create key\" button. Name your key, set expiration, and copy it immediately.*\n\n#### OpenAI\n\n1. Go to [platform.openai.com](https://platform.openai.com/)\n2. Sign up for an account or log in\n3. Navigate to the [API keys section](https://platform.openai.com/api-keys) and create a new key\n4. **Important:** Copy your API key immediately as it won't be displayed again\n\n![OpenAI API keys page](/img/connecting-api-provider/connecting-api-provider-6.png)\n\n*OpenAI platform with \"Create new secret key\" button. Name your key and copy it immediately after creation.*\n\n## Configuring Saarthi in VS Code\n\nOnce you have your API key:\n\n1. Open the Saarthi sidebar by clicking the Saarthi icon in the VS Code Activity Bar\n2. In the welcome screen, select your API provider from the dropdown\n3. Paste your API key into the appropriate field\n4. Select your model:\n - For **OpenRouter**: select `anthropic/claude-3.7-sonnet` ([model details](https://openrouter.ai/anthropic/claude-3.7-sonnet))\n - For **Anthropic**: select `claude-3-7-sonnet-20250219` ([model details](https://www.anthropic.com/pricing#anthropic-api))\n\n:::info Model Selection Advice\nWe strongly recommend **Claude 3.7 Sonnet** for the best experience—it generally \"just works\" out of the box. Saarthi has been extensively optimized for this model's capabilities and instruction-following behavior.\n\nSelecting alternative models is an advanced feature that introduces complexity. Different models vary significantly in how they follow tool instructions, parse formats, and maintain context through multi-step operations. If you do experiment with other models, choose ones specifically designed for structured reasoning and tool use.\n:::\n\nWe recommend these options for accessing the powerful **Claude 3.7 Sonnet** model:\n\n- **OpenRouter:** Provides access to multiple AI models through a single API key. Ideal for getting started quickly with minimal setup. [View pricing](https://openrouter.ai/models?order=pricing-low-to-high).\n- **Anthropic:** Direct access to Claude models. Requires API access approval and may have [rate limits depending on your tier](https://docs.anthropic.com/en/api/rate-limits#requirements-to-advance-tier). See [Anthropic's pricing page](https://www.anthropic.com/pricing#anthropic-api) for details.\n\n5. Click \"Let's go!\" to save your settings and start using Saarthi\n\n\n\n" }, "docs/saarthi/getting-started/installing.mdx": { "content": "---\nsidebar_label: Installing Saarthi\n---\n\n# Installing Saarthi\n\n[Saarthi](https://marketplace.visualstudio.com/items?itemName=godspeed-systems.saarthi) is a VS Code extension that brings AI-powered coding assistance directly to your editor. Install from\n**VS Code Marketplace** - fastest method for standard VS Code and Cursor users\n\n## VS Code Marketplace\n\n1. Open VS Code\n2. Access Extensions: Click the Extensions icon in the Activity Bar or press `Ctrl+Shift+X` (Windows/Linux) or `Cmd+Shift+X` (macOS)\n3. Search for \"Saarthi\"\n4. Select \"Saarthi\" by godspeed systems and click **Install**\n5. Reload VS Code if prompted\n\nAfter installation, find the Saarthi icon in the Activity Bar to open the Saarthi panel.\n\n## Local Setup & Development\n\nFor instructions on setting up Saarthi for local development, building from source, or contributing, please see the [Local Development Setup](../advanced-usage/local-development-setup) guide in the Advanced Usage section.\n\n## Troubleshooting\n\n*VS Code Output panel showing Saarthi logs for troubleshooting*\n\n**Extension Not Visible**\n* Restart VS Code\n* Verify Saarthi is listed and enabled in Extensions\n* Try disabling and re-enabling\n* Check Output panel for errors (View → Output, select \"Saarthi\")\n\n**Installation Problems**\n* Ensure stable internet connection\n* Verify VS Code version 1.84.0 or later\n\n" }, "docs/saarthi/index.md": { "content": "---\nsidebar_label: Welcome\n---\n\n# Saarthi\n\nSaarthi is an AI-powered autonomous coding agent that lives in your editor. It helps you code faster and smarter, whether you're starting a new project, maintaining existing code, or learning new technologies.\n\n## What Can Saarthi Do?\n\n- 🚀 **Generate framework-aligned code** using Godspeed mode with RAG\n- 🔧 **Refactor & Debug** existing code\n- 📝 **Write & Update** documentation\n- 🤔 **Answer Questions** about your codebase\n- 🔄 **Automate Devops** Generates Docker configs and deploys to Docker or Render\n- ✅ **Review code** Performs detailed reviews across six dimensions including compliance, completeness, and bugs.\n- 🏗️ **Create** new files and projects\n\n## Quick Start\n\n1. [Install Saarthi](./getting-started/installing.mdx)\n2. [Connect Your AI Provider](./getting-started/connecting-api-provider.md)\n3. [Try Your First Task](./getting-started/your-first-task.md)\n\n## Key Features\n\n### Multiple Agents\nSaarthi adapts to your needs with specialized [agents anda modes](./basic-usage/using-modes):\n- **Code:** For general-purpose coding tasks\n- **Architect:** For planning and technical leadership\n- **Ask:** For answering questions and providing information\n- **Debug:** For systematic problem diagnosis\n- **Orchestrator:** For managing complex tasks and delegating work\n- **Godspeed:** For developing and managing Godspeed applications, microservices architecture, and enterprise-grade systems\n- **Code Reviewer:** Performing code reviews, identifying potential issues, and ensuring code quality\n- **Devops:** Streamlines deployment tasks for all types of projects.\n- **[Custom Modes](./features/custom-modes):** Create unlimited specialized agents for security auditing, performance optimization, documentation, or any other task\n\n**Coming Soon in Saarthi:** QA, Interviewer, Tech Documenter and Product Manager agents. \n\n### Smart Tools\nSaarthi comes with powerful [tools](./basic-usage/how-tools-work) that can:\n- Read and write files in your project\n- Execute commands in your VS Code terminal\n- Control a web browser\n- Use external tools via [MCP (Model Context Protocol)](./features/mcp/overview)\n\nMCP extends Saarthi's capabilities by allowing you to add unlimited custom tools. Integrate with external APIs, connect to databases, or create specialized development tools - MCP provides the framework to expand Saarthi's functionality to meet your specific needs.\n\n### Customization\nMake Saarthi work your way with:\n- [Custom Instructions](./features/custom-instructions) for personalized behavior\n- [Custom Modes](./features/custom-modes) for specialized tasks\n- [Local Models](./advanced-usage/local-models) for offline use\n- [Auto-Approval Settings](./features/auto-approving-actions) for faster workflows\n\n\n" } }