/** * ### HTML-to-Main-Content Extractor #2 * * 1. The algorithm starts by loading the HTML content using linkedom, a lightweight DOM parser for Node.js. * 2. It then applies a series of cleaning and scoring techniques to identify the main content of * the page, starting with stripping unlikely candidates (e.g., elements with class names like "comment" * or "sidebar"). * 3. The HTML is converted into a series of paragraph elements, which are then scored based on various * factors such as text length, number of commas, and the presence of certain class names or IDs. * 4. The algorithm assigns scores to parent and grandparent elements based on the scores of their * children, with parents receiving the full score and grandparents receiving half. * 5. After scoring, the algorithm finds the top candidate element by selecting the node with the * highest score. * 6. The top candidate's siblings are then examined to see if they should be included in the main * content, based on their scores and other factors like link density. * 7. The algorithm then cleans the selected content by removing unnecessary tags, attributes, and empty * elements. * 8. It also handles special cases like cleaning up header tags, images, and other potentially irrelevant * content. * 9. Throughout the process, the algorithm uses various regular expressions and scoring heuristics to * identify positive and negative indicators of content relevance. * 10. Finally, the cleaned and extracted content is returned as an HTML string, representing the main * body of the article or webpage. * * [Article Extraction Benchmark](https://trafilatura.readthedocs.io/en/latest/evaluation.html) * * @param {string} html - The HTML content to extract from. * @param {Object} [opts] - The options for content extraction. * @param {boolean} opts.stripUnlikelyCandidates default=true - Remove elements that match non-article- * like criteria first (e.g., elements with a classname of "comment"). * @param {boolean} opts.weightNodes default=true - Modify an element's score based on certain classNames or * IDs (e.g., subtract if a node has a className of 'comment', add if a node has an ID of 'entry-content'). * @param {boolean} opts.cleanConditionally default=true - Clean the node to remove superfluous content * like forms, ads, etc. Initially, pass in the most restrictive options which will return the highest * quality content. On each failure, retry with slightly more lax options. * @returns {string} The extracted content as an HTML string, or null if extraction fails. * @author [vtempest (2025)](https://github.com/vtempest) * Based on [Postlight Mercury Parser (2017-)](https://github.com/postlight/parser/tree/main/src) * @example var url = "https://en.wikipedia.org/wiki/David_Hilbert" * var html = await (await fetch(url)).text(); * var content = extractMainContentFromHTML(html); * console.log(content); // HTML content of main article body * @category Extract */ export declare function extractMainContentFromHTML2(html: any, opts: any): any; /** * Sets the score attribute of a node. * @param {Node} node - The node to set the score on. * @param {Document} document - The document object. * @param {number} score - The score to set. * @returns {Node} The node with the set score. * @private */ export declare function setScore(node: any, document: any, score: any): any; /** * Scores a paragraph node. * @param {Node} node - The paragraph node to score. * @private * @returns {number} The score of the paragraph. */ export declare function scoreParagraph(node: any): number;