import { chunk } from 'lodash'; import { In } from 'typeorm'; import { Page } from '@/entities'; import { sequence } from '@/utils'; import { CSVPage } from './types'; export default async ( csvPages: CSVPage[], { afterInsert, }: { /** * Whether or not to run afterInsert listener after all pages are inserted. */ afterInsert: boolean; } ): Promise => { const productLowestPriceMap = new Map< string, { price: number; pageId: string } >(); csvPages.forEach((csvPage) => { const existing = csvPage.productFirebaseId ? productLowestPriceMap.get(csvPage.productFirebaseId) : undefined; if (!csvPage.productFirebaseId || !csvPage.id || !csvPage.price) { return; } if (!existing) { productLowestPriceMap.set(csvPage.productFirebaseId, { price: csvPage.price, pageId: csvPage.id, }); } else if (existing.price > (csvPage.price ?? 0)) { productLowestPriceMap.set(csvPage.productFirebaseId, { price: csvPage.price, pageId: csvPage.id, }); } }); await sequence( chunk( csvPages .filter( ({ firebaseId, productFirebaseId, price, retailer, url }) => firebaseId && productFirebaseId && price && retailer && url ) .map(({ lowestPricePageId, productFirebaseId, ...rest }) => ({ productFirebaseId, discoverable: true, ...rest, })), 100 ).map((pages) => async () => { await Page.insert(pages); }) ); // update lowest price pages await sequence( [...productLowestPriceMap.entries()].map( ([productFirebaseId, { pageId }]) => async () => { await Page.update( { productFirebaseId, }, { lowestPricePageId: pageId, } ); } ) ); const pages = await Page.find({ where: { id: In(csvPages.map(({ id }) => id)) }, }); if (afterInsert) { await Promise.all(pages.map((page) => page.afterInsert())); } return pages; };