Hi, I had a question, under integrations, there is an option for web crawling. I just wanted to know where the scraped data is actually stored and also for example if we scrape the website, and later if any modifications are done on the website, will quick suite integration get notified ? or does it automatically modified ?
cause when we look into the business point of view, it is not possible to scrape the website each and every time to get the updated version.
if quick suite does not have this feature, what can be other way of achieving this task other than scraping it every time
Hi @Irene_Sibi
Quick’s Web Crawler integration, part of Amazon Q in Quick (Enterprise edition), crawls public websites to index content for knowledge bases used in Q&A, data stories, and generative AI features.
Scraped data from the Web Crawler is indexed directly into Amazon Q’s knowledge base within Quick Suite, not stored as raw files in user accessible locations like S3. This indexed format supports search and retrieval but acts as a snapshot tied to sync configurations.
Quick Suite does not automatically detect or notify about website changes post-crawl, updates require manual triggers or scheduled syncs. You can configure sync modes like full sync (recrawls everything) or incremental (detects new/modified/deleted content) with frequencies such as daily or hourly. No real-time notifications exist.
Scraped data from the Web Crawler is indexed directly into Amazon Q’s knowledge base within Quick Suite
ARE WE ABLE TO ACCESS IT ONCE SCRAWLED
Hi @Irene_Sibi
Yes, once data is crawled via Web Crawler integration, it becomes accessible for Q&A queries, data stories, and generative BI features.
Note: Raw scraped data isn’t exposed as downloadable files.
Hi, can you provide which team is responsible in granting access to Web crawler and Amazon S3 on Quick with the Amzonbi Account
Thank you
Hi @enyarkor
Welcome to the Quick Suite community!
This is a community forum managed by users, we don’t have access to private account details, so we’re unable to assist directly with this request. I kindly suggest reaching out to your AWS account admin or Quick Suite admin, who will be in the best position to help with these permissions.
Hi @Irene_Sibi,
Just checking back in since we haven’t heard from you in a bit. I wanted to see if the guidance shared earlier helped resolve your question, or if you found a solution in the meantime.
If you still have any additional questions related to your initial post, feel free to share them. Otherwise, any update you’re able to provide within the next 3 business days would be helpful for the community.
Thank you