Skip to article
AHFE AHFE Discoveries AHFE International Open Access Journal
AHFE OPEN ACCESS OPEN ACCESS CONFERENCE PROCEEDINGS PAPER

Automating customer feedback in online marketplaces with retrieval augmented generation

Faculty of Computer Science and Engineering, Innopolis University, Innopolis, Tatarstan, Russia ↗
Publication year
Digital Object Identifier10.54941/ahfe1005849

Abstract

This paper proposes a solution for automating customer review responses in online marketplaces. The goal is to save time and resources for sellers. The proposed system combines traditional methods with Large Language Models, allowing the sellers to provide a personalized service and improve customer satisfaction. The system has been implemented and integrated with two main online marketplaces in Russia: Wildberries and Ozon. The study demonstrates promising results in terms of response quality and efficiency. In particular, the system was used to answer more than 3800 reviews for three sellers, which was estimated to be an equivalent of 120 working hours.

INTRODUCTION

The E-commerce market is growing fast, and according to Shopify estimates, the volume of sales has already reached $6.33 trillion in 2024 (Lin, 2024). E-commerce is also growing in Russia, the two biggest players being Wildberries (WB) and Ozon, whose share increased to 76% in 2023 from 67% in 2022, and their orders grew by 212 million units year-over-year (Virin, 2024). An important trend in E-commerce is automation, with a Tidio study reporting that automation can lead up to a 250% increase in purchases and engagement for marketplace sellers (Stefanowicz, 2024).

Marketplace sellers have many different tasks, one of them is responding to user feedback and questions. It is a frequent and time-consuming activity, with approximately 60% of sellers engage in daily product review management, and every other vendor having to respond to such reviews personally (Tinkoff Data, 2023). Yet marketplaces encourage customer reviews as they play a vital role in sales. Buyers tend to trust sellers whose products are backed by reviews. A substantial volume of reviews coupled with high ratings signifies widespread satisfaction among buyers, minimizing the risks of time and money wastage. Studies indicate that customer reviews can boost conversion rates by as much as 270% (Durovi´c and Kniepkamp, 2022).

© 2025. Published by AHFE Open Access. All rights reserved. 447

The goal of our study is to automate the process of responding to customer reviews to save time and resources for sellers. Our main technical contribution is the use of large language models (LLM) and retrieval- augmented generation (RAG) in automating the review management process. These technologies have a significant potential for use in customer support by improving access to data and automating the generation of personalized responses (Kodif.inc., 2023). This allows companies to provide personalized service, which is becoming increasingly important according to research (Durovi´c and Kniepkamp, 2022). Management also recognizes the value of personalization and sees the potential for ChatGPT to improve responses (Kodif.inc., 2023).

The focus of our solution is on Russian marketplaces. As the first step, we studied the Russian marketplaces, existing solutions for automating feedback management, and conducted customer development interviews. Then we proceeded to develop the system while collaborating with sellers on the Wildberries and Ozon platforms, iterating over several versions of the system to arrive at the fully functional and usable minimum viable product (MVP).

The following sections describe in more detail our literature review and market research process, the development of the system, as well as the evaluation results of the business efficiency improvements achieved through the automated feedback management process.

LITERATURE REVIEW AND MARKET RESEARCH Our goal is to automate responses to customer reviews on marketplaces. Prior to developing our system, we researched customer review automation, identified current business trends, and analysed existing solutions in the retail automation market.

We conducted keyword searches using terms like “marketplace dynamics,” “automation techniques,”“LLM applications in user support,”“E-commerce automation trends,” “LangChain technology,” and “ChatGPT and YandexGPT in E-commerce.” We also utilized services such as Statista, MarketResearch.com, and GlobalData to gather data on the Russian market.

E-Commerce Automation Artificial intelligence (AI), particularly Large Language Models (LLMs) like ChatGPT, is transforming customer service by automating tasks, offering personalized interactions, and providing efficient support. LLMs simplify routine tasks and can sometimes outperform humans due to fewer errors and faster processing (Armanr and Lamiyar, 2023). Parmar (2023) highlights benefits such as personalized customer interaction, effective support, and increased engagement.

LLMs process user input and generate human-like responses through natural language processing (NLP) and machine learning (ML). Businesses can use LLMs to develop virtual assistants that handle customer inquiries,

Automating Customer Feedback in Online Marketplaces 449

allowing employees to focus on more complex tasks (Javaid et al., 2023). Moreover, this automation can operate continuously without interruptions, leading 80% of service executives to prioritize shifting to self-service channels (Amit, 2024). Kumar et al. (2023) support these benefits and encourage vendors to adopt ChatGPT in their operations. Zhang’s findings (2023) reveal that ChatGPT can enhance customer service by offering personalized services, improving service quality and efficiency, reducing manual workload, and supporting the growth of the E-commerce sector.

The global retail automation market is steadily growing, driven by the need for streamlined operations and greater efficiency. Technologies such as robotic process automation (RPA) and AI contribute to cost reduction and operational efficiency. In 2023, the market was valued at USD 24.12 billion and is projected to grow at a CAGR of 9.3% from 2024 to 2030, fuelled by the adoption of advanced technologies like IoT and ML (Grand View Research, 2023).

Small retail stores are increasingly implementing cost-effective POS systems that integrate various functions to improve operational efficiency. For instance, in January 2023, Market POS introduced a cloud-based POS software tailored for SMEs, groceries, and supermarkets.

Celerant Technology highlights several key benefits of automation:

  • Providing a unified shopping experience across sales channels
  • Saving retail staff time and increasing efficiency
  • Enabling data-driven decision-making
  • Supporting business growth with scalable POS and E-commerce software
  • Enhancing customer service for repeat business Advanced technologies like RPA, AI, and ML are expected to further drive cost reductions and increase automation in retail operations (Celerant Technology, 2024). For example, Brain Corporation and Dane Technologies collaborated to develop inventory-scanning systems for retailers, leveraging expertise in autonomous systems and AI (Brain Corp., 2023).

The increasing use of robotics automation streamlines repetitive tasks like inventory management, ensuring quick and accurate operations, and enhancing customer experience by expediting transactions (TikFlow RPA, 2023). The growing preference for online retail transactions pushes software providers to innovate retail solutions, leading to market expansion (Grand View Research, 2023).

Entrepreneurs often engage in operational and routine tasks, especially when operating solo. A survey indicates that 60% of vendors on Wildberries and Ozon allocate at least an hour daily to order management, with 42% dedicating multiple hours. Approximately 60% of sellers engage in daily product review management, with half personally responding to reviews. The demand for shop managers on marketplaces has grown in Russia, with a 27% increase in vacancies in the first quarter of 2024 compared to the previous year (Tinkoff Data, 2023).

Analysing Existing Solutions: Customer Review Automation in Russia Universal marketplaces such as Wildberries and Ozon are the main drivers of E-commerce growth in Russia (Tinkoff Data, 2023). The majority of sellers on Wildberries and Ozon are individual entrepreneurs (82% and 74% respectively) (Virin, 2024). Both marketplaces and sellers can leverage AI technologies to enhance personalization, recommendations, analytics, and data processing, thereby improving customer satisfaction and operational efficiency (Virin, 2024).

We analyzed existing services that automate responses to customer reviews on marketplaces in Russia. Most solutions focus on Wildberries (WB) due to its easier API integration compared to other platforms. The majority of services also support Ozon, given its prominence in the Russian market.

Most solutions use web services for ease of use. Pricing varies, with solutions offering analytics typically being more expensive.

For key competitors, we analyzed functionality details. Solutions offering analytics are generally more expensive. Our analysis indicates that many competitors work exclusively with WB reviews due to API integration convenience. Our project distinguishes itself by supporting multiple marketplaces and offering a more user-friendly platform for automating responses. Pricing strategies will be discussed in the next section.

METHODOLOGY AND DESIGN Our system’s goal is to save the time and resources of online marketplace sellers through the automation of responses to customer reviews.

To develop our system, we conducted the following activities in an iterative manner over several minimum viable product (MVP) versions: system requirements and Design and system validation and evaluation Below we explain each step in more detail.

Analysing Existing Solutions: Customer Review Automation in Russia Our context diagram (see Figure 1) show the interactions between the marketplace feedback response system and other actors (external factors) with which the system must interact. It can be seen that there are three key models: Feedback Managers, Buyers and our model (a combination of the YaGPT model with a trained prompt and the FAQ feedback database). The context diagram provided illustrates the current and future states of a feedback response system for handling customer feedback after purchasing a product.

In the current system (As-Is), managers manually respond to customer feedback, while the feedback system merely forwards customer input to them. In the future system (To-Be), OtvechAI automates responses, with managers only initiating the system and monitoring feedback. The automated process significantly improves efficiency by automatically generating and

Automating Customer Feedback in Online Marketplaces 451

sending replies, reducing the need for manual intervention. Key changes include automation of responses, a shift in the manager’s role, and increased efficiency.

Figure 1: Context diagram.
Original PDF Figure 1:

System Requirements and Design The system requirements for our automated feedback response system were derived from the findings of the customer development process and the usability testing. These requirements serve as the foundation for the design and development of our system.

The following use case diagram (see Figure 2) represents the common use case flow for the manager. It describes the tasks and interactions that occur when a manager interacts with the feedback response system.

Functional requirements include the system’s ability to receive customer feedback, generate automated responses using AI, allow managers to monitor feedback, and provide a user-friendly interface for sellers with personalized reply settings. Non-functional requirements emphasize high accuracy, scalability, low response time, security, and integration with existing marketplace platforms.

Review Processing: The review processing algorithm (see Figure 3). Upon receiving new reviews on the platform, the system first categorizes the review as either a text review or an evaluation. For Empty Feedback (Evaluation), the system decides between employing a Template response or generating a response via YaGPT, based on the evaluation score. This score guides the system in choosing whether to utilize a pre-existing template or to craft a generated response. In instances of Text Feedback, there are four available response strategies: the two aforementioned options, consultation of the System FAQ Database, or a combination thereof. Following the selection

Figure 2: Use case diagram.
Original PDF Figure 2:

of the response strategy, the system methodically addresses each piece of feedback individually.

System Architecture Our system consists of three major components:

  • Web scraping service Data storage and backend service
  • Automation service with frontend and browser extension The web scraping service is responsible for gathering customer feedback from marketplaces. The data is then stored in the backend service.

The backend service is responsible for storing customer feedback in the relational data storage (RDBMS), generating automated responses, and serving the frontend service.

The frontend service provides a user-friendly interface for sellers to interact with the system. It is implemented as a standalone website for WB sellers and a browser extension for Ozon sellers (Wildberries, 2024; Yandex Cloud, 2023; Lupadin, 2023; Ozon blog, 2022; 2024).

SYSTEM EVALUATION The effectiveness of the system was assessed using two primary metrics:

  • The reduction in response time for the sellers.
  • The quality of the generated responses.

We conducted customer development interviews and usability tests with over 54 sellers (May 2024 data), identifying a need for simplicity, personalization, and mobile accessibility in our system. Based on this feedback, we iterated on our product, resulting in a user-friendly interface, customizable response options, and mobile responsiveness. Key metrics include responding to over 3800 reviews, saving clients more than 58 800 rubles and 127 hours, and compiling over 3000 FAQ responses. Our system

Automating Customer Feedback in Online Marketplaces 453

Figure 3: Review processing pipeline.
Original PDF Figure 3:

has significantly improved efficiency, increased customer satisfaction, and is scalable for future growth.

Mathematical equation reproduced from the original PDF
Equation reproduced from the original PDF.

÷ 60 (4)

Rreviews × Tresponse

Mathematical equation reproduced from the original PDF
Equation reproduced from the original PDF.

where: Nsellers is the number of sellers interviewed, Rreviews is the number of reviews responded to, Canswer is the cost per answer through our service in rubles, Smanager is the average salary of a manager on the marketplace in rubles, Ctotal is the total cost of responding to the reviews through our service, Srubles is the amount of money saved for clients in rubles, Tresponse is the time taken to respond to one review in minutes, Thours is the total time saved for clients in hours, FFAQ is the number of FAQ responses compiled.

The quality of responses: We collected customer feedback on generated responses and adjusted prompts for better results. Personalized prompts were added to meet individual needs, such as including signatures when necessary. Our system handles positive reviews (4–5 stars) well and sometimes responds effectively to negative reviews. Interaction with FAQs helps resolve recurring issues (Soshnikov, 2023).

Reduction in response time: Automated responses to customer reviews significantly reduced response time. Positive feedback is handled in 1 minute, negative feedback in 5 minutes, compared to previous times of 2–3 and 10 minutes respectively. Manual work has decreased by 70%, allowing salespeople to focus on strategic tasks.

DISCUSSION

The business goal of our project is to simplify the work of sellers on marketplaces. At the moment we are automating the work with reviews. We

have learnt a lot of useful points during the time we have been working on this project. Responding to feedback is an integral part of the process of selling goods on marketplaces. Through them, buyers can get more information about the experience of use, the quality of the product, and ask questions. For sellers, reviews are able to increase sales, attract customers to themselves, highlight their product among competitors. Sellers with smaller shops are able to respond to reviews manually, but it takes quite a bit of time. In our experience, when the number of reviews goes over 100 reviews per month, sellers hire a person who deals exclusively with customer enquiries and reviews. We believe that modern technology allows us to do it faster and better. With responses to positive reviews, the algorithm is simple: as a rule, you just need to copy one of the prepared reviews and send the person words of gratitude. With negative feedback or questions, the situation is more complicated. You need to find the right words to keep the customer. This is why we have implemented FAQ systems, which searches for similar requests and answers among those that have been written earlier. Our system is already showing good results in terms of saved time and resources, but we have room to grow and develop. For further development we are considering a mobile application, because some of our customers respond to feedback via their mobile phones. We can also consider the option of PWA. Also an important step is to add Ozon and WB to a single service through which sellers could respond to reviews from one place. Plus, support for other marketplaces.

Interoperability With Marketplaces and Customer Acquisition During our research into the topic of online shopping, we focused on the Russian market. Having identified the two most popular marketplaces (Ozon and WB) we began to think about implementation. WB has an API, so after a few interviews we decided to create a separate platform for ease of use. With Ozon, the situation is more complicated. They do not provide an API, and account transfer does not look very secure for sellers. We decided to create a browser extension which is embedded in the personal account page. At the moment, this method is the most convenient.The key to a useful product is working with a real salesperson. This is the only way to understand the real pains and problems. Our relationship with sellers on marketplaces has shown a very interesting picture. We noticed that sellers were too busy to understand our project and subsequent implementation. Our challenge was and is to break this pattern. We realised that an Aha moment is very important, when the seller himself sees real progress in time and resources. It is very important for us to convey that we are not building a barrier between buyer and seller through automation. Our project should be seamlessly integrated and show the seller the real picture through analytics, so that he stays in contact with the buyers.

The Power of Data We understand the importance of analytics. That is why we are looking in this direction first and foremost. Two key areas of analytics were identified in the survey. Firstly, sellers have a demand for product analytics through

Automating Customer Feedback in Online Marketplaces 455

customer feedback. As an example, a seller sells household chemicals and has several types of washing powder. Sales are going well, but with some frequency he receives feedback about unpleasant odour. Due to the large amount of feedback, these reviews get lost in the general flow. In such a case, it would be useful to have an algorithm that separates the feedback into clusters. The feedback in each cluster is then formed into groups. When there are recurring complaints in one of the resulting groups, the system signals this to the seller. In this way, it is possible to either remove the product completely or correct the problem.

The Problem of Fake Reviews We have previously discussed the problem of fake reviews (Soshnikov, 2023; Bodrov, 2022; Dyachenko, 2024), which is a significant issue for some of our clients. In fact, we encountered a case where a seller was forced to close his shop due to fake reviews. Such reviews mislead both buyers, who do not see the real picture of the product, and sellers. Unfortunately, shop support is very reluctant to resolve these complaints. A system for calculating such fake reviews could help both sellers and buyers. Additionally, we have realised that some sellers and shops use fake reviews to downgrade their competitors’ sales or to artificially boost their own sales. Typically, these reviews are similar in nature, posted at the same time, but from different accounts. While marketplaces are attempting to combat this, those providing fake review services continuously find ways around the blockages. Algorithms are not yet adept at distinguishing between genuine reviews and those written by the sellers themselves. After researching this area, we realised that a solution to help sellers confirm a fake attack or expose competitors could be very useful for both regular buyers and sellers themselves.

Overall Impact and Future Improvements The implementation of our system has led to measurable improvements in both the efficiency of review management and the satisfaction levels of customers and sellers alike.

CONCLUSION

In conclusion, this paper investigated the topic of automating the process of responding to customer feedback in online marketplaces in order to save time and resources for merchants. A system based on large language models (LLM) and augmented retrieval augmented generation (RAG) was proposed to solve the problem of automating review management.

The research methodology included literature review, market research, system development and close collaboration with vendors to create a minimum viable product (MVP). The system was developed using an iterative approach including customer needs research, design and prototyping, usability testing, iterative development and performance evaluation.

The technical implementation of the system included a web-based feedback scraping service, a data storage and processing service, and an automation

service with a frontend and browser extension. The system showed high performance in feedback processing, saving time and money for sellers.

The results of the research and testing confirmed the importance of answering feedback correctly to increase sales and customer satisfaction. The system also enabled the collection of a database of feedback responses, which can be used for further automation and personalisation.

Overall, the system for automating responses to customer feedback has shown its positive impact on feedback management processes, customer satisfaction and the effectiveness of sellers in online marketplaces. We also discussed plans for further development of the system, including integration with additional marketplaces, improved personalisation and development of a mobile application.

REFERENCES

Amit RG. (2024). How to Automate Ecommerce Customer Service. RichPanel.

https://www.richpanel.com/blog/automate-ecommerce-customer-service Bodrov, S. (2022). Кaк paспознaть фейковые отзывы нa мapкетплейсaх. PCКPп. https://rskrf.ru/tips/eksperty-obyasnyayut/feykovye-otzyvy-na-marketpl eysakh/ Brain Corp. (2023). Brain Corp and Dane Technologies Extend Partnership to Bring Inventory Scanning Retail Solutions to Market. Brain Corp. https://www.braincorp.com/resources/brain-corp-and-dane-technologies-ext end-partnership-to-bring-inventory-scanning-retail-solutions-to-market-55982 Celerant Technology. (2024). POS System and eCommerce: 5 Benefits of an All-In-

One Retail System. LinkedIn. https://www.linkedin.com/pulse/pos-system-ecom merce-5-benefits-all-in-one-retail-imn7e/ Durovi´c, M., Kniepkamp, T. (2022). Good advice is expensive – bad advice even more: the regulation of online reviews. Research Gate, 156. Dyachenko, O. (2024). Зa что кукушкa хвaлит петухa: фaльшивые отзывы и

мошенничество нa мapкетплейсaх. PИAМО. https://riamo.ru/articles/aktsenty /za-chto-kukushka-hvalit-petuha-falshivye-otzyvy-i-moshennichestvo-na-mark etplejsah/ Fedotov, N. (2023). Последствия от нaкpутки отзывов нa Wildberries. MoneyPlace. https://moneyplace.io/wildberries/stat-wb/stoit-li-zakazyvat-otz yvy-na-vajldberriz/ Grand View Research. (2023). Retail Automation Market Size, Share & Trends Analysis Report By Product (Camera, PoS), By Implementation (In-store, Warehouse), By End-use (Fuel Stations, Supermarkets), By Region, And Segment Forecasts, 2024 - 2030. Grand View Research, 120. Javaid, M., Haleem, A., Singh, R. P. (2023). A study on ChatGPT for Industry 4.0:

Background, potentials, challenges, and eventualities. Science Direct, 143. Kodif.inc. (2023). ChatGPT and GPT-4 use cases in Customer Support. Kodif.

https://kodif.ai/2023/03/21/chatgpt-and-gpt-4-use-cases-in-customer-support/ Kumar, A., Gupta, N., Bapat, G. (2023). Who is making the decisions? How retail managers can use the power of ChatGPT. Emerald insight, 169. Lin, Y. (2024). Global Ecommerce Sales Growth Report. Shopify. https://www.shop

ify.com/blog/global-ecommerce-sales Lupadin, I. (2023). Кaк paботaть с отзывaми нa Wildberries, Ozon и Ян- декс. Мapкете. Workspace. https://workspace.ru/blog/kak-rabotat-s-otzyvami-n a-marketpleisah/

Automating Customer Feedback in Online Marketplaces 457

Md Armanr, Umama Rashid Lamiyar. (2023). Exploring the Implication of ChatGPT AI for Business: Efficiency and Challenges. Applied Quantitative Analysis, 22. Ozon blog. (2022). Bсем товapaм — по 10 отзывов. Ozon. https://seller.ozon.ru/m

edia/news/vsem-tovaram-po-10-otzyvov/ Ozon blog. (2024). Отзывы покупaтелей. Ozon. https://seller-edu.ozon.ru/how-tosell-effectively/loyalty/reviews/buyers-reviews Parmar, D. (2023). Enhancing Customer Relationship Management with Salesforce

Einstein GPT. Thesius, 42. Soshnikov, D. (2023). Creating a subject-oriented chatbot using LangChain and Yandex GPT. Soshnikov.com. https://soshnikov.com/ai/creating-problem-domain -specific-chat-assistant-with-yandex-gpt-and-langchain-ru/ Stefanowicz, B. (2024). What Is Ecommerce Automation and How to Use It? Tidio.

https://www.tidio.com/blog/ecommerce-automation/ TikFlow RPA. (2023). Retail Automation: Revolutionizing the Industry for Efficiency and Growth. LinkedIn. https://www.linkedin.com/pulse/retail-automa tion-revolutionizing-industry-efficiency-tikflow-rpa/ Tinkoff Data. (2023). Исследовaние Tinkoff eCommerce: пpодaвцы нa мapкетплейсaх в Pоссuu в 2023 году. Tinkoff Data, 25. Virin, F. (2024). Мapкетинговое исследовaние Мapкетплейсы 2024: тpенды, стpaтегии, точки pостa. Data Insight. https://datainsight.ru/DI_FedorVirin_Sel lersnMarketplaces_2024 Wildberries. (2024). Wildberries Документaция | API Пpодaвцa. Wildberries. https://openapi.wb.ru Yandex Cloud. (2023). Generating a response to customer feedback. Yandex

Cloud. https://cloud.yandex.ru/ru/docs/yandexgpt/prompts/customer-service-gen erating-response Zhang, M. (2023). Research on the Impact of ChatGPT on The Economic Development of E-commerce Industry. Research Gate, 100.

Cover of the proceedings volume Intelligent Human Systems Integration (IHSI 2025): Integrating People and Intelligent Systems

Explore the complete digital volume

Intelligent Human Systems Integration (IHSI 2025): Integrating People and Intelligent Systems

Read this individual research paper free through the original publisher, or explore the full digital proceedings book.

ISBN 978-1-964867-36-6

Explore this ISBN in the AHFE Bookstore Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional

Explore Related Research

Related research 8 matches

More papers from this volume

← Back to Research ExplorerBrowse the full library →