# Introduction to Glarity ## Glarity - **Your Free and Ad-Free ChatGPT Cross-Language Reading & Writing Assistant** (Update 2023.10.31) *Also known as :* Glarity Summary: an open-source ChatGPT Summary extension for YouTube, Google, Twitter, and any webpage. It provides cross-language summaries to effortlessly summarize videos, searches, PDFs, emails, and webpages. It supports free side-by-side translations, email writing assistance, Web Content Q\&A, and much more. This is an open-source extension that will show a recap from ChatGPT next to website content. With Glarity, you can quickly and easily digest the most important information from website content **Summary for Website （**[**Glarity Summary for: website list** ](https://blog.glarity.app/getting-started/update-history)**)** *For example: Google search/ YouTube / Twitter/ PubMed/ NewsPicks/ Nikkei/ GitHub/ Yahoo! JAPAN News...*

Glarity Logo

**Side-by-Side Translation** (mirror translation) Instant Translation with mirrored display for effortless comparison, enhancing clarity and understanding. **Writing Made Easy** Gmail Email Assistant (Craft email responses with just one sentence) Extension installation Chrome: FireFox: MAC/iOS Safari: ### ⭐️**Features:** * Supports the official OpenAI API * Supports ChatGPT Plus * Markdown rendering * Code highlights * Dark mode * Provide feedback to improve ChatGPT * Copy to clipboard * Switch languages * YouTube videos transcript * YouTube videos key moment * Ask Glarity anything you'd like ### Source code: {% embed url="" %} ### Twitter [@glarity\_summary](https://twitter.com/Glarity_summary) ### ### FAQ 1. What is ChatGPT? ChatGPT is a language model developed by OpenAI. It's a large, pre-trained Transformer-based neural network designed to generate human-like text in response to prompts provided by users. It has been trained on a diverse range of internet text and can respond to a wide range of topics, including general knowledge questions, conversational responses, and creative writing. 2.What is Glarity - Summary for Google/YouTube (ChatGPT)? Glarity Summary is a ChatGPT extension for YouTube and Google that can summarize YouTube videos and Google searches. It also supports Yahoo! ニュース, PubMed, PMC, NewsPicks, Github, Nikkei, Bing, Google Patents, Twitter, Gmail, PDF, and provides summaries for any webpage. Glarity also supports a side-by-side translation feature and serves as a Writing Assistant. 3.Is Glarity - Summary for Google/YouTube (ChatGPT) free to use? Yes, using the extension is completely free. However, if you use the Glarity mode or the OpenAI key mode, service providers may charge a certain fee based on token consumption. Currently, the ChatGPT mode is entirely free; you can select different modes according to your needs. 4. Do I need a ChatGPT/OpenAI account? No, you will not need a ChatGPT account to use Glarity. You can create a free account at glarity.app.If you want to use Glarity for free, you will need a ChatGPT account to enable this extension. You can create an account for free at [chat.openai.com](https://chat.openai.com/) ### Languages [Bahasa Indonesia](https://glarity.app/id) [Bahasa Melayu](https://glarity.app/ms) [Deutsch](https://glarity.app/de) [English](https://glarity.app/en) [English (United States)](https://glarity.app/en-US) [Filipino](https://glarity.app/fil) [Français](https://glarity.app/fr) [Kiswahili](https://glarity.app/sw) [Nederlands](https://glarity.app/nl) [Norsk](https://glarity.app/no) [català](https://glarity.app/ca) [Tiếng Việt](https://glarity.app/tr) [dansk](https://glarity.app/da) [eesti](https://glarity.app/et) [español](https://glarity.app/es) [español (Latinoamérica)](https://glarity.app/es-419) [hrvatski](https://glarity.app/hr) [italiano](https://glarity.app/it) [latviešu](https://glarity.app/iv) [magyar](https://glarity.app/hu) [polski](https://glarity.app/pl) [português (Brasil)](https://glarity.app/pt-BR) [português (Portugal)](https://glarity.app/pt-PT) [română](https://glarity.app/ro) [slovenský](https://glarity.app/sk) [slovenščina](https://glarity.app/sl) [suomi](https://glarity.app/fi) [svenska](https://glarity.app/sv) [中文 (简体)](https://glarity.app/zh-CN) [中文 (繁體)](https://glarity.app/zh-TW) [日本語](https://glarity.app/ja-JP) [한국어](https://glarity.app/ko) Older version of the logo
glarity logo older
# User Guide Glarity - Summary for Website with ChatGPT (AD-Free) ## Glarity User Guide ### First Use How to use Glarity-Summary for Google/YouTube (ChatGPT)? 👉[Video](https://youtu.be/jls2He4TTAU) ### Advanced Use Glarity Youtube Summary: Customize Your Summary to Match Your Reading Habits👉[Video](https://youtu.be/84cPgNI_1KM) # FAQs Glarity Summary - FAQs ### Glarity does not work #### 1. Unable to select the OpenAI API model ![](/files/hOcC8oo0rhhiVDkXNtXy) {% hint style="info" %} Please delete and reinstall the plugin. {% endhint %} #### 2. {'error":('message":"Invalid URL (POST /v1/chat/completions)" ![](/files/DH5iuGVrtuR2PvlhghKt) {% hint style="info" %} Please delete and reinstall the plugin. {% endhint %} #### 3. Please login and pass Cloudflare check at [chat.openai.com](https://chat.openai.com/) ![](/files/IgkqIowxY1YnlMSoAGcR) {% hint style="info" %} Open the chat.openai.com website and login, then return to see if it works. {% endhint %} {% hint style="info" %} ![](/files/kVHNtEBPhmo9HYaCsXCC)\ \ If it still doesn't work, go to Extension Options and set the AI Provider to OpenAI API, then enter the Key and return to retry. {% endhint %} ### Usage Tips #### 1. YouTube Custom Prompt 1. Open the extension Options and find "Customize Prompt for Summary(YouTube)". 2. Copy Prompt.\ ![](/files/iXMyVU0dPlHwmyOCJqfK) 3. Paste or type Prompt, then save.\ ![](/files/K67GfhWC6GFXoKD4GJIb) # Summary List Comment Summary （update 2024.02.06) 📄 Web Summary ✅ Generate webpage summaries for any webpage. ✅ Engage in conversations with AI about web content, supporting custom prompts such as extracting page content. Including but not limited to: ### NEWS Website * [x] Yahoo news * [x] yahoo.com * [x] yahoo.jp * [x] msn.com * [x] nytimes.com * [x] bbc.co.uk * [x] cnn.com * [x] bbc.com * [x] dailymail.co.uk * [x] theguardian.com * [x] foxnews.com * [x] usatoday.com * [x] washingtonpost.com * #### tiktok ![](/files/h8pTfswpWA5NNNVBn0HZ) * [x] twitter\ ![](/files/PdTKm4LPB4aYzQZESPuo)![](/files/KZ4nvzW6ou3kRjPzRYGj) * [x] reddit\ ![](/files/hRwChHYHwiFC1gWcmi8Q) * [x] tripadvisor\ ![](/files/9d2pJjBYtkXgW2RCk6dK)![](/files/oYqA2Q2Wp9lCOlalEAbZ)![](/files/hY7xDfCZHgEfyR0PCSEt)![](/files/i8BTbVBwC9EypItrgMeL) * [x] imdb\ ![](/files/HVXTgCaBvpzJLR7YoxRd)![](/files/Xbp8AwNnWCx5aKyI3I8W) * [x] ebay\ ![](/files/B8JjDLvIGGNAkRXVz9XM)![](/files/1ukaALUri84c9sxrDnGi)![](/files/J0UgX3NjWOwIO0Mq2zzk)![](/files/p4px0xzmbGnyNou9tqLV) * [x] booking\ ![](/files/4OhbjOS6fk7IDTG7Phir)![](/files/hEYFBl85q2fizxQVekOX) * [x] douban\ ![](/files/fw0wWJ0Tiaul5ykeJcFK)![](/files/6RwTeZJQJMgSeJnZFYtq) * [x] amazon\ ![](/files/TvzxwxWQqyCJKaDMeZRC) * [x] temu\ ![](/files/7IoOXKTiVsW385nUQ2Gq) * [x] 京东\ ![](/files/sKlv1pmCkvDoEO8g25i7) * [x] 天猫\ ![](/files/uEeKI9pDuQEc4A8fq4OW) * [x] 淘宝\ ![](/files/qaeQqhy3RNbw5PIOcF9h) * [x] yelp\ ![](/files/qhJEKP1bLUqOIUDQ6PGV)\ ![](/files/jJ3BmlHfpOpnc3gJ5Wtq) * [x] meituan\ ![](/files/YS1PJojxuSH7XgnkwcBh) * [x] dianping\ ![](/files/6Cw8DF01axnIMpzjXZ2X)![](/files/8Wn2HAgg4cEBqg1vroWU) * [x] quora\ ![](/files/NeL0lsyLIfFrUny2iMuj)![](/files/gk2Gxk9dSLb08ToPC5AB) * [x] zhihu\ ![](/files/ch6EIflFxhPfeYmLwMJe)![](/files/wGVnSKzGFpJbWrEy97UV)![](/files/vUnvw3j6gwzp6sLqJYtE) * [x] gamespot\ ![](/files/Sd28VOAgUvGqfiTJeZwZ)![](/files/oUTSenIaoZIq2E2ZGc2c) * [x] IGN\ ![](/files/M4REO35uuZnsbVryg3GD) * [x] tabelog\ ![](/files/GOxl99W9xiupzBeMHMqQ) * [x] mercari\ ![](/files/XkRUfu1OFPWhieY25TMy) * [ ] todo **Glarity Summary for Website List (2023.03.06)** ### Search Engine * Google * Bing * Baidu ### Video Site * YouTube ### News Site * *Yahoo! JAPAN News* * *Nikkei* * *NewsPicks* ### Academic Website * *PubMed* * PMC * Google Patents ### Developer * *GitHub* # Redeem Code ## [Redeem Code Tutorial](#how-to-use-the-glarity-pro-redeem-code) | [兌換碼使用教學](#ru-he-shi-yong-glarity-pro-dui-huan-ma) | [コード・チュートリアル](#glarity-pronoridmukdono) \----------------------------------------------------------- **Redeem Address**: （You need to be logged in to your Glarity or Felo.ai account to redeem) ## How to Use the Glarity Pro Redeem Code The Glarity Pro redeem code allows you to experience our premium services, originally priced at $14.99 USD, for free. If you have this redeem code, follow the steps below to redeem it: 1. **Open the Glarity extension settings page, click on "My Account" to enter the Glarity personal dashboard.** * ![](/files/eNjA9qvKre0qgSeGmfpv) 2. **Click on the "Redeem Code" menu.** 3. **Input the "Redeem Code" and submit it.** ![](/files/kU1U27N9q12Tv9oMOuLe) 4. **Done, you've successfully gained Pro trial access.** Note: The duration of access might differ according to the specific redeem code. {% embed url="" %} ## Glarity Proのリデームコードの使用方法 コード使用アドレス: (コードを使用する前に、GlarityまたはFelo.AIアカウントへのログインが必要です。） Glarity Proのリデームコードを使用すれば、元々14.99ドルのプレミアムサービスを無料で体験できます。既にこのリデームコードをお持ちの場合、以下の手順でそれを使用することができます： 1. **Glarityの拡張設定ページを開き、"My Account"をクリックしてGlarityのパーソナルダッシュボードに入ります。** 2. **"Redeem Code"のメニューをクリックします。** 3. **"Redeem Code"を入力して提出します。** 4. **終わりました、あなたはProの体験アクセスを正常に取得しました。** 注意：リデームコードにより、アクセスの期間が異なる可能性があります。 ## 如何使用 Glarity Pro 兌換碼 兌換網址： （請先登入您的Glarity或Felo.ai帳戶以進行兌換） Glarity Pro兌換碼讓您免費體驗我們原價14.99美元的高級服務。若您擁有此兌換碼，請按照以下步驟進行兌換： 1、開啟Glarity擴充程式設定頁面，點擊“我的帳戶”進入Glarity個人儀表板。 2、點擊“兌換碼”選單。 輸入“兌換碼”並提交。 完成，您已成功獲得Pro試用權限。 注意：存取時間可能根據特定兌換碼而有所不同。 # Subscribe Glarity Mode **What is Glarity Mode**? Glarity Mode is an extension that offers users a new AI-enhanced linking service, allowing users to access more stable AI services in addition to their existing ChatGPT and OpenAI key. Glarity harnesses a variety of AI models to provide advanced AI-assisted services. ### **How to Register & Subscribe to Glarity Mode?** 1. After you have exhausted your anonymous trial quota, you can log in to your Glarity account to receive monthly free tokens. Click "[Sign in now](https://account.felo.ai/)" to access the account registration and login page, as shown in the image below. ![](/files/NZaQ5ovhkLLNkzAs8cER)![](/files/uUdScVv9ksBsG3rDn5xq) 2. [**Create your own Glarity account**](https://account.felo.ai/) to enjoy Glarity services better. (We recommend using your Google account for convenient authorization, eliminating the need for a password.) (Note: Glarity is a product under felo.ai, so you can use your felo.ai account for service.) 3. In the Glarity extension settings or on the [Pricing page](https://glarity.app/pricing), you can view your current plan and upgrade to a more suitable one at any time. Click "**Upgrade Now**" to **start your subscription**. ![](/files/LT9YRQ5hP48VtbuOpzx2)![](/files/8fP5hD9Jc0SCNg5FlAIt) 4. Glarity partners with the world's leading payment service provider, **Stripe,** to offer subscription services, ensuring the security of your funds and service experience.
5. After successfully binding your card and subscribing, you can **enjoy Glarity Pro/Plus services.** *** ### **How to Use Your Discount Code When Subscribing to Glarity Plan?** 1. **If you have a discount code(promotion code) :** Choose your subscription plan and proceed to the checkout page. You will find an option to enter your **discount code** (**promotion code**) , as shown in the image below.
2. **Enter your discount code (promotion code) and click "Apply".** If your code is valid, the subscription price will be adjusted accordingly.
**Please Note:** 1. **Different from free Redeem Code,** discount codes for subscription plans should not be used interchangeably. 2. **The discount level may vary for codes obtained during different promotions.** 3. **Discount codes are not case-sensitive.** 4. **Each discount code has specific usage conditions,** so please pay attention to the details provided in different promotional materials. 5. **If you unsubscribe after using a discount code,** only the actual paid amount will be refunded. If you have any questions or suggestions while using Glarity, please feel free to contact us. Glarity is here to[ support](https://account.felo.ai/scta) you anytime. # Glarity Plan FAQs ## **Englsih** **Q: What are the differences between the Glarity Plan and the chatgpt/openai key model?** **A:** The Glarity Plan offers greater stability and speed compared to the chatgpt/openai key model. It also supports longer text and video summarization, and is capable of handling multiple summarization tasks simultaneously. In the future, the Glarity Plan will include additional exclusive features. **Q: Why does the token consumption exceed the word count of the text?** **A:** The token consumption comprises two components: the queries made to the AI and the content returned by the AI. This includes not only the text but also some portions of instructional content. Therefore, it consumes more tokens than what is calculated based solely on the text. ## 日本語 **質問：Glarity Planとchatgpt/openai keyモデルにはどのような違いがありますか？** 回答：Glarity Planは、より安定しており、より高速です。さらに、より長いテキストやビデオの要約をサポートしており、複数の要約タスクを同時に処理できます。 今後、Glarity Planにはさらに専用の機能が追加される予定です。 **質問：トークンの消費量が記事の文字数よりも多いのはなぜですか？** 回答：それはトークンの消費がAIにクエリを送信する部分とAIから返ってくる内容を含むからです。テキスト以外にも、いくつかの指示が含まれています。そのため、テキストの文字数よりも多くのトークンが消費されるのです。 # ChatGPT Plugin - Glarity Videlo Summary Glarity ChatGPT Plugin: Video Summary allows users to capture key highlights from YouTube videos effortlessly. By installing the plugin and following simple steps, users can summarize videos. This plugin can also be combined with other plugins, such as the Show Me plugin, to create rich content like video mind maps. {% embed url="" %} ### How to access the ChatGPT plugins store and add plugins 1. Click on the **No Plugins Enabled** message followed by **Plugin Store**. 2. If it’s your first time visiting the store, you may see an information box pop up: About Plugins. You can choose to read now, but it can always still be accessed on the store to the bottom right of its screen. 3. You can now proceed to the **Plugin store** to search and add any plugins you’re interested in by hitting the **Install** button. Repeat this process to add new plugins. 4. Exit the screen by clicking on X and you should now see a series of icons beneath the GTP-4 tab. The plugins have been successfully added. ![You can search for and add plugins via the Plugin Store](https://ph-files.imgix.net/d95a34a3-2cac-40ca-93f0-be4aaf3d0936.png?auto=compress\&codec=mozjpeg\&cs=strip\&auto=format\&fit=max) You can search for and add plugins via the Plugin Store # Glarity Firefox Version Glarity Firefox Version download (Update 2023.11.07) **Glarity Firefox Version** Due to temporary incompatibility with the new version of ChatGPT, the Firefox browser has been temporarily removed from the supported platforms. During this period, you can opt for the Chrome version. If you still wish to use the Firefox version, you can download the installation package and install it manually: **Glarity Firefox Version download**
**Installation Instructions:** **Step 1:** Download the Glarity extension installation package. **Step 2:** Open the Firefox browser's Extensions and Themes page. Click the menu button in the upper right corner of the Firefox browser (usually three horizontal lines or a Firefox icon), then select "Add-ons" (in older versions, it might be labeled as "Plugins" or "Extensions").
**Step 3:** Install the local extension package. On the Extensions and Themes page, you will see a gear icon (in newer versions of Firefox, this icon is located at the bottom of the left sidebar). Click on this gear icon, then select "**Debug Add-ons**".
**Step 4:** Click "**Load temporay add-on.**.". Choose the local extension package file. # Update history Glarity - Summary for Website with ChatGPT (AD-Free) Update history **Weekly Update** **Version 4.5.5 Update (02/06/2024)** **Enhancements and Fixes:** General Improvements: * Resolved issue where ChatGPT Plus accounts were unable to summarize. * Enhanced translation feature with the ability to toggle tips on/off. * Optimized copilot input interaction for a smoother experience. YouTube Summarization: * Added new feature: FAQ video generation for YouTube summaries. OpenAI Key Mode: * Introduced a new option: Temperature parameter settings for OpenAI key mode. Bug Fixes: * Addressed various user experience issues. **Version 4.4.11 Update (01/29/2024)** 1. Optimize translation. 2. Provide answer options generated by GPT-4. Note: ChatGPT with GPT-4 support is on the way. Stay tuned! #### Version 3.43.4 (2023.10.20) Enhanced news site summaries: Now supports a wider range of news sites and enables cross-language summaries in various media ([refer to the visible sites list for specific details](https://blog.glarity.app/getting-started/user-guide/summary-list)). #### #### Version 3.38.18 (2023.9.26) Glarity Plan has been added to the AI mode #### Version 3.0.0 (2023.8.28) * Added YouTube AI chapter feature #### Version 2.6.0 (2023.7.13) * Extended interface, option to support multiple languages. * New users can experience Glarity for free without ChatGPT/openAI. #### Version 2.5.0 (2023.6.17) * PDF/Web Content/YouTube Full text summary support. * Fix:ChatGPT GPT4 request failed. * Add model gpt-3.5-turbo-16k. * Add AI chapter. * Add a read-aloud summary of content features. * The new Glarity height adjustment feature. * Optimise YouTube default prompt to output highlights by content, no limit of 3. * Fix:Some github projects do not show Glarity. * Q\&A to low cost model. ![](/files/Uqpy6kE7cxOgw3LXvcMX)![](/files/AhdkYm5MzZ4RpCSvlb5u) ![](/files/PGsfCe5avVGrQq1shgev)![](/files/0UxE6SyWDqBcfWchWkZn) #### Version 2.3.0 (2023.6.5) * Fix: The extension is making code blocks on every website unreadable, like corrupted #112. * Fix: GPT-4 API exception. * Support for Azure openAI API. * Fix: ChatGPT Plus member summaries are too long, and the default model causes the summary to be lost issuess. * The copy feature is supported by Q & A answers. * Fix: Render blank in Markdown. * Optimize: Rapid page switching causing previous requests to not finish, resulting in abnormal display on the new page. * Optimize: Enhance request hiding from the browser history, reducing the frequency of appearing in the history records. #### Version 2.2.0 (2023.5.30) * Refactoring the UI.\ ![](/files/Op2N22d1XEzGNYIDwURV)![](/files/LXrExZ1Ibv7PRE0lgoPT)\ ![](/files/EkqpmQuimLyVLkNWW8Tv)![](/files/ooAIOG1VqTGJ3Z6XUUxT)\ ![](/files/ls9f2JX4s9HSrwRa13qN)![](/files/aszrvjF0kUpNgQf6JEz9)\ ![](/files/McLsGfQIRLHifaaPQACv)![](/files/dwruH76SbFOjqGRcWU70)\ ![](/files/ZY1rFcuaAix9pxfQ7xVc)![](/files/O8C2hPViyRVrKmZz5DrJ)\ \ ![](/files/jnSQBCAqIwm8YrYvrcNL)![](/files/JzbyClQ4JF9micYZe2qI) * The light/dark mode supports the auto-follow system. * Code highlight.\ ![](/files/EUxNBcCMZi1PhQkArSdN)![](/files/kuNOvyNut8r4Qzvz6o4c) * Optimisation of output specified language success rate, tail enhancement. * Automatically removes spaces at the beginning and end of the API/Key. * Langchain supports custom API Hosts. * PDF.js loading failure prompt. * Automatic switch to API mode when entering host/key. * PDF auto-open summary. * Adding recently translated languages, fixing a second translation failure.\ ![](/files/wT6mPhViUFA7uWBM1xj3) * PDF encoding error issue * Optimize the plug-in volume and optimize the experience * YouTube dark mode plugin experience optimization #### Version 2.0.0 (2023.5.15) * Q\&A supports ChatGPT mode, contextual questioning is supported for the same conversation id.\ ![](/files/Axc9ej8o9kXo2YDYBWSJ) * ChatGPT mode obtains the maximum max\_tokens through the API, and some users support a maximum of 8191 tokens. * Beautify Glarity scrollbars. * Fix Google search result summary URL reference invalid. * YouTube doesn't have subtitle summaries * Icon sidebar snapping feature * Ctrl + M switch icon feature #### Version 1.17.0 (2023.5.6) * Support for a full range of content for Q\&A.\ ![](/files/n5UHAhS0mOMJInFqtKhz)![](/files/E0GfDbHDaQHeDQEfXyop) #### Version 1.16.0 (2023.4.25) * Summary of online/local PDF documents.\ \ ![](/files/AI9UIcvWUB4qC3rAdPWn)![](/files/V9EbLL6QByQJXuNlhEca) #### Version 1.15.1 (2023.4.25) * Optimising Page Selection interactions to reduce distractions, click on the icon to display the menu.\ ![](/files/L75x3mR471ZUeRTiNeKY) * Optimize "Create a tweet" Prompt.
#### Version 1.15.0 (2023.4.24) * Page Selection Enable/Disable Features.\ ![](/files/TpAZuHUNej57o7VHRDkQ)![](/files/gHkTlwk4AppeQ5lUFtfl) * Page Summary Typography Optimisation.\ ![](/files/sCH3HkKOmXXInvCN5qJz) * Maximise the use of tokens and limit custom Prompt to 400 tokens. #### Version 1.14.0 (2023.4.21) * Quick access to Q & A, Create a tweet, 3 bullet points summary.\ ![](/files/IRtdyFNUkh4QKCBdvxvx) * Select web text to summarise, explain and translate.\ ![](/files/ubTSvBrKM7dfoBTiI60X)\ ![](/files/gUZSkBuOflshpPEP4SaJ)\ ![](/files/LMrXLb9a9ubEm2YPiCKE) #### Version 1.13.0 (2023.4.18) * Support for stop response feature.\ ![](/files/P4uV6Qfr235XyOcepEmp) * Page Summary supports the movement of Glarity Icon and Box positions.\ ![](/files/5Bs5ayiulMqD2TJn7PBc) * Support for presearch search result summaries.\ ![](/files/yZY4LADOiIc3FoQmqXim) * Supports setting the height of the Glarity Box.\ ![](/files/Rwjg7wTNiCfpfPM6Q6EC) * Tabelog/mercari reviews summary.\ ![](/files/A0meP1px32iYUz5grsKm)![](/files/qWKNEL6FNNPXOavJq9up)
#### Version 1.11.0 (2023.4.12) * Tripadvisor/ebay/booking/yelp/quora/temu/meituan/dianping/douban/zhihu reviews summary.\ ![](/files/TiPDy79pLtH0xJVuATXC)![](/files/wj2zZe8IDDyjl3YQ3ASm)![](/files/v9SD2EiMkXEtbpymbIUB) #### Version 1.10.0 (2023.4.10) * ChatGPT supports switching GPT-4/GPT-3.5.\ ![](/files/NfWPxPQGU3he2tRORGRy) * JD/Taobao/Tmall product reviews summary.\ ![](/files/UqCBg4o4R6IPjS2pej7x)\ ![](/files/35qUI7TXDCeaPZkToi6r)![](/files/WudBTr798UkuOc3IcxZ1) #### Version 1.8.0 (2023.4.6) * Add summary translation.\ ![](/files/QwEgsJ0hEfurY7zwTkch) #### Version 1.7.0 (2023.4.3) * Custom Comment Summary Prompt.\ ![](/files/nOFnlaSV9rFkocQ4yWLD)![](/files/gREMQYMpso3BUDYCBPvx) * Custom API host.\ ![](/files/5aT2EntXwjSfVr8lbA53) * Hide Page Summary icon Update interaction, add description.\ ![](/files/8RIJiqHgNjJthC2Bmivo) #### Version 1.6.0 (2023.3.27) * Support YouTube Comment Summary. ![](/files/TTQYckXQYYgQrkiF26n3) #### Version 1.5.0 (2023.3.26) * Support Amazon product reviews summary. ![](/files/sEwnl8w1EY8ZIZFHFkIP) #### Version 1.4.2 (2023.03.24) * Page Summary supports custom prompts. * The Glarity box limits the maximum height and automatically scrolls to the latest content when it is exceeded. #### Version 1.4.0 (2023.03.24) * Google / Bing search support for custom Prompt. * Page Summary support for switches and blacklists and whitelists. ![](/files/IvGhuZ6PXkCBXEWyBfA8)![](/files/FdAl2cWYc4x0faVlDgNF) #### Version 1.3.0 (2023.03.21) * Support bilibili. * Optimize page summary prompt to avoid excessive content length. * Support for YouTube transcriptions based on timeline playback. ![](/files/cKo52tjCE4pJatVAd0HI) #### Version 1.2.1 (2023.03.17) * Support for iOS Safari and MacOS Safari. ![](/files/Mxg73CRGNPPMQNB806jh) ![](/files/KsPOKUH0l1JreiKgVQUB) ![](/files/iP4rFio444CAxrb59BTe) ![](/files/wrWkHSaPXdV9lXmsoIuj) ![](/files/pQjNEot32miY2isAIrKs) ![](/files/J2ub8Yr6fkA9A9stwSdX) ![](/files/ANcxsewtrTEelflFkA05) #### Version 1.2.0 (2023.03.15) * All page summary features. ![](/files/gAprb8D4Q1DaFa2uqKhj)\ ![](/files/ttJcTX9u5C8Q8nxWPRIk)\ ![](/files/wVWDVeNjRBjI8rlzgNN6)\ ![](/files/JlmNrr29v9iPjGWeUYPP) #### Version 1.1.8 (2023.03.09) * More references when searching Google. * Fix: Glarity may not be displayed if the optimisation network is poor. * Fix: #37. * Improve the stability of ChatGPT mode and optimize the tab opening method to avoid repeated opening. #### Version 1.1.7 (2023.03.07) * Supports bing. * Support Site Enable/Disable Glarity. * Supports Google Patents summaries. * Fix: Style global pollution. * Fix:Long video transcript does not show hours. * Fix:#24 Long video crash issues. * Support switching between gpt-3.5-turbo and text-davinci-003 models.
#### Version 1.1.6 (2023.03.06) * Supports **NewsPicks** article summaries. * Supports the official OpenAI API (GPT-3.5-turbo). * Fix: OpenAI API selection issues. * Supports Nikkei article summaries. * Supports GitHub project summaries. * Supports PMC article summaries. * Fix reference links 404.
#### Version 1.1.5 (2023.03.04) * Fix the reply translation issue * Fix tokens limitation * Add traditional Chinese to ChatGPT response #### Version 1.1.4(2023.03.01) * Supports Yahoo! JAPAN News summaries. * Supports PubMed free article summaries. * Customize Prompt updated UI and interaction, added Clickbait scoring. * Add refresh request ChatGPT feature. * Fix the problem that there may be a : sign at the beginning of the answer. (Update Instructions Video #### #### # Google Search Summary Glarity's Google Summary: Elevate Your Search Experience **Transform Your Search with Precision and Ease** In today's fast-paced digital world, accessing precise information instantly has never been more crucial. Glarity's innovative "Google Summary" feature is designed to redefine your search experience by delivering direct answers to your inquiries, sourced from the vast expanse of the internet. This cutting-edge functionality stands out by providing accurate answers and relevant references, ensuring you receive the most reliable information at your fingertips. [**Install now**](https://glarity.app/)
**Key Features** * **Direct Access to Answers:** Google Summary by Glarity offers a streamlined search experience that cuts through the clutter to present you with the exact information you seek. Our technology sifts through the web's extensive resources to bring accurate answers and essential references directly to you, saving time and enhancing your efficiency. * **Cross-Language Search and Answers:** Understanding and communicating in multiple languages is a significant asset in our globalized environment. Google Summary breaks down linguistic barriers by generating and translating search results and answers across various languages. Whether you're conducting research, learning a new language, or working internationally, this feature ensures you can access and understand the information you need, regardless of language constraints. **Why Choose Google Summary?** * **Efficiency and Speed:** In an era where time is of the essence, Google Summary optimizes your search process by eliminating unnecessary steps and waiting times. Get straight to the point with answers that matter without sifting through pages of irrelevant information. * **Accuracy and Reliability:** Our advanced algorithms are designed to prioritize accuracy and relevance in the information provided. Trust in a tool that offers dependable answers drawn from a comprehensive analysis of available online resources. * **Versatility in Language:** Expand your reach and understanding with our cross-language functionality. Access information not just in your native language but in multiple languages, enhancing your knowledge and capabilities in a multicultural world. * **Informed Decisions:** With precise information and a broad perspective, make more informed decisions, whether for academic research, professional projects, or personal curiosity. Google Summary empowers you to act based on accurate, well-rounded insights. **Embrace the Future of Search** Glarity's Google Summary is more than just a search tool; it's a gateway to a world of information, tailored to your needs and language preferences. By choosing Google Summary, you're not only streamlining your search process but also embracing an advanced approach to information gathering and understanding. Elevate your search experience with Glarity today and discover a world where answers are just a click away, transcending language barriers and redefining efficiency in digital exploration. # Gmail Quick Reply ## Turbocharge Your Email Replies: Glarity Gmail quick reply In today’s ever-evolving digital age, effective email communication can be a time-consuming task. GLARITY's Gmail Quick Reply is here to change that, offering an AI-powered, privacy-focused email assistant specifically for Gmail. Powered by OpenAI's ChatGPT, this revolutionary Chrome extension is crafted to elevate your email response experience like never before.
### What GLARITY Gmail Quick Reply Brings to Your Inbox GLARITY Gmail Quick Reply is no ordinary email assistant. It is filled with innovative features meant to streamline your email correspondence: 1. **AI-Powered Responses:** The assistant provides top-notch responses that outrank any other tool currently available. It’s akin to having your personal linguistic expert straightening out any errors and suggesting more effective wording. 2. **Context-Smart Reply Suggestions:** GLARITY Gmail Quick Reply excels at curating reply suggestions based on the context of the received emails, helping you to construct impressive replies without starting from scratch. 3. **Multilingual Support:** The tool is capable of assisting in all languages, offering you robust support no matter what language your email is in. 4. **Toggle On or Off:** The handy feature allows you to adapt the tool into your workflow. Enable it when required, and disable it when not in use. ### Your Guide to Utilizing GLARITY Gmail Quick Reply Implementing this AI-powered tool couldn’t be simpler. Here's how: 1. Open an email in Gmail that requires a response. 2. click the" idea for reply". Instantly, Glarity will spin into action, curating your email response meticulously. 3. Scrutinize the AI-constructed draft that's presented to you. If it meets your standards, simply click on it to turn it into your email response. ### The Competitive Edge of this Chrome Extension Here are the unique advantages that make this tool a standout: ✔️ Impeccable quality responses that beat other tools. ✔️ Seamless integration with Gmail. ✔️ Comprehensive language support. ✔️ Reply suggestions based on the context of received emails. ✔️ Easily switchable Toggle On or Off feature. ### Navigating GLARITY Gmail Quick Reply within Gmail Here’s how you can master the use of this tool within the Gmail environment. 1. Launch Gmail. 2. Open an email that needs a reply and click on the "quick reply" button. 3. Immediately, you will see a suggestion from Chat GPT to construct your email response. 4. If you agree with the suggested response, simply click on it. 5. You can now choose to either copy the suggested content or straightaway insert it into your reply box. Amplify your email response efficacy with the assistance of an AI. Upgrade your email correspondence with GLARITY Gmail Quick Reply for a smarter, speedier, and more sophisticated Gmail experience! # Side-by-Side Translation ## Glarity Side-by-Side Translation: Read the World in Your Mother Tongue！ Ever found yourself twisting and turning, perplexed by the overwhelming language barriers when browsing foreign websites? Ever wished for an effortless way to compare a translation with the original text to fully grasp the content? Today , **Glarity Side-by-Side Translation**, a tool reshaping and transcending online communication, propelling us to uncharted realms of language understanding with mirror translation and immersive translation experiences. {% embed url="" %} ### Free and powerful **Side-by-Side Translation** Glarity Side-by-Side Translation, a cost-free, potent tool that is revolutionising online language understanding. This AI-driven extension, integrated with the GPT and LLM systems, offers instinctively natural translations, providing you with an effortless navigational experience on foreign websites. Its unique mirror translation feature enables a visual comparison of the original and translated text simultaneously. So regardless of your language proficiency, Glarity's free and powerful Side-by-Side Translation allows you to delve headfirst into the vast ocean of multilingual web content. ### Mirror Translation: Your Learning Companion Armed with an ingenious design feature, **Glarity Side-by-Side Translation** stands apart from its peers. It doesn’t just translate text — it employs mirror translation to display both the original and translated text simultaneously, delivering an immersive translation experience. The unique bilingual side-by-side layout allows for on-the-spot comparison, fostering better understanding and making foreign language learning easier. ### Custom Optimization for Mainstream Websites To promote a unified, streamlined browsing experience, **Glarity Side-by-Side Translation** comes custom-optimized for various mainstream websites such as Google, Twitter, Reddit, Yahoo, and Wikipedia. This optimization breathes life into more efficient searching, nurturing global connections, and making information gathering a cakewalk. **How to Use Side-by-Side Translation** It's incredibly simple to start taking advantage of Glarity's Side-by-Side translation. First, you'll need to [add the Glarity extension to your Chrome browser](https://glarity.app/). Once it's installed, open a webpage with text you need translated.
To activate the translation function, simply right-click anywhere on the webpage. Instantly, you'll see the translated context running parallel to the original text. Its intuitive design allows you to ease through the translation process while ensuring minimal interruption to your browsing experience. Navigate the multilingual web effortlessly with Glarity's free and powerful Side-by-Side Translation. **Try Glarity Side-by-Side Translation Now**! # Chat and summarize with PDF ## Mastering PDF Summaries with Glarity: A User-Friendly Guide Every day, we interact with an ocean of PDF documents. From academic papers and books to business contracts and technical manuals, our digital world is steeped in valuable information locked in PDF format. But sifting through these mammoth files can be daunting. That's where the Glarity PDF Summary feature steps in, offering an effortless and cost-free way to interact with your PDFs. ### Glarity Product Breakdown Glarity harnesses the power of ChatGPT to offer an intuitive and easily accessible PDF summary functionality that doesn't require registration. It not only extracts key points from your documents but also offers a smart interaction feature, allowing you to engage in unrestricted question-and-answer interactions with your PDFs.
Glarity's notable features include: #### Cross-Language Summaries No need to worry about language barriers. Glarity spans across languages, translating and summarizing key PDF content into easily digestible insights. #### Versatile Applications Students, researchers, professionals, or simply curious minds - everyone can benefit from Glarity. Its versatility enhances information retrieval for diverse purposes with guaranteed accuracy. #### User-Friendly Interface The Glarity interface is designed for ease of use. You don't need to be tech-savvy to utilize Glarity; it caters to users of all experience levels. #### Fully Automated Smart Interaction Experience automated responses thanks to Glarity's intelligent technology. Ask any question related to your PDF content and get rapid, accurate answers. #### Supports Diverse PDF Formats Books, research papers, manuals, or legal contracts - Glarity has it all covered. It's designed to handle an array of PDF documents, adapting to diverse user needs effectively. ### Tutorial: Using Glarity PDF Summary Ready to dive in? Follow these simple steps to access effortless PDF interactions: 1. Open your desired PDF in any web browser. 2. Click on the Glarity icon on your toolbar. 3. Patiently wait for Glarity to complete the analysis. 4. Once the analysis is complete, engage in unrestricted question-and-answer interactions with the PDF. Easy, right? With Glarity, delving into the heart of your PDF documents has never been more seamless. ### Video Guide For a more visual experience, don't forget to check our [video tutorial](https://www.youtube.com/watch?v=XBPuhwwMGxQ) that walks you through the entire process in real time. {% embed url="" %} Chat with PDFs {% endembed %} In conclusion, Glarity is your personal AI-enabled assistant that breaks down language barriers, extracts key information from PDFs, and delivers concise insights, all for free. So why wait? Start using Glarity today and transform your PDF interactions. # Twitter Summary ## Harnessing the Power of AI With Glarity's Twitter Summary Feature Engaging with Twitter threads has just been revolutionized, all thanks to Glarity's Twitter Summary feature. Designed to break down language barriers and offer a novel perspective, the Twitter Summary feature lets you interact with Twitter content like never before.

Twitter Summary

### Understanding Glarity's Twitter Summary Feature One of the most exciting features of Glarity, the Twitter Summary feature, has been a game-changer in the way we consume and understand Twitter threads. Powered by advanced AI technology, this unique tool swiftly summarizes Twitter threads in multiple languages, allowing you to tap into diverse content, opinions, and discussions effortlessly. #### **Dive Deep Into Multilingual Summaries** Navigating the vast, global spectrum of Twitter content can be challenging, especially when language barriers arise. With Glarity's Twitter Summary feature, you can now make sense of Twitter threads in languages beyond your own. This feature swiftly generates a summary of the main posts in multiple languages, providing you easy access to diverse content, and facilitating a cross-cultural Twitter experience. #### **Navigate the Tweet-sphere with Ease** The Twitter Summary tool doesn't just stop at summarizing threads. It empowers you to understand the nuances of opinions and evaluations with an insightful comment analysis, paving the way for meaningful conversations. #### **Experience Enhances Engagement** Experience a truly global perspective with Glarity's Twitter Summary feature. Its ability to break language barriers allows you to engage with Twitter content from around the world, fostering an understanding of diverse cultures and viewpoints. Witness how effortlessly you can engage with users by leveraging the understanding of their perspectives and feedback. ### Getting Started with Glarity's Twitter Summary Incorporating Glarity's Twitter Summary feature into your Twitter routine is as simple as three steps: 1. **Install Glarity**: First off, get Glarity installed on your device. 2. **Visit Twitter Threads**: Then, simply visit the Twitter thread that you want to summarize. 3. **Let Glarity Do Its Magic**: Glarity automatically displays a comprehensive summary. For more on how this exciting feature works, feel free to **watch our video tutorial here**. {% embed url="" %} Twitter Summary {% endembed %} The novelty of the Twitter Summary feature lies in its sophistication and simplicity — harnessing the power of AI to make global Twitter discussions seamlessly accessible and understandable. Take your Twitter experience to the next level with the Glarity Twitter Summary feature today. # News summary ## Efficient Cross-Language Reading: News Edition ### **Explore the world, uncover the truth! Easily read foreign news with Glarity - helping you understand the global perspective.** \ Glarity utilizes the cross-language capabilities of GPT to bring the world closer and make understanding different cultures easier. Committed to breaking down language barriers, Glarity delves into various news events, whether on a global scale like the Israel-Palestine conflict or the Russia-Ukraine conflict, or in daily life. Using the powerful GPT technology, Glarity can instantly summarize news from different languages, meaning you can effortlessly comprehend and grasp the essence of news, no matter its source language. Say goodbye to language barriers, Glarity opens the door for you to explore the world!
#### Glarity: Easy Reading of News in Different Languages, Explore the World, Endless Excitement Glarity allows you to see and hear voices from around the world, presenting you with diverse perspectives and news events. Whether it's from yahoo.com, nytimes.com, bbc.co.uk, or cnn.com, Glarity supports cross-language summarization and translation of the most influential global news reports, enabling you to deeply understand important news and be informed about what's happening in the world in real time.Currently, supported media websites include but are not limited to: * yahoo.com * yahoo.jp * msn.com * nytimes.com * bbc.co.uk * cnn.com * bbc.com * dailymail.co.uk * news.google.com * theguardian.com * foxnews.com * usatoday.com * washingtonpost.com \*We are continuously adding more news websites. If you have specific requirements, feel free to reach out to us on Twitter. We prioritize generating summaries for your favorite news sites.
#### Real-time Cross-Language Summarization, Grasp the Essence of News Glarity, with its powerful GPT capabilities, supports output in over 90 languages, enabling real-time cross-language summarization of news content. This means, regardless of the source language, you can effortlessly comprehend and understand the essence of news. No more worries about language barriers, Glarity opens the door for you to understand the world!
(Summary in Japanese)
(Summary in Spanish)
#### Side by Side Translation, Native Language Assistance in Reading Foreign News To provide you with a deeper understanding of news events, Glarity also offers GPT translation functionality. Through bilingual comparison reading, you can more accurately grasp the original meaning of the news and understand the author's perspective. This unique reading method allows you to truly immerse yourself in the world of news and feel the power of every viewpoint. (This feature is currently in beta testing and is expected to be available next week. If you're interested in experiencing it early, you can join our Discord channel.)
\ Whether it's brief Twitter news or in-depth long-form articles, Glarity will provide you with accurate summaries and translations. It is your efficient news reading assistant, guiding you to understand the world and uncover the truth. \ Join us now and explore the infinite possibilities of this world together! Glarity, easy reading of foreign news! Understand the world, uncover the truth! # AI Search Engine 多语言评测报告 (v 1.0) ## 一、**摘要** 随着ChatGPT的问世，基于对话的搜索引擎技术迅速获得了广泛关注。市场上已经出现了多个通用型问答搜索引擎，如Perplexity、iASK，以及专注于特定垂直领域的搜索解决方案。 \ 我们认为这类基于对话的搜索产品在提供直接答案方面，相较于传统的基于关键词搜索和用户自行浏览信息的搜索引擎，具有显著优势，并可能成为搜索技术发展的颠覆式范式。然而，在实际使用过程中，我们也注意到了一些问题，尤其是在答案的准确性和可靠性方面。不准确的答案和所谓的“幻觉答案”（即与用户查询不相关或毫无意义的答案）频繁出现，这严重影响了用户体验。 \ 鉴于全球用户群体的语言多样性，我们的评估报告选择了包括英语、日语、简体中文、俄语和繁体中文等在内的多种语言，对这些问答搜索引擎产品的准确性进行了初步的测试和评估。 \ 在本报告中，我们将详细介绍我们的评估方法、测试过程以及所得出的结论。我们的目标是为开发者、研究人员以及最终用户提供一个全面的性能评价，以便更好地理解这些问答搜索引擎在不同语言环境下的表现，并指出它们在当前阶段的局限性和改进方向。 \ 在本次对问答式搜索引擎的综合性能评估中，我们得出了以下主要发现： 1. 总体而言，各参评产品的表现均未达到我们的预期水平。然而，值得注意的是，一家主要服务中国市场的公司——秘塔（Metaso）在评估中整体表现最为出色，以微弱的优势超越了Perplexity。

图1 评测产品的准确率

2. 综合所有产品的数据分析结果显示，对于英文问题的回答准确率最高。相比之下，俄文问题的准确率最低，日文问题的准确率也相对较低。

图2 所有语言的准确率

3. 在语种细分的表现上，Perplexity在英文问题的回答上遥遥领先，而其简体中文的表现也相当不俗。Metaso在简体中文和繁体中文的表现上同样令人瞩目。然而，这两家公司在其他语种的表现均未达到及格水平。iASK的能力在各语种中表现为相对均衡，但整体上属于中等水平；You.com 则仅在英文问题回答上有较好的表现。

图3 评测产品在不同语言下的准确率

注1：本次评估的各产品，均选择的是免费版本（metaso选择的是深入模式）。针对Pro版本的评估，会在之后进行。 注2：本次评估仅关注答案的准确性，忽略返回语言和格式等其它方面。 ## 二、产品选择 \ 为了进行本次问答式搜索引擎的性能评测，我们参照了 aicpb.com 网站提供的AI搜索引擎领域的领先产品清单。在排除了传统关键词搜索引擎之后，我们精心挑选了市场上表现最为突出的四款AI问答式搜索引擎产品作为评测对象。 \ 这些产品被认为是行业内的佼佼者，它们的技术和市场表现代表了当前AI搜索引擎的最高水平。我们的评测旨在深入分析这些头部产品的性能，特别是在理解和回答用户查询方面的能力，以及它们在处理不同语言的查询时的准确性和可靠性。 \ 上述4款产品的具体信息如下：（参考自 aicpb.com 的统计） | 排名 | 产品名 | 分类 | 3月访问量 | 环比 | | -- | -------------- | ---------------- | ------ | ------- | | 2 | **Perplexity** | AI Search Engine | 64.14M | 25.17% | | 3 | **You.com** | AI Search Engine | 10.44M | 14.61% | | 5 | **Metaso** | AI Search Engine | 7.21M | 551.36% | | 7 | **iAsk** | AI Search Engine | 3.43M | 15.43% | ## 三、评测数据 \ 为了全面评估上述AI问答式搜索引擎产品在多语言环境下的性能，我们选取了英语、日语、简体中文等六种主要语言进行测试。多语言的测试用例分布情况如下： | **评测语种** | **语种占比** | | -------- | -------- | | 英语 | 20% | | 日语 | 20% | | 简体中文 | 20% | | 繁体中文 | 20% | | 俄罗斯语 | 10% | | 韩语 | 10% | | **合计** | **100%** | \ 在本次评估报告中，我们特别设计了五种不同的使用场景，包括实时新闻获取、本地信息查询、技术问题解答、产品特性询问以及商业咨询，以模拟用户在日常生活中可能遇到的查询需求。 | **内容场景** | **评测点** | **用例数量** | **数量占比** | | -------- | --------------------------- | -------- | -------- | | 技术咨询 | 评估在理解和回答专业技术问题方面的能力 | 20 | 20% | | 时事新闻 | 评估在跟踪、处理实时信息和提供最新新闻事件 | 20 | 20% | | 本地信息 | 评估对本地(如餐馆、商家、景点、交通等)的信息检索 | 20 | 20% | | 产品查询 | 评估提供准确和有用的产品信息 | 20 | 20% | | 商业咨询 | 评估在商业领域内，对业务进行专业的信息分析和检索的能力 | 20 | 20% | | **合计** |


| **100** | **100%** | \ 我们认识到，尽管这五种场景涵盖了一系列常见的查询类型，但它们可能无法完全覆盖用户的所有潜在需求。因此，我们计划在未来的评估中不断扩展我们的测试集合，引入更多的类别和场景，以确保我们的评估能够更加全面地反映产品的实际性能和用户体验。
我们的完整的测试用例，开源如下： 数据集：
## 四、**评测方法及结果** 1. **测试方法** * **准确率**：人工评测 人工将真实答案与Ground truth和Key words进行比较。工审核通过为True，否则为False。
* **Answer Correctness**: GPT4评分 选择Ragas平台的Answer correctness指标，评测模型采用GPT-4-Turbo，Answer correctness 计算公式如下：
指标说明： 2. **评估指标** | **评估内容** | **评估定义** | **优点** | **缺点** | | ------------------ | ----------------------------------- | ------ | ------ | | Accuracy | 人工基于Ground truth和key words对真实答案进行审核 | 评估精准 | 评估工作量大 | | Answer Correctness | GTP4基于Ground truth与真实答案进行对比 | 评估工作量小 | 评估质量一般 | \ 摘要中已经提供了Accuracy评测结果，Answer correctness评测结果如下所示。 |


| 准确率 | AC | | ----------- | --- | ------ | | Perplexity | 65% | 65.22% | | Metaso | 66% | 62.7% | | iAsk | 60% | 62.27% | | You.com | 34% | 58.27% | ## 五、**案例分析** ### 实时新闻 **Question：最近の埼玉県行田市事故の死傷者数は何人ですか?** **Ground truth：最近の埼玉県行田市の事故での死傷者数は以下の通りです:\n死亡者数: 1人\n負傷者数: 3人** **Comment：**Perplexity, You.com 没有搜索到相关结果。Metaso回答的结果跟Ground truth基本一致。iAsk回答为英文，但描述和回答来源正确。
Search AI EngineAnswerAccuracy
Perplexity
申し訳ありませんが、与えられた検索結果には埼玉県行田市での事故の死傷者数に関する情報がありません。3の埼玉県警察のウェブサイトには県内の交通事故発生状況が掲載されていますが、行田市に限定した情報はありません。したがって、この質問に対する正確な答えを出すことはできません。False
Metasoしかし、この質問に対する直接的な回答として、最新の事故で1人死亡、3人けがをしたことが正确です。True
iAskThe recent accident in Saitama Prefecture, specifically in the city of Gyoda, resulted in 4 people being killed or injured. The accident involved a 76-year-old female driver whose car veered out of its lane and collided with a total of 4 vehicles, including cars and trucks. The incident occurred on a single-lane city road in Gyoda. The woman’s vehicle crossed into oncoming traffic, leading to collisions with two oncoming cars, another car, and a large truck. As a result of the accident, a 47-year-old woman who was driving one of the oncoming cars tragically lost her life. Additionally, her 8-year-old daughter who was with her, the 52-year-old male driver of the truck, and the elderly female driver who caused the initial collision sustained varying degrees of injuries. The Gyoda Police Station is currently investigating the exact cause of the accident while monitoring the recovery of the injured parties. These sources were utilized to gather information regarding the recent accident in Gyoda City, Saitama Prefecture, providing details about the number of casualties and the circumstances surrounding the tragic incident.True
You.com最近の埼玉県行田市の事故の死傷者数に関する情報は、提供された検索結果からは見つかりませんでした。申し訳ありませんが、詳細な情報を提供することができません。お手数ですが、別の情報False
### 本地信息 **Question：進撃の巨人はいつ発行されましたか？** **Ground truth：**「進撃の巨人」は、諫山創による日本の漫画作品で、2009年9月9日に発行が開始されました。そして、2021年4月9日に連載が完結しました。最新刊（34巻）は2021年6月9日に発売されました⁵。この作品は世界中で非常に人気があり、多くの派生作品が生まれています。 **Comment：**在回答"進撃の巨人"发行日期的问题中，Perplexity和iAsk的回答时间都是正确的，其中Perplexity还提供了更详细的信息。而Metaso和You.com的回答则不准确。Perplexity的回答更能显示出其在信息搜集和处理方面的优势。
Search AI EngineAnswerAccuracy
Perplexity
進撃の巨人の原作漫画は、2009年9月から2021年4月まで講談社の「別冊少年マガジン」にて連載されていました。13 コミックスは全34巻で、最終巻である34巻が2021年6月9日に発売されました。2 つまり、進撃の巨人の漫画連載期間は2009年9月から2021年4月の約12年間で、コミックス化は2021年6月に完結しています。True
Metaso「進撃の巨人」の漫画版は2010年3月17日に発行されました。False
iAsk進撃の巨人は2009年9月に発行されました。True
You.com進撃の巨人は2013年に発行されました。False
### 技术咨询 **Question：**Anthoropic Claude のデータポリシーは？ **Ground Truths：**ユーザーの個人データは、モデルのトレーニングに使用されないため、ユーザーデータを利用してモデルを構築することはありません。 **Comment：**Perplexity的回答与题目无关，没有正确回答问题。Metaso的回答最为准确，详细地解释了Anthoropic Claude的数据政策。iAsk的回答虽然不在 **Ground Truths**，经核实它引用的来源，确认它的回答仍然正确。You.com则没有找到相关信息。
Search AI EngineAnswerAccuracy
Perplexity
Anthropic のAIアシスタントClaudeのトレーニングデータは2023年8月までのものであり、それ以降のイベントについては認識していないことが分かります。4 具体的には、Anthropic のサポート記事によると、「Claudeのデータは常に更新していますが、Claudeは2023年8月までのデータで訓練されました。この日付より後のイベントについて尋ねた場合、Claudeはそれらを認識していません。」と説明されています。4 つまり、Anthropic Claude のトレーニングデータは2023年8月までのものであり、それ以降の最新の情報については認識していないということが分かります。False
MetasoAnthoropic Claudeのデータポリシーは、業界標準のベストプラクティスを使用しており、ユーザーが入力したプロンプトや会話はモデルのトレーニングに使用されません。これは、用户的な情報の漏洩を防ぐための措置であり、用户的なデータの処理や保持に関する透明性と安全性を保証しています。これは、用户的な情報の漏洩を最小限に抑えるための措置であり、用户的なデータの処理や保持に関する透明性と安全性を保証しています。True
iAskAnthropic Claude のデータポリシーは以下の通りです: Anthropicは、個人データを保持する期間や使用方法について明確なポリシーを設けています。消費者向け製品やベータ/評価製品の場合、個人データは、目的や基準に合理的に必要な限り保持されます。一方、ビジネスやエンタープライズのお客様向けには、サービス契約でデータ保持期間が説明されています。 主なポイント: データ保持期間: 消費者向け製品やベータ/評価製品の場合、受信または生成後90日以内にバックエンドでプロンプトと出力が自動的に削除されます。 特別なケース: 例外として、信頼と安全の分類器にフラグが立てられたプロンプトは最大2年間保持されます。 その他の規定: オプトインや明示的同意がある場合、データは10年間保持されることもあります。 Anthropicは個人データの性質や関連活動、対話時期と期間、正当な利益と法的義務を考慮してデータ保持期間を決定します。法律上必要な場合や使用ポリシー違反を防ぐために必要な場合を除き、個人データは適切な措置の下で管理されます。True
You.comAnthropic Claude のデータポリシーについては、具体的な情報が見つかりませんでした。データポリシーに関する詳細な情報を提供することができず、申し訳ありません。Anthropic の公式ウェブサイトや公式文書を参照することで、より詳細な情報を入手することができるかもしれませんFalse
### 产品咨询 **Question：**갤럭시 S24의 화면 해상도는 어떻게 되나요? **Ground Truths：**갤럭시 S24 기본 모델의 화면 해상도는 2340 x 1080 픽셀입니다 **Comment：**Perplexity给出的是一个不存在的分辨率，而iAsk则提供了一个更高的分辨率，但实际上是错误的。You.com 的回答也不正确。Metaso表现最好，它提供了准确的分辨率和有关不同型号的额外信息。
Search AI EngineAnswerAccuracy
Perplexity
갤럭시 S24의 화면 해상도는 1440 x 3088 픽셀입니다.5 화면 해상도 갤럭시 S24는 6.1인치 OLED 디스플레이를 탑재하며, 1440 x 3088 픽셀의 높은 해상도를 지원합니다.5 이는 매우 선명하고 생생한 화질을 제공할 것으로 예상됩니다. 디스플레이 주요 사양 디스플레이 종류: 6.1인치 OLED5 해상도: 1440 x 3088 픽셀5 재생률: 120Hz5 따라서 갤럭시 S24는 QHD+ 해상도의 고화질 디스플레이를 탑재하여 영화, 게임, 웹서핑 등 다양한 멀티미디어 활용에 최적화된 시청 경험을 제공할 것으로 보입니다.5False
Metaso三星Galaxy S24系列的屏幕分辨率根据不同的型号有所不同。Galaxy S24采用了6.2英寸的Dynamic AMOLED屏幕，分辨率为2340x1080，支持120Hz自适应刷新率。而Galaxy S24+和Galaxy S24 Ultra则提供了更高的分辨率。S24+的屏幕分辨率为3120x1440，这是一个QHD+级别的分辨率。同样，Galaxy S24 Ultra也配备了分辨率为3120x1440的第二代动态AMOLED显示屏。因此，可以总结说，三星Galaxy S24系列中，标准版S24的屏幕分辨率为2340x1080，而S24+和S24 Ultra的屏幕分辨率为3120x1440。True
iAsk갤럭시 S24의 화면 해상도는 QHD+ (3120x1440)입니다. Top 3 Authoritative Sources Used in Answering this Question: BetaNews.net Samsung Newsroom Official Samsung Galaxy Unpacked Event 2024 Coverage The information regarding the Galaxy S24’s display resolution of QHD+ (3120x1440) was sourced from multiple reliable outlets, including BetaNews.net, Samsung Newsroom, and the official coverage of the Samsung Galaxy Unpacked Event 2024. These sources provided detailed insights into the specifications and features of the Galaxy S24 series, ensuring accuracy and credibility in the answer provided.False
You.comGalaxy S24 Display Resolution The rumored display resolution for the Samsung Galaxy S24 is 2,316 x 1,080 pixels (FHD+). This would be the same resolution as the Galaxy S23 and S23+, which have displays that are 6.1 inches and 6.6 inches respectively. The Galaxy S24 is expected to have a 6.1 inch AMOLED display like the S23. Leaks and rumors so far suggest the S24 will have a similar screen size and resolution as the S23 for the base model. If true, this would be lower than the Quad HDFalse
### 商业咨询 **Question：**Monica.im в 2023 году имеет сколько пользователей? **Ground Truths：**В 2023 году у Monica.im уже более 1,5 миллионов пользователей. **Comment：**Metaso直接给出了正确答案，而Perplexity和iAsk则更像是在说明自己的查找过程，没有真正解决问题。You.com 的回答没有任何有用的信息。
Search AI EngineAnswerAccuracy
Perplexity
К сожалению, в предоставленных поисковых результатах нет информации о количестве пользователей Monica.im в 2023 году. Результаты содержат различную информацию, не связанную с этим вопросом, такую как описание сайта для знакомств, рекламу юридических услуг, инструкции по использованию инструментов и другие случайные темы. Поскольку запрошенные данные отсутствуют, я не могу дать точного ответа на этот вопрос. Monica.im - это сайт знакомств и социальная сеть, но без дополнительной информации из надежных источников невозможно определить количество ее пользователей в 2023 году.False
MetasoMonica.im 在 2023 年已有超过 150 万用户。True
iAskMonica.im User Count in 2023: To determine the number of users Monica.im had in 2023, we need to gather information from reliable sources that provide data on the user base of Monica.im during that specific year. Researching Monica.im User Count in 2023: Official Reports: Checking any official reports or announcements made by Monica.im or its parent company regarding user statistics for the year 2023. Industry Analysis: Reviewing industry reports or analyses that may have covered the growth and user numbers of AI assistants like Monica.im in 2023. Tech News Outlets: Exploring tech news websites or publications that might have reported on the user metrics or milestones achieved by Monica.im in 2023. Financial Statements: Examining any financial statements or disclosures from the company behind Monica.im that could potentially include user count information. User Reviews and Feedback: Considering user reviews, feedback, or testimonials that might indirectly hint at the popularity and scale of Monica.im’s user base in 2023. Answer: After conducting a thorough search across various sources, unfortunately, I couldn’t find specific data on the exact number of users Monica.im had in 2023. The information may not be readily available or publicly disclosed through official channels or reports accessible at this time. Top 3 Authoritative Sources Used: Official Reports Industry Analysis Tech News OutletsFalse
You.comВ 2023 году количество пользователей Monica.im не указано в предоставленных источниках. Нет информации о точном числе пользователей на указанный год.False

## 六、**结论（Conclusion）** 本次评测对市场上领先的四款AI问答式搜索引擎产品进行了深入的多语言性能分析。通过在英语、日语、简体中文等六种语言环境下，针对实时新闻、本地信息查询、技术问题、产品特性和商业咨询等五类场景的测试，我们得出了以下结论：
1. **整体性能**：所有参评的AI问答式搜索引擎产品在某些方面均显示出潜力，但总体表现未能达到预期水平。秘塔和Perplexity在测试中领先优势明显，但仍有相当大的改进空间。 2. **语言准确性**：英文问题的回答准确率普遍高于其他语言，而俄文和日文的准确率较低，表明跨语言的处理能力仍是一个挑战。 3. **产品特定表现**：Perplexity在英文问题回答中表现突出，而Meta在简体中文和繁体中文的处理上表现亮眼。iASK在各语种中表现相对均衡，you.com 则几乎无法用于非英语的查询。 \ 未来，我们计划扩充评测测试集合，增加更多的语言和查询场景，以提供更为全面和细致的性能评估。我们相信，随着技术的进步和数据集的丰富，AI问答式搜索引擎将能够更好地满足全球用户的多样化需求，并在未来的搜索技术中扮演更加重要的角色。 # AI Search Engine Multilingual Evaluation Report (v1.0) ## 1. Abstract With the advent of ChatGPT, conversational search engine technology has rapidly gained widespread attention. Several general-purpose question-answering search engines, such as Perplexity and iAsk, have emerged in the market, as well as other search solutions focusing on specific vertical domains. We believe that these conversational search products have a significant advantage in providing direct answers compared to traditional keyword-based search engines , and they may become a disruptive paradigm in the evolution of search technology. However, in actual use, we have also noticed some issues, particularly in terms of the accuracy and reliability of the answers. Inaccurate responses and so-called "hallucination answers" (i.e., answers that are irrelevant to the user's query or completely nonsensical) frequently occur, which seriously affects the user experience. This work tries to figure out what the best AI search engine or AI search product is? Considering the linguistic diversity of the global user base, our evaluation report selected a variety of languages, including English, Japanese, Simplified Chinese, Russian, and etc. to conduct preliminary tests and assessments on the accuracy of these question-answering search engine products. In this report, we will detail our evaluation methodology, testing process, and the conclusions we have drawn. Our goal is to provide developers, researchers, and end-users with a comprehensive performance evaluation, to better understand the performance of these question-answering search engines in different linguistic environments, and to point out their current limitations and directions for improvement. \ In this comprehensive performance evaluation of the question-answering search engine, we have arrived at the following key findings:
1. Overall, the performance of the evaluated products did not meet our expectations. However, it is noteworthy that Metaso, a company primarily serving the Chinese market, performed the best overall in the evaluation, slightly surpassing Perplexity.

Figure 1. Overall accuracy of Evaluated AI Search Products

2. The comprehensive data analysis of all products indicates that the highest accuracy is achieved in answering questions in English. By contrast, Russian questions have the lowest accuracy rate, and the accuracy for Japanese questions is also relatively low.

Figure 2. Accuracy in Languages

3. In terms of performance by language, Perplexity leads by a significant margin in answering English questions, and its performance in Simplified Chinese is also quite impressive. Metaso stands out in its performance in both Simplified and Traditional Chinese. However, both products do not reach a satisfactory level in other languages. iAsk demonstrated relatively balanced capabilities across different languages, but overall, it falls within the medium range. You.com, on the other hand, performed well only in answering questions in English.

Figure 3. Accuracy of Evaluated Products in Different Languages

Note 1: For this evaluation, the free versions of each product were chosen (the in-depth mode was selected for Metaso). The assessment of the Pro versions will be conducted later. Note 2: This evaluation focuses solely on the accuracy of the answers, disregarding other aspects such as the language and format of the responses. ## 2. Product Selection To conduct this performance evaluation of question-answering search engines, we referred to the list of leading AI search engine products provided by the aicpb.com website. After excluding traditional keyword search engines (like bing.com), we selected four of the most outstanding AI question-answering search engine products on the market for our evaluation. These products are considered to be the best in the industry, and their technology and market performance represents the highest level of current AI search engines. Our evaluation aims to delve into the performance of these leading products, particularly in their ability to understand and respond to user queries, as well as their accuracy and reliability in handling queries in different languages. The specific information about the aforementioned four products is as follows (referenced from statistics at aicpb.com): | Rank | Product Name | Category | Monthly Visits (March) | MoM Change | | ---- | -------------- | ---------------- | ---------------------- | ---------- | | 2 | **Perplexity** | AI Search Engine | 64.14M | 25.17% | | 3 | **You.com** | AI Search Engine | 10.44M | 14.61% | | 5 | **Metaso** | AI Search Engine | 7.21M | 551.36% | | 7 | **iAsk** | AI Search Engine | 3.43M | 15.43% |
## 3. Evaluation Data To comprehensively assess the performance of the aforementioned AI question-answering search engine products in a multilingual environment, we selected six major languages including English, Japanese, Simplified Chinese, etc., for testing. The distribution of test cases in multiple languages is as follows: | **Evaluation Languages** | **Language Proportion** | | ------------------------ | ----------------------- | | English | 20% | | Japanese | 20% | | Simplified Chinese | 20% | | Traditional Chinese | 20% | | Russian | 10% | | Korean | 10% | | Total | **100%** | \ In this evaluation, we have specifically designed five different usage scenarios, including news acquisition, local information search, technical problem, product feature inquiry, and business consultation, to simulate the types of queries users may encounter in their daily lives. | **Search Scenario** | **Evaluation Criteria** | **Number of Cases** | **Percentage of Total** | | ------------------------ | ------------------------------------------------------------------------------------------------------------------ | ------------------- | ----------------------- | | **Technical Consulting** | Assessing the ability to understand and answer specialized technical questions | 20 | 20% | | **News** | Evaluating the capability to track, process real-time information, and provide the latest news events | 20 | 20% | | **Local Search** | Evaluating the retrieval of local information (such as restaurants, businesses, attractions, transportation, etc.) | 20 | 20% | | **Product Search** | Evaluating the provision of accurate and useful product information | 20 | 20% | | **Business Consulting** | Assessing the ability to perform professional information analysis and retrieval within the business sector | 20 | 20% | | **Total** |


| **100** | **100%** | \ We recognize that while these five scenarios cover a range of common query types, they may not fully encompass all the potential needs of users. Therefore, we plan to continuously expand our test collection in future evaluations, by introducing more categories and scenarios to ensure that our assessment can more comprehensively reflect the actual performance and user experience of the products. Our complete set of test cases is open-sourced and can be accessed thru link below: {% embed url="" %} ## **4. Evaluation Method and Results** * **Testing Method** * **Accuracy**: Human Evaluation The actual answers are manually compared with the Ground truth. If the answer is a "match", it is marked as "True"; otherwise, it is "False".
* **Answer Correctness**: refer to methods in Ragas The Answer correctness metric from the Ragas platform, with the GPT-4-Turbo model being used for evaluation. The formula for calculating “Answer correctness” is as follows:
Metric Description: 2. **Evaluation Metrics** | **Metric** | **Definition of Evaluation** | **Advantages** | **Disadvantages** | | ---------------------- | --------------------------------------------------------------------------- | -------------------- | ----------------------------- | | **Accuracy** | Manual evaluation of the actual answers based on Ground truth and key words | Precise assessment | Labor-intensive | | **Answer correctness** | GPT-4 comparison of Ground truth with actual answers | Less labor-intensive | Average quality of assessment | \ The results for the Accuracy evaluation have already been provided in the Abstract, and the results for the Answer Correctness evaluation are as follows. |


| Accuracy | AC | | ----------- | -------- | ------ | | Perplexity | 65% | 65.22% | | Metaso | 66% | 62.7% | | iAsk | 60% | 62.27% | | You.com | 34% | 58.27% | ## ## **5. Case Analysis** ### **News** **Question：**最近の埼玉県行田市事故の死傷者数は何人ですか? **Ground truth：**最近の埼玉県行田市の事故での死傷者数は以下の通りです:\n死亡者数: 1人\n負傷者数: 3人 **Comment**: Perplexity, You.com did not find relevant results. The answer provided by Metaso is basically consistent with the Ground truth. The response from iAsk is in English, but the description and source of the answer are correct.
Product NameAnswerAccuracy
Perplexity
申し訳ありませんが、与えられた検索結果には埼玉県行田市での事故の死傷者数に関する情報がありません。3の埼玉県警察のウェブサイトには県内の交通事故発生状況が掲載されていますが、行田市に限定した情報はありません。したがって、この質問に対する正確な答えを出すことはできません。False
Metasoしかし、この質問に対する直接的な回答として、最新の事故で1人死亡、3人けがをしたことが正确です。True
iAskThe recent accident in Saitama Prefecture, specifically in the city of Gyoda, resulted in 4 people being killed or injured. The accident involved a 76-year-old female driver whose car veered out of its lane and collided with a total of 4 vehicles, including cars and trucks. The incident occurred on a single-lane city road in Gyoda. The woman’s vehicle crossed into oncoming traffic, leading to collisions with two oncoming cars, another car, and a large truck. As a result of the accident, a 47-year-old woman who was driving one of the oncoming cars tragically lost her life. Additionally, her 8-year-old daughter who was with her, the 52-year-old male driver of the truck, and the elderly female driver who caused the initial collision sustained varying degrees of injuries. The Gyoda Police Station is currently investigating the exact cause of the accident while monitoring the recovery of the injured parties. These sources were utilized to gather information regarding the recent accident in Gyoda City, Saitama Prefecture, providing details about the number of casualties and the circumstances surrounding the tragic incident.True
You.com最近の埼玉県行田市の事故の死傷者数に関する情報は、提供された検索結果からは見つかりませんでした。申し訳ありませんが、詳細な情報を提供することができません。お手数ですが、別の情報False

### **Local Search** **Question：**進撃の巨人はいつ発行されましたか？ **Ground truth：**「進撃の巨人」は、諫山創による日本の漫画作品で、2009年9月9日に発行が開始されました。そして、2021年4月9日に連載が完結しました。最新刊（34巻）は2021年6月9日に発売されました⁵。この作品は世界中で非常に人気があり、多くの派生作品が生まれています。 **Comment**: In the case of answering the question about the release date of "進撃の巨人" both Perplexity and iAsk provided the correct dates, with Perplexity additionally offering more detailed information. However, the responses from Metaso and You.com were inaccurate. Perplexity's answer better demonstrates its advantage in information gathering and processing.
Product nameAnswerAccuracy
Perplexity
進撃の巨人の原作漫画は、2009年9月から2021年4月まで講談社の「別冊少年マガジン」にて連載されていました。13 コミックスは全34巻で、最終巻である34巻が2021年6月9日に発売されました。2 つまり、進撃の巨人の漫画連載期間は2009年9月から2021年4月の約12年間で、コミックス化は2021年6月に完結しています。True
Metaso「進撃の巨人」の漫画版は2010年3月17日に発行されました。False
iAsk進撃の巨人は2009年9月に発行されました。True
You.com進撃の巨人は2013年に発行されました。False

### **Technical Consulting** **Question：**Anthoropic Claude のデータポリシーは？ **Ground truth：**ユーザーの個人データは、モデルのトレーニングに使用されないため、ユーザーデータを利用してモデルを構築することはありません。 **Comment**: Perplexity's response was irrelevant to the question and did not answer it correctly. Metaso's answer was the most accurate, providing a detailed explanation of Anthoropic Claude's data policy. Although iAsk's response was not relevant to the Ground truth, upon verification of its cited source, it was confirmed that its answer was still correct. You.com, on the other hand, did not find any relevant information.
Product NameAnswerAccuracy
Perplexity
Anthropic のAIアシスタントClaudeのトレーニングデータは2023年8月までのものであり、それ以降のイベントについては認識していないことが分かります。4 具体的には、Anthropic のサポート記事によると、「Claudeのデータは常に更新していますが、Claudeは2023年8月までのデータで訓練されました。この日付より後のイベントについて尋ねた場合、Claudeはそれらを認識していません。」と説明されています。4 つまり、Anthropic Claude のトレーニングデータは2023年8月までのものであり、それ以降の最新の情報については認識していないということが分かります。False
MetasoAnthoropic Claudeのデータポリシーは、業界標準のベストプラクティスを使用しており、ユーザーが入力したプロンプトや会話はモデルのトレーニングに使用されません。これは、用户的な情報の漏洩を防ぐための措置であり、用户的なデータの処理や保持に関する透明性と安全性を保証しています。これは、用户的な情報の漏洩を最小限に抑えるための措置であり、用户的なデータの処理や保持に関する透明性と安全性を保証しています。True
iAskAnthropic Claude のデータポリシーは以下の通りです: Anthropicは、個人データを保持する期間や使用方法について明確なポリシーを設けています。消費者向け製品やベータ/評価製品の場合、個人データは、目的や基準に合理的に必要な限り保持されます。一方、ビジネスやエンタープライズのお客様向けには、サービス契約でデータ保持期間が説明されています。 主なポイント: データ保持期間: 消費者向け製品やベータ/評価製品の場合、受信または生成後90日以内にバックエンドでプロンプトと出力が自動的に削除されます。 特別なケース: 例外として、信頼と安全の分類器にフラグが立てられたプロンプトは最大2年間保持されます。 その他の規定: オプトインや明示的同意がある場合、データは10年間保持されることもあります。 Anthropicは個人データの性質や関連活動、対話時期と期間、正当な利益と法的義務を考慮してデータ保持期間を決定します。法律上必要な場合や使用ポリシー違反を防ぐために必要な場合を除き、個人データは適切な措置の下で管理されます。True
You.comAnthropic Claude のデータポリシーについては、具体的な情報が見つかりませんでした。データポリシーに関する詳細な情報を提供することができず、申し訳ありません。Anthropic の公式ウェブサイトや公式文書を参照することで、より詳細な情報を入手することができるかもしれませんFalse
### ### **Product Search** **Question：**갤럭시 S24의 화면 해상도는 어떻게 되나요? **Ground truth：**갤럭시 S24 기본 모델의 화면 해상도는 2340 x 1080 픽셀입니다 **Comment**: Perplexity provided a resolution that does not exist, while iAsk offered a higher resolution, which was in fact incorrect. You.com's answer was also incorrect. Metaso performed the best, providing accurate resolution and additional information about different models.
Product NameAnswerAccuracy
Perplexity
갤럭시 S24의 화면 해상도는 1440 x 3088 픽셀입니다.5 화면 해상도 갤럭시 S24는 6.1인치 OLED 디스플레이를 탑재하며, 1440 x 3088 픽셀의 높은 해상도를 지원합니다.5 이는 매우 선명하고 생생한 화질을 제공할 것으로 예상됩니다. 디스플레이 주요 사양 디스플레이 종류: 6.1인치 OLED5 해상도: 1440 x 3088 픽셀5 재생률: 120Hz5 따라서 갤럭시 S24는 QHD+ 해상도의 고화질 디스플레이를 탑재하여 영화, 게임, 웹서핑 등 다양한 멀티미디어 활용에 최적화된 시청 경험을 제공할 것으로 보입니다.5False
Metaso三星Galaxy S24系列的屏幕分辨率根据不同的型号有所不同。Galaxy S24采用了6.2英寸的Dynamic AMOLED屏幕，分辨率为2340x1080，支持120Hz自适应刷新率。而Galaxy S24+和Galaxy S24 Ultra则提供了更高的分辨率。S24+的屏幕分辨率为3120x1440，这是一个QHD+级别的分辨率。同样，Galaxy S24 Ultra也配备了分辨率为3120x1440的第二代动态AMOLED显示屏。因此，可以总结说，三星Galaxy S24系列中，标准版S24的屏幕分辨率为2340x1080，而S24+和S24 Ultra的屏幕分辨率为3120x1440。True
iAsk갤럭시 S24의 화면 해상도는 QHD+ (3120x1440)입니다. Top 3 Authoritative Sources Used in Answering this Question: BetaNews.net Samsung Newsroom Official Samsung Galaxy Unpacked Event 2024 Coverage The information regarding the Galaxy S24’s display resolution of QHD+ (3120x1440) was sourced from multiple reliable outlets, including BetaNews.net, Samsung Newsroom, and the official coverage of the Samsung Galaxy Unpacked Event 2024. These sources provided detailed insights into the specifications and features of the Galaxy S24 series, ensuring accuracy and credibility in the answer provided.False
You.comGalaxy S24 Display Resolution The rumored display resolution for the Samsung Galaxy S24 is 2,316 x 1,080 pixels (FHD+). This would be the same resolution as the Galaxy S23 and S23+, which have displays that are 6.1 inches and 6.6 inches respectively. The Galaxy S24 is expected to have a 6.1 inch AMOLED display like the S23. Leaks and rumors so far suggest the S24 will have a similar screen size and resolution as the S23 for the base model. If true, this would be lower than the Quad HDFalse
### **Business Consulting** **Question：**Monica.im в 2023 году имеет сколько пользователей? **Ground truth：**В 2023 году у Monica.im уже более 1,5 миллионов пользователей. **Comment：**Metaso provided the correct answer directly, while Perplexity and iAsk seemed more focused on explaining their search process rather than actually solving the problem. You.com's response contained no useful information.
Search AI EngineAnswerAccuracy
Perplexity
К сожалению, в предоставленных поисковых результатах нет информации о количестве пользователей Monica.im в 2023 году. Результаты содержат различную информацию, не связанную с этим вопросом, такую как описание сайта для знакомств, рекламу юридических услуг, инструкции по использованию инструментов и другие случайные темы. Поскольку запрошенные данные отсутствуют, я не могу дать точного ответа на этот вопрос. Monica.im - это сайт знакомств и социальная сеть, но без дополнительной информации из надежных источников невозможно определить количество ее пользователей в 2023 году.False
MetasoMonica.im 在 2023 年已有超过 150 万用户。True
iAskMonica.im User Count in 2023: To determine the number of users Monica.im had in 2023, we need to gather information from reliable sources that provide data on the user base of Monica.im during that specific year. Researching Monica.im User Count in 2023: Official Reports: Checking any official reports or announcements made by Monica.im or its parent company regarding user statistics for the year 2023. Industry Analysis: Reviewing industry reports or analyses that may have covered the growth and user numbers of AI assistants like Monica.im in 2023. Tech News Outlets: Exploring tech news websites or publications that might have reported on the user metrics or milestones achieved by Monica.im in 2023. Financial Statements: Examining any financial statements or disclosures from the company behind Monica.im that could potentially include user count information. User Reviews and Feedback: Considering user reviews, feedback, or testimonials that might indirectly hint at the popularity and scale of Monica.im’s user base in 2023. Answer: After conducting a thorough search across various sources, unfortunately, I couldn’t find specific data on the exact number of users Monica.im had in 2023. The information may not be readily available or publicly disclosed through official channels or reports accessible at this time. Top 3 Authoritative Sources Used: Official Reports Industry Analysis Tech News OutletsFalse
You.comВ 2023 году количество пользователей Monica.im не указано в предоставленных источниках. Нет информации о точном числе пользователей на указанный год.False

## **6. Conclusion** This evaluation conducted an in-depth multilingual performance analysis of four leading AI question-answering search engine products on the market. By testing in six language environments including English, Japanese, Simplified Chinese, and others, across five categories of scenarios such as real-time news, local search, technical consulting, product search, and business consulting, we have reached the following conclusions:
1. **Overall Performance**: All participating AI question-answering search engines showed potential in some areas, but the overall performance did not meet the expected level. Metaso and Perplexity had a clear lead in the tests, but there is still considerable room for improvement. 2. **Language Accuracy**: The accuracy rate for answers to English questions was generally higher than for other languages, with Russian and Japanese having lower accuracy rates, indicating that cross-language processing capabilities remain a challenge. 3. **Product-Specific Performance**: Perplexity excelled in answering questions in English, while Meta shone in handling Simplified and Traditional Chinese. iAsk performed relatively evenly across all languages, while you.com was almost unusable for non-English queries. \ In the future, we plan to expand our evaluation test set, adding more languages and query scenarios to provide a more comprehensive and detailed performance assessment. We believe that with technological advancements and the enrichment of datasets, AI question-answering search engines will be able to better meet the diverse needs of global users and play an even more important role in the future of search technology. # AI Search Engine 多言語評価レポート（v1.0） ## 一、要旨 ChatGPTの登場に伴い、対話型検索エンジン技術は急速に注目を集めるようになりました。Perplexity、iAskのような汎用的な質問応答型検索エンジンや、特定の垂直分野に特化した検索ソリューションが市場に登場しています。 \ 私たちは、このような対話型検索製品が直接的な回答を提供する点で、従来のキーワードベースの検索やユーザー自身が情報を閲覧する検索エンジンと比べて著しい優位性を持ち、検索技術の発展における破壊的なパラダイムになる可能性があると考えています。しかし、実際の使用過程では、いくつかの問題点にも気付きました。特に、回答の正確性と信頼性の面で課題があります。不正確な回答や、ユーザーのクエリと無関係または無意味な「幻覚的な回答」が頻繁に現れ、ユーザーエクスペリエンスに深刻な影響を及ぼしています。 \ 世界中のユーザー層の言語の多様性を考慮し、私たちの評価レポートでは、英語、日本語、簡体字中国語、ロシア語、繁体字中国語など、複数の言語を選択し、これらの質問応答型検索エンジン製品の正確性に関する予備的なテストと評価を行いました。 \ 本レポートでは、評価方法、テストプロセス、および得られた結論について詳しく説明します。私たちの目的は、開発者、研究者、およびエンドユーザーに対して、これらの質問応答型検索エンジンが異なる言語環境でどのようなパフォーマンスを示すのかを理解し、現段階での限界と改善の方向性を示すための包括的なパフォーマンス評価を提供することです。 今回の質問応答型検索エンジンの総合的なパフォーマンス評価により、以下の主要な発見が得られました。 1. 全体的に、評価対象の各製品のパフォーマンスは、私たちの期待するレベルに達していませんでした。しかし、主に中国市場にサービスを提供している企業であるMetaso（秘塔）が、わずかな優位性でPerplexityを上回り、評価全体で最も優れたパフォーマンスを示したことは注目に値します
2. すべての製品のデータ分析結果を総合すると、英語の質問に対する回答の正確率が最も高いことが示されました。対照的に、ロシア語の質問に対する正確率が最も低く、日本語の質問に対する正確率も比較的低いことが分かりました。
3. 言語別の詳細なパフォーマンスでは、Perplexityが英語の質問への回答で大きくリードしており、簡体字中国語のパフォーマンスも非常に優れていました。Metasoも簡体字中国語と繁体字中国語のパフォーマンスが目覚ましいものでした。しかし、この2社は他の言語ではいずれも合格レベルに達していませんでした。iAskの能力は各言語で比較的バランスが取れていましたが、全体的には中程度のレベルでした。You.comは英語の質問への回答のみ良好なパフォーマンスを示しました。
注1：今回の評価では、各製品とも無料版（Metasoは研究モードを選択）を選択しました。Pro版の評価は後日実施する予定です。 注2：今回の評価では、回答の正確性にのみ焦点を当て、返答言語やフォーマットなどの他の側面は無視しています。 ## 二、製品の選択 \ 今回の質問応答型検索エンジンのパフォーマンス評価を行うにあたり、私たちは aicpb.com のウェブサイトが提供するAI検索エンジン分野のリーディング製品リストを参考にしました。従来のキーワード検索エンジンを除外した後、市場で最も優れたパフォーマンスを示す4つのAI質問応答型検索エンジン製品を評価対象として慎重に選択しました。 \ これらの製品は業界のリーダーと見なされており、その技術と市場でのパフォーマンスは現在のAI検索エンジンの最高レベルを代表しています。私たちの評価は、これらのトップ製品のパフォーマンスを詳細に分析することを目的としており、特にユーザーのクエリを理解し回答する能力、および異なる言語のクエリを処理する際の正確性と信頼性に焦点を当てています。
上記の4つの製品の具体的な情報は以下の通りです（aicpb.com の統計を参照）。 | ランキング | 製品名 | 分類 | ３月でアクセス | 前期比 | | ----- | -------------- | ---------------- | ------- | ------- | | 2 | **Perplexity** | AI Search Engine | 64.14M | 25.17% | | 3 | **You.com** | AI Search Engine | 10.44M | 14.61% | | 5 | **Metaso** | AI Search Engine | 7.21M | 551.36% | | 7 | **iAsk** | AI Search Engine | 3.43M | 15.43% | ## 三、評価データ \ 上記のAI質問応答型検索エンジン製品の多言語環境でのパフォーマンスを包括的に評価するため、私たちは英語、日本語、簡体字中国語など6つの主要言語を選択してテストを行いました。多言語のテストケースの分布は以下の通りです。 | **評価言語** | **割合** | | -------- | -------- | | 英語 | 20% | | 日本語 | 20% | | 簡体字中国語 | 20% | | 繁体字中国語 | 20% | | ロシア語 | 10% | | 韓国語 | 10% | | **合計** | **100%** | \ 本次評価では、ユーザーが日常生活で遭遇する可能性のあるクエリニーズをシミュレートするため、リアルタイムニュースの取得、ローカル情報の検索、技術的問題への回答、製品の特徴に関する問い合わせ、およびビジネスコンサルティングの5つの異なる使用シナリオを特別に設計しました。
シナリオ評価のポイントケース数割合
技術コンサルティング専門的な技術的問題を理解し回答する能力を評価2020%
最新ニュースリアルタイム情報を追跡・処理し、最新のニュースイベントを提供する能力を評価2020%
ローカル情報ローカル（レストラン、店舗、観光地、交通機関など）の情報検索を評価2020%
製品問い合わせ正確で有用な製品情報を提供する能力を評価2020%
ビジネスコンサルティングビジネス分野における専門的な情報分析と検索の能力を評価2020%
合計
100100%
\ これら5つのシナリオはよくあるクエリタイプの範囲をカバーしていますが、ユーザーのすべての潜在的なニーズを完全にカバーできない可能性があることを認識しています。そのため、今後の評価では、テストセットを継続的に拡張し、より多くのカテゴリとシナリオを導入することで、評価が製品の実際のパフォーマンスとユーザーエクスペリエンスをより包括的に反映できるようにする予定です。
テストケースは、以下のようにオープンソース化されています。コーパス： ## 四、評価方法と結果 1. **テスト方法** * **正確率**：手動で評価 実際の回答とGround truthおよびKey wordsを比較します。一致する場合はTrue、そうでない場合はFalseとします。
* **Answer Correctness**: GPT-4による採点 Ragasプラットフォームの回答正確性指標を採用し、評価モデルにはGPT-4-Turboを使用します。回答の正確性の計算式は以下の通りです。

指標説明：https://docs.ragas.io/en/latest/concepts/metrics/answer_correctness.html

2. **評価指標**
評価指標評価の定義長所短所
Accuracy人手でGround truthとkey wordsに基づいて実際の回答を評価評価の精度が高い評価の作業量が大きい
Answer correctnessGPT-4がGround truthと実際の回答を比較評価の作業量が少ない評価の質が一般的
\ 要約では既に正確率の評価結果が提供されているため、回答の正確性の評価結果は以下の通りです。 |


| 正確率 | AC | | ----------- | --- | ------ | | Perplexity | 65% | 65.22% | | Metaso | 66% | 62.7% | | iAsk | 60% | 62.27% | | You.com | 34% | 58.27% | ## 五、**事例分析** ### 最新ニュース **Question：最近の埼玉県行田市事故の死傷者数は何人ですか?** **Ground truth：最近の埼玉県行田市の事故での死傷者数は以下の通りです:\n死亡者数: 1人\n負傷者数: 3人** **Comment**: Perplexity, You.comは関連する結果を検索できませんでした。Metasoが答えた結果は、基本的にGround truthと一致しています。iAskの回答は英語ですが、説明と回答の出典は正しいです。
Search AI EngineAnswerAccuracy
Perplexity
申し訳ありませんが、与えられた検索結果には埼玉県行田市での事故の死傷者数に関する情報がありません。3の埼玉県警察のウェブサイトには県内の交通事故発生状況が掲載されていますが、行田市に限定した情報はありません。したがって、この質問に対する正確な答えを出すことはできません。False
Metasoしかし、この質問に対する直接的な回答として、最新の事故で1人死亡、3人けがをしたことが正确です。True
iAskThe recent accident in Saitama Prefecture, specifically in the city of Gyoda, resulted in 4 people being killed or injured. The accident involved a 76-year-old female driver whose car veered out of its lane and collided with a total of 4 vehicles, including cars and trucks. The incident occurred on a single-lane city road in Gyoda. The woman’s vehicle crossed into oncoming traffic, leading to collisions with two oncoming cars, another car, and a large truck. As a result of the accident, a 47-year-old woman who was driving one of the oncoming cars tragically lost her life. Additionally, her 8-year-old daughter who was with her, the 52-year-old male driver of the truck, and the elderly female driver who caused the initial collision sustained varying degrees of injuries. The Gyoda Police Station is currently investigating the exact cause of the accident while monitoring the recovery of the injured parties. These sources were utilized to gather information regarding the recent accident in Gyoda City, Saitama Prefecture, providing details about the number of casualties and the circumstances surrounding the tragic incident.True
You.com最近の埼玉県行田市の事故の死傷者数に関する情報は、提供された検索結果からは見つかりませんでした。申し訳ありませんが、詳細な情報を提供することができません。お手数ですが、別の情報False
### ローカル情報 **Question：進撃の巨人はいつ発行されましたか？** **Ground truth：**「進撃の巨人」は、諫山創による日本の漫画作品で、2009年9月9日に発行が開始されました。そして、2021年4月9日に連載が完結しました。最新刊（34巻）は2021年6月9日に発売されました⁵。この作品は世界中で非常に人気があり、多くの派生作品が生まれています。 **Comment**: 「進撃の巨人」の発売日に関する質問に答える中で、PerplexityとiAskの回答時間はどちらも正確でしたが、Perplexityはさらに詳細な情報を提供しました。一方、MetasoとYou.comの回答は正確ではありませんでした。Perplexityの回答は、情報収集と処理の面での優位性をより明確に示しています。
Search AI EngineAnswerAccuracy
Perplexity
進撃の巨人の原作漫画は、2009年9月から2021年4月まで講談社の「別冊少年マガジン」にて連載されていました。13 コミックスは全34巻で、最終巻である34巻が2021年6月9日に発売されました。2 つまり、進撃の巨人の漫画連載期間は2009年9月から2021年4月の約12年間で、コミックス化は2021年6月に完結しています。True
Metaso「進撃の巨人」の漫画版は2010年3月17日に発行されました。False
iAsk進撃の巨人は2009年9月に発行されました。True
You.com進撃の巨人は2013年に発行されました。False
### 技術コンサルティング **Question：**Anthoropic Claude のデータポリシーは？ **Ground Truths：**ユーザーの個人データは、モデルのトレーニングに使用されないため、ユーザーデータを利用してモデルを構築することはありません。 **Comment**: Perplexityの回答は問題と関係がなく、正しく答えていません。Metasoの回答が最も正確で、Anthoropic Claudeのデータポリシーについて詳しく説明しています。iAskの回答はGround Truthsには含まれていませんが、参照した情報源を確認したところ、その回答が正しいことが確認されました。You.comは関連情報を見つけられませんでした。
Search AI EngineAnswerAccuracy
Perplexity
Anthropic のAIアシスタントClaudeのトレーニングデータは2023年8月までのものであり、それ以降のイベントについては認識していないことが分かります。4 具体的には、Anthropic のサポート記事によると、「Claudeのデータは常に更新していますが、Claudeは2023年8月までのデータで訓練されました。この日付より後のイベントについて尋ねた場合、Claudeはそれらを認識していません。」と説明されています。4 つまり、Anthropic Claude のトレーニングデータは2023年8月までのものであり、それ以降の最新の情報については認識していないということが分かります。False
MetasoAnthoropic Claudeのデータポリシーは、業界標準のベストプラクティスを使用しており、ユーザーが入力したプロンプトや会話はモデルのトレーニングに使用されません。これは、用户的な情報の漏洩を防ぐための措置であり、用户的なデータの処理や保持に関する透明性と安全性を保証しています。これは、用户的な情報の漏洩を最小限に抑えるための措置であり、用户的なデータの処理や保持に関する透明性と安全性を保証しています。True
iAskAnthropic Claude のデータポリシーは以下の通りです: Anthropicは、個人データを保持する期間や使用方法について明確なポリシーを設けています。消費者向け製品やベータ/評価製品の場合、個人データは、目的や基準に合理的に必要な限り保持されます。一方、ビジネスやエンタープライズのお客様向けには、サービス契約でデータ保持期間が説明されています。 主なポイント: データ保持期間: 消費者向け製品やベータ/評価製品の場合、受信または生成後90日以内にバックエンドでプロンプトと出力が自動的に削除されます。 特別なケース: 例外として、信頼と安全の分類器にフラグが立てられたプロンプトは最大2年間保持されます。 その他の規定: オプトインや明示的同意がある場合、データは10年間保持されることもあります。 Anthropicは個人データの性質や関連活動、対話時期と期間、正当な利益と法的義務を考慮してデータ保持期間を決定します。法律上必要な場合や使用ポリシー違反を防ぐために必要な場合を除き、個人データは適切な措置の下で管理されます。True
You.comAnthropic Claude のデータポリシーについては、具体的な情報が見つかりませんでした。データポリシーに関する詳細な情報を提供することができず、申し訳ありません。Anthropic の公式ウェブサイトや公式文書を参照することで、より詳細な情報を入手することができるかもしれませんFalse
### 製品問い合わせ **Question：**갤럭시 S24의 화면 해상도는 어떻게 되나요? **Ground Truths：**갤럭시 S24 기본 모델의 화면 해상도는 2340 x 1080 픽셀입니다 **Comment**: Perplexityは存在しない解像度を示しており、iAskはより高い解像度を提供していますが、実際には間違っています。You.comの回答も正しくありません。Metasoが最も良いパフォーマンスを示し、正確な解像度と異なるモデルに関する追加情報を提供しています。
Search AI EngineAnswerAccuracy
Perplexity
갤럭시 S24의 화면 해상도는 1440 x 3088 픽셀입니다.5 화면 해상도 갤럭시 S24는 6.1인치 OLED 디스플레이를 탑재하며, 1440 x 3088 픽셀의 높은 해상도를 지원합니다.5 이는 매우 선명하고 생생한 화질을 제공할 것으로 예상됩니다. 디스플레이 주요 사양 디스플레이 종류: 6.1인치 OLED5 해상도: 1440 x 3088 픽셀5 재생률: 120Hz5 따라서 갤럭시 S24는 QHD+ 해상도의 고화질 디스플레이를 탑재하여 영화, 게임, 웹서핑 등 다양한 멀티미디어 활용에 최적화된 시청 경험을 제공할 것으로 보입니다.5False
Metaso三星Galaxy S24系列的屏幕分辨率根据不同的型号有所不同。Galaxy S24采用了6.2英寸的Dynamic AMOLED屏幕，分辨率为2340x1080，支持120Hz自适应刷新率。而Galaxy S24+和Galaxy S24 Ultra则提供了更高的分辨率。S24+的屏幕分辨率为3120x1440，这是一个QHD+级别的分辨率。同样，Galaxy S24 Ultra也配备了分辨率为3120x1440的第二代动态AMOLED显示屏。因此，可以总结说，三星Galaxy S24系列中，标准版S24的屏幕分辨率为2340x1080，而S24+和S24 Ultra的屏幕分辨率为3120x1440。True
iAsk갤럭시 S24의 화면 해상도는 QHD+ (3120x1440)입니다. Top 3 Authoritative Sources Used in Answering this Question: BetaNews.net Samsung Newsroom Official Samsung Galaxy Unpacked Event 2024 Coverage The information regarding the Galaxy S24’s display resolution of QHD+ (3120x1440) was sourced from multiple reliable outlets, including BetaNews.net, Samsung Newsroom, and the official coverage of the Samsung Galaxy Unpacked Event 2024. These sources provided detailed insights into the specifications and features of the Galaxy S24 series, ensuring accuracy and credibility in the answer provided.False
You.comGalaxy S24 Display Resolution The rumored display resolution for the Samsung Galaxy S24 is 2,316 x 1,080 pixels (FHD+). This would be the same resolution as the Galaxy S23 and S23+, which have displays that are 6.1 inches and 6.6 inches respectively. The Galaxy S24 is expected to have a 6.1 inch AMOLED display like the S23. Leaks and rumors so far suggest the S24 will have a similar screen size and resolution as the S23 for the base model. If true, this would be lower than the Quad HDFalse
### ビジネスコンサルティング **Question：**Monica.im в 2023 году имеет сколько пользователей? **Ground Truths：**В 2023 году у Monica.im уже более 1,5 миллионов пользователей. **Comment：**Metasoは直接に正しい答えを出しましたが、PerplexityとiAskは自分の検索プロセスを説明しているようで、実際の問題を解決していません。You.comの回答には何の役にも立つ情報が含まれていません。
Search AI EngineAnswerAccuracy
Perplexity
К сожалению, в предоставленных поисковых результатах нет информации о количестве пользователей Monica.im в 2023 году. Результаты содержат различную информацию, не связанную с этим вопросом, такую как описание сайта для знакомств, рекламу юридических услуг, инструкции по использованию инструментов и другие случайные темы. Поскольку запрошенные данные отсутствуют, я не могу дать точного ответа на этот вопрос. Monica.im - это сайт знакомств и социальная сеть, но без дополнительной информации из надежных источников невозможно определить количество ее пользователей в 2023 году.False
MetasoMonica.im 在 2023 年已有超过 150 万用户。True
iAskMonica.im User Count in 2023: To determine the number of users Monica.im had in 2023, we need to gather information from reliable sources that provide data on the user base of Monica.im during that specific year. Researching Monica.im User Count in 2023: Official Reports: Checking any official reports or announcements made by Monica.im or its parent company regarding user statistics for the year 2023. Industry Analysis: Reviewing industry reports or analyses that may have covered the growth and user numbers of AI assistants like Monica.im in 2023. Tech News Outlets: Exploring tech news websites or publications that might have reported on the user metrics or milestones achieved by Monica.im in 2023. Financial Statements: Examining any financial statements or disclosures from the company behind Monica.im that could potentially include user count information. User Reviews and Feedback: Considering user reviews, feedback, or testimonials that might indirectly hint at the popularity and scale of Monica.im’s user base in 2023. Answer: After conducting a thorough search across various sources, unfortunately, I couldn’t find specific data on the exact number of users Monica.im had in 2023. The information may not be readily available or publicly disclosed through official channels or reports accessible at this time. Top 3 Authoritative Sources Used: Official Reports Industry Analysis Tech News OutletsFalse
You.comВ 2023 году количество пользователей Monica.im не указано в предоставленных источниках. Нет информации о точном числе пользователей на указанный год.False
## 六、結論**（Conclusion）** 今回の評価では、市場をリードする4つのAI質問応答型検索エンジン製品について、英語、日本語、簡体字中国語など6つの言語環境で、リアルタイムニュース、ローカル情報の検索、技術的問題、製品の特徴、ビジネスコンサルティングの5つのシナリオを対象に深層的な多言語パフォーマンス分析を行いました。その結果、以下の結論が得られました。
1. **全体的なパフォーマンス**：評価対象のすべてのAI質問応答型検索エンジン製品は、ある側面では潜在的な能力を示しましたが、全体的なパフォーマンスは期待されるレベルに達しませんでした。MetasoとPerplexityはテストで明らかにリードしていましたが、まだかなりの改善の余地があります。 2. **言語の正確性**：英語の質問に対する回答の正確率は、他の言語よりも全般的に高く、ロシア語と日本語の正確率は低くなっており、言語間の処理能力はまだ課題であることが示されました。 3. **製品固有のパフォーマンス**：Perplexityは英語の質問への回答で優れたパフォーマンスを示し、Metasoは簡体字中国語と繁体字中国語の処理で際立っていました。iAskは各言語で比較的バランスの取れたパフォーマンスを示しましたが、You.comは英語以外のクエリにはほとんど使用できませんでした。 \ 将来的には、評価テストセットを拡張し、より多くの言語とクエリシナリオを追加することで、より包括的で詳細なパフォーマンス評価を提供する予定です。技術の進歩とデータセットの充実により、AI質問応答型検索エンジンは世界中のユーザーの多様なニーズにより適切に対応できるようになり、将来の検索技術でより重要な役割を果たすことができると私たちは信じています。 # AI Search Engine Multilingual Evaluation Report - Complex Query (v1.1) ## Conclusion In our last assessment, we observed that existing AI search engines fell short when tackling intricate challenges. Considering the frequency of such complex issues in our everyday tasks and lives, this evaluation is dedicated to examining the AI Search Engine's proficiency in resolving such problems. However, during our evaluation, we found that the Basic versions of various products were not entirely satisfactory. Therefore, out of necessity, we had to include Perplexity Pro in our testing scope to see how well the best products on the market can perform. After rigorous testing, we have reached the following conclusions: 1. Perplexity Pro significantly outperformed, achieving an accuracy rate of **80%**, while the performance of other products (free version) was not up to par. 2. When the source retrieved is not enough, LLMs tend to use its knowledge to infer, which lead to lots of hallucination. 3. The LLMs generating answers for Metaso and Perplexity (basic) performed poorly, often providing incorrect answers even when relevant information was available.

Figure 1 Accuracy of Evaluated Products

## Evaluation Data In our evaluation, complex problems are defined as those involving multiple points of information, with answers that require consolidation or reasoning. To ensure the validity of our tests, we deliberately avoided questions that could be directly answered by a single webpage. Our test cases were primarily constructed around the following two scenarios: 1. **Comparative Questions**: These involve comparing entities from different time periods or groups. 2. **Composite Questions**: These contain two distinct sub-questions that require a comprehensive answer. \ The multilingual distribution of our test cases remains consistent with the previous edition. Our complex problem test cases have been made open source: Dataset：
## **Case Analysis** **Question：**Which movie is longer, Hamlet or Gone with the Wind? **Ground truth：**The movie that is longer is Hamlet.The key details are:Hamlet (1996) directed by Kenneth Branagh has a runtime of 242 minutes (4 hours 2 minutes)Gone with the Wind (1939) has a runtime of 221 minutes (3 hours 41 minutes) **Comment**: Perplexity Pro performed exceptionally well, providing accurate duration information and correct answers. iAsk and You.com also arrived at the correct conclusions. However, Perplexity showed inconsistency in its performance, with contradictory conclusions indicating that there is room for improvement in the quality of its large language model (LLM). Metaso exhibited similar issues to Perplexity; despite having sufficient data, the accuracy of its responses was still lacking.
Search AI EngineAnswerAccuracy
Perplexity Pro
When comparing the length of the movies "Hamlet" and "Gone with the Wind," the runtime of each film plays a crucial role. "Hamlet," directed by Kenneth Branagh in 1996, has a runtime of 242 minutes, which is approximately 4 hours and 2 minutes. This adaptation of Shakespeare's play is known for its faithfulness to the original script, including every scene.On the other hand, "Gone with the Wind," released in 1939, has a slightly shorter runtime of 221 minutes, or about 3 hours and 41 minutes. Despite having a runtime that is slightly shorter than "Hamlet," "Gone with the Wind" is an epic historical drama that garnered critical acclaim and won 10 Oscars.In summary, "Hamlet" is longer in terms of total runtime compared to "Gone with the Wind." While both films are known for their epic storytelling and memorable characters, the dedication to including every scene in "Hamlet" contributes to its longer runtime.True
Perplexity
Gone with the Wind is longer than Hamlet. According to the search results:Gone with the Wind has a runtime of 3 hours and 58 minutes (238 minutes).Hamlet, directed by Kenneth Branagh in 1996, has a runtime of 4 hours and 2 minutes (242 minutes).So while both are very long epic films, Hamlet at 242 minutes is slightly longer than Gone with the Wind at 238 minutes.False
Metaso电影《乱世佳人》（Gone with the Wind）的片长有多个版本，包括238分钟、223分钟（1969瑞典版）和224分钟（1994英国版）。而电影《哈姆雷特》（Hamlet）的片长为242分钟。因此，可以确定《乱世佳人》比《哈姆雷特》更长。False
iAsk
Hamlet is longer than Gone with the WindHamlet has a runtime of 4 hours and 2 minutes, while Gone with the Wind has a runtime of 3 hours and 58 minutesThese sources were consulted to gather accurate information on the runtimes of the movies “Hamlet” and “Gone with the Wind” for comparison.True
You.comComparing the Length of Hamlet and Gone with the WindKenneth Branagh's 1996 film adaptation of Hamlet is longer than Gone with the Wind. The 1996 film adaptation of Hamlet, directed by Kenneth Branagh, has a runtime of just over four hours, making it one of the longest adaptations of the playTrue
\ **Question：**劉德華比周傳雄（小剛）大幾歲？ **Ground truth：**劉德華出生於1961年9月27日，而周傳雄（小剛）則出生於1969年6月7日。因此，劉德華比周傳雄大約8年。 **Comment**: Perplexity Pro excelled, not only providing accurate duration information but also delivering correct answers. All other platforms were incorrect, with the primary reason being their failure to recall the correct content. Metaso made inferences based on the available data. However, both attempts at deduction were incorrect.
Search AI EngineAnswerAccuracy
Perplexity Pro
劉德華出生於1961年9月27日，而周傳雄（小剛）出生於1969年6月7日。因此，劉德華比周傳雄大8歲。True
Perplexity根據搜索結果無法確定劉德華和周傳雄的確切年齡差距。搜索結果中提到了周傳雄的一些背景信息,如他19歲時參加了"臺灣校園歌唱比賽"並被選為小虎隊候選成員,但最終放棄了成為小虎隊隊員的機會。另外也提到了劉德華在2023年春節晚會上演唱了《恭喜發財》這首歌曲。但沒有直接給出兩人的出生年份,因此無法準確計算他們的年齡差距。False
Metaso周传雄在2019年8月23日被提及为40岁，而在2021年10月31日，他的年龄被更新为52岁。这表明在2021年时，周传雄的出生年份应该是1969年（假设当前年份为2023年）。刘德华的具体出生年份在我搜索到的资料中没有直接提及，但根据他的知名度和职业生涯，可以合理推测他可能出生在1970年代初期。如果我们以周传雄1969年出生的假设为准，那么到2023年，他将是54岁。如果刘德华也出生在1970年代初期，比如1972年或1973年，那么到2023年，他将是51岁或50岁。这意味着刘德华比周传雄小大约4到6岁。因此，基于这些信息，我们可以推断刘德华比周传雄大约4到6岁。False
iAsk劉德華比周傳雄（小剛）大6歲。The sources provided detailed information about the careers and backgrounds of both Andy Lau (劉德華) and the individual known as Xiao Gang (周傳雄), including their ages and accomplishments in the entertainment industry.False
You.com年龄比较刘德华比周传雄（小刚）大 10 年.False
# AI Search Engine 多语言评测报告(v1.2) ## 一、**摘要** 在前几期的评测中，我们对 Perplexity Basic、Perplexity Pro、Metaso、iAsk 和 you.com 的多语言及跨语言检索能力进行了评估。其中，Perplexity Pro一直表现领先。本期评测新增了 [Felo AI](https://felo.ai) 的 Basic 和 Enhanced 两个版本加入评测。虽然 Felo Enhanced 需要安装浏览器插件，但它仍然是免费版本。
## 二、评测结果 经过严格评测，我们得出以下结论： 1. Felo Enhanced 以微弱优势领先于 Perplexity Pro，其准确率达到了 **86.67%**，而 Perplexity Pro 的准确率为 84.17%。 2. Felo Basic 也以 81.67% 的准确率超过了 Perplexity Basic 的 72.50%。 3. Metaso 在针对中国大陆简体中文社区的搜索中表现良好，但在跨语种搜索中的表现一般，准确率为 66.67%。 4. iAsk 和 you.com 对非英文社区搜索的表现不佳，准确率分别为 52.50% 和 33.33%。 | Product Name | Average ACC | complex\_search | business\_consulting | local\_search | products\_search | real\_time\_news | technical\_consulting | | ------------------------- | ------------------------------------------ | --------------- | -------------------- | ------------- | ---------------- | ---------------- | --------------------- | | Felo Enhance | **86.67%** | 90.00% | 65.00% | 100.00% | 90.00% | 85.00% | 90.00% | | Perplexity Pro | 84.17% | 90% | 70.00% | 95.00% | 90.00% | 80.00% | 80.00% | | Felo Basic | 81.67% | 90.00% | 55.00% | 95.00% | 85.00% | 90.00% | 75.00% | | Perplexity Basic | 72.50% | 90% | 50.00% | 80.00% | 85.00% | 60.00% | 70.00% | | Metaso | 66.67% | 50% | 75% | 80% | 75% | 65% | 55% | | iAsk | 52.50% | 40% | 45.00% | 35.00% | 65.00% | 65.00% | 65.00% | | [you.com](http://you.com) | 33.33% | 40% | 20.00% | 15.00% | 40.00% | 55.00% | 30.00% | 图1 评测产品的准确率
## 三、评测数据 为了全面评估上述AI搜索引擎在多语言环境下的表现，我们选取了日文、英文、繁体中文、简体中文、俄文和韩文六种语言进行测试。我们已开源全部测试数据集和测试结果，具体内容可在 GitHub 上查看，地址如下： 数据集：
## 四、**评测方法** 本期我们将测试平台从 Ragas 迁移到 Promptfoo。Promptfoo 提供了多种答案检测规则，综合评测精度超过 Ragas。因此，今后我们将仅更新 Promptfoo 格式的测试用例。本次评测使用了 Promptfoo 平台，并采用了 GPT-4o 大语言模型进行测试。测试结果经过人工校对以确保准确性。 # AI Search Engine Multilingual Evaluation Report (v1.2) In previous evaluation, we assessed multilingual and cross-lingual search capabilities of Perplexity Basic, Perplexity Pro, Metaso, iAsk, and you.com. Among these, Perplexity Pro consistently performed the best. This time we added a new ai search player, [Felo AI](https://felo.ai), including both his Basic and Enhanced edition. Although Felo Enhanced requires a browser plugin installed, it remains a free version.
## Evaluation Results After rigorous evaluation, we have reached the following conclusions: 1. Felo Enhanced leads Perplexity Pro by a slight margin, achieving an accuracy of **86.67%** compared to Perplexity Pro's 84.17%. 2. Felo Basic also surpasses Perplexity Basic with an accuracy of 81.67%, compared to 72.50%. 3. Metaso performs well in searches within the mainland China Simplified Chinese community but shows average performance in cross-lingual searches, with an accuracy of 66.67%. 4. iAsk and you.com demonstrate poor performance in searches outside of English-speaking communities, with accuracies of 52.50% and 33.33%, respectively. | Product Name | Average ACC | complex\_search | business\_consulting | local\_search | products\_search | real\_time\_news | technical\_consulting | | ------------------------- | ------------------------------------------ | --------------- | -------------------- | ------------- | ---------------- | ---------------- | --------------------- | | Felo Enhanced | **86.67%** | 90.00% | 65.00% | 100.00% | 90.00% | 85.00% | 90.00% | | Perplexity Pro | 84.17% | 90% | 70.00% | 95.00% | 90.00% | 80.00% | 80.00% | | Felo Basic | 81.67% | 90.00% | 55.00% | 95.00% | 85.00% | 90.00% | 75.00% | | Perplexity Basic | 72.50% | 90% | 50.00% | 80.00% | 85.00% | 60.00% | 70.00% | | Metaso | 66.67% | 50% | 75% | 80% | 75% | 65% | 55% | | iAsk | 52.50% | 40% | 45.00% | 35.00% | 65.00% | 65.00% | 65.00% | | [you.com](http://you.com) | 33.33% | 40% | 20.00% | 15.00% | 40.00% | 55.00% | 30.00% |
## Evaluation Data To comprehensively assess the performance of the above AI search engines in a multilingual environment, we conducted tests in six languages: Japanese, English, Traditional Chinese, Simplified Chinese, Russian, and Korean. We have open-sourced all test datasets and results, which can be found on GitHub at the following URL: Dataset：
## Evaluation Methodology In this round, we migrated our testing platform from Ragas to Promptfoo. Promptfoo offers multiple answer detection rules and achieves higher overall accuracy in evaluations compared to Ragas. Therefore, we will only update test cases in the Promptfoo format going forward. For this evaluation, we utilized the Promptfoo platform and conducted tests using the GPT-4o large language model. All test results underwent manual verification to ensure accuracy.