1. Gabatarwa
Tsarin AI mai tattaunawa, kamar Siri, Google Assistant, Cortana, da Alexa, sun canza daga almara na kimiyya zuwa sassa muhimmanci na rayuwar yau da kullun. Wannan takarda tana magana ne akan tambaya mai mahimmanci ta yadda za a kimanta "nasarar" AI mai tattaunawa da ke mai da hankali kan bincike, tare da sanin cikakkiyar rikitarwa a cikin ayyana da auna wannan nasarar. Marubutan suna ba da shawarar ƙetare ma'auni mai girma ɗaya zuwa cikakken tsarin kimantawa mai fuskoki da yawa.
1.1. Bambanci tsakanin Chatbot da Mataimakin Mai Hikimar Wucin Gadi (PA)
Takardar ta zana bambanci mai mahimmanci:
- Chatbot: Da farko tsarin da ke dogara ne akan ƙa'idodi da aka tsara don tattaunawa (rubutu/magana) a cikin takamaiman fage ko don tattaunawar gaba ɗaya. Sune sassa na manyan tsarin AI kuma yawanci ba sa koyo ko aiwatar da ayyuka masu rikitarwa (misali, bots na Facebook Messenger).
- Mataimakin Mai Hikimar Wucin Gadi (PA): An gina su akan hadaddun algorithms na NLP, ML, da ANN. Suna da manufa, suna koyo daga hulɗa, kuma suna nufin samar da keɓaɓɓen gogewar taimako mai kama da na ɗan adam (misali, Siri, Alexa).
1.2. Halayen Mataimakin Mai Hikimar Wucin Gadi (PA)
Ya kamata PA masu kyau su ƙunshi mahimman halayen mataimakin ɗan adam:
- Hasashen Bukatun Mai Amfani: Fahimtar abubuwan da mai amfani ya fi so, mahallin, da halaye na musamman.
- Tsari Mai Inganci: Gudanar da bayanai, takardu, da ayyuka bisa tsari.
- Taimako Mai Ƙwazo: Ƙetare martani mai ra'ayi zuwa hasashe da ba da shawarar ayyuka.
- Sanin Mahalli: Kiyaye tarihin tattaunawa da mahallin yanayi.
2. Ra'ayoyin Kimantawa da Ake Shawarawa
Babban gudummawar shine tsarin kimantawa mai fuskoki huɗu don AI mai tattaunawa:
2.1. Ra'ayin Kwarewar Mai Amfani (UX)
Yana mai da hankali kan gamsuwar mai amfani, shiga ciki, da amfanin da ake ganin yana da amfani. Ma'auni sun haɗa da ƙimar nasarar aiki, santsin tattaunawa, makin gamsuwar mai amfani (misali, SUS, SUX), da ƙimar riƙewa. Wannan ra'ayi yana tambaya: Shin hulɗar tana da daɗi, inganci, kuma tana da taimako daga mahangar mai amfani?
2.2. Ra'ayin Dawo da Bayanai (IR)
Yana kimanta ikon tsarin na dawo da ingantaccen bayani mai dacewa don amsa tambayoyin mai amfani. Yana daidaita ma'auni na IR na gargajiya kamar Daidaito ($P = \frac{\text{Bayanai Masu Dacewa da aka dawo da su}}{\text{Dukkan Bayanan da aka dawo da su}}$), Tunawa ($R = \frac{\text{Bayanai Masu Dacewa da aka dawo da su}}{\text{Dukkan Bayanan Masu Dacewa}}$), da makin F1 ($F1 = 2 \cdot \frac{P \cdot R}{P + R}$) zuwa mahallin tattaunawa, la'akari da tarihin tattaunawa a matsayin wani ɓangare na tambaya.
2.3. Ra'ayin Harshe (Linguistic)
Yana kimanta ingancin samar da harshe da fahimta. Ma'auni sun haɗa da daidaiton nahawu, santsi, haɗin kai, da dacewar salon/murya. Kayan aiki kamar BLEU, ROUGE, da METEOR za a iya daidaita su, ko da yake suna da iyakoki don tattaunawar buɗe yanki.
2.4. Ra'ayin Hikimar Wucin Gadi (AI)
Yana auna "hikima" na tsarin—ikonsa na koyo, tunani, da daidaitawa. Wannan ya haɗa da kimanta daidaiton samfurin akan ayyukan rarraba niyya da gane ƙungiya, ingancin karatunsa (rikitarwar samfur), da ikonsa na sarrafa yanayin da ba a gani ba (gabaɗaya).
3. Matsayin Keɓancewa Ga Mutum
Takardar ta jaddada keɓancewa ga mutum a matsayin mabuɗin bambanci ga manyan PA. Ya haɗa da daidaita martani, shawarwari, da salon hulɗa bisa bayanan mai amfani ɗaya (abubuwan da ya fi so, tarihi, hali). Dabarun sun haɗa da tacewa ta haɗin gwiwa, tacewa ta tushen abun ciki, da ƙarfafa koyo tare da siginar lada ta musamman ga mai amfani. Kalubalen yana cikin daidaita keɓancewa tare da sirri da kuma guje wa kumfa na tacewa.
4. Kalubalen Yanzu & Hanyoyin Gaba
Kalubale: Ayyana "nasarar" gabaɗaya, ƙirƙirar ma'auni daidaitattun, cimma cikakkiyar fahimtar mahalli, tabbatar da ƙarfi da AI na ɗa'a, da kuma sarrafa amincewar mai amfani da sirri.
Hanyoyin Gaba: Haɓaka mataimakan nau'i-nau'i da yawa (haɗa hangen nesa, sauti), ci gaba a cikin tunani na hankali (amfani da albarkatu kamar ConceptNet ko samfura kamar GPT), mai da hankali kan ƙwaƙwalwar ajiyar lokaci mai tsawo da samfurin mai amfani, da ƙirƙirar ƙarin bayanan kimantawa masu rikitarwa da ƙalubale (fiye da sauƙaƙan Q&A).
5. Cikakkun Bayanai na Fasaha & Tsarin Lissafi
Ana iya tsara kimantawa. Bari tattaunawa ta zama jerin juyawa $D = \{ (U_1, S_1), (U_2, S_2), ..., (U_T, S_T) \}$, inda $U_t$ shine shigarwar mai amfani kuma $S_t$ shine martanin tsarin a juyi $t$. Ana iya ƙirƙira ingancin tsarin gabaɗaya $Q$ a matsayin haɗakar maki daga kowane ra'ayi mai nauyi:
$Q(D) = \alpha \cdot UX(D) + \beta \cdot IR(D) + \gamma \cdot Ling(D) + \delta \cdot AI(D)$
inda $\alpha, \beta, \gamma, \delta$ su ne ma'auni da ke nuna fifikon aikace-aikacen, kuma kowane aiki (misali, $UX(D)$) yana tattara ma'auni na matakin juyi ko matakin tattaunawa daga ra'ayinsa na daban.
Sakamakon Gwaji & Bayanin Chati: Yayin da guntun PDF da aka bayar ya ambaci Hoto na 1 da 2 (nuna fasali/iyakoki da ƙididdigar amfani na manyan PA), cikakken kimantawa zai haɗa da amfani da wannan tsarin ga takamaiman tsarin. Misali, mutum zai iya auna makin F1 (Ra'ayin IR) don tambayoyin gaskiya, matsakaicin makin mai amfani (Ra'ayin UX) akan ma'auni mai maki 5, da makin BLEU (Ra'ayin Harshe) don samar da martani, yana zana waɗannan ma'auni a cikin nau'ikan tsarin daban-daban ko a kan ma'auni na abokan hamayya a cikin ginshiƙi mai ginshiƙai da yawa.
6. Tsarin Bincike & Misalin Lamari
Aiwatar da Tsarin: Don kimanta sabon PA na yin ajiyar tafiye-tafiye, "TravelMate":
- UX: Gudanar da nazarin mai amfani wanda ke auna ƙimar kammala aikin don "yi ajiyar jirgi zuwa London mako mai zuwa ƙasa da $800" kuma a tattara Makin Mai Gabatarwa Net (NPS).
- IR: Lissafa Daidaito@1 don shawarwarin otal bisa ma'aunin mai amfani (misali, "mai son dabbobi, kusa da cikin gari").
- Harshe: Yi amfani da masu kimantawa na ɗan adam don ƙididdige dabi'ar martani akan ma'auni na 1-5 don tambayoyi masu rikitarwa kamar "Canza ajiyara zuwa wurin zama na taga, amma kawai idan ba a biya ƙarin kuɗi ba."
- AI: Auna daidaiton mai rarraba niyya akan saitin gwaji da aka ajiye wanda ke ɗauke da kalmomin da ba a gani ba don niyyar "book_car_rental".
Wannan tsari mai tsari yana ba da cikakken bayanin aiki, yana gano cewa yayin da TravelMate ya yi fice a IR (Daidaito@1 = 0.92), makin UX nasa yana da ƙasa saboda jinkirin lokutan amsa—wani fifiko bayyananne don gudu na ci gaba na gaba.
7. Ra'ayin Manazarcin: Cikakken Fahimta & Zargi
Cikakken Fahimta: Babban gudummawar Jadeja da Varia shine bayyanannen raba kimantawar AI mai tattaunawa zuwa fuskoki huɗu daban-daban, waɗanda galibi suka ci karo da juna. Yawancin ƴan masana'antu suna sha'awar ƙunƙuntaccen ma'auni na AI (kamar daidaiton niyya) ko binciken UX mai laushi, suna rasa daji saboda bishiyoyi. Wannan takarda ta yi hujja daidai cewa samfurin SOTA akan ma'auni na GLUE na iya zama mummunan mataimaki idan martaninsa yana da santsi a harshe amma ba shi da alaƙa (ya kasa IR) ko daidai amma an isar da shi tare da tausayin jadawali (ya kasa UX). Gaskiyar "nasarar" ita ce daidaiton Pareto, ba ma'auni na girman kai ba.
Kwararar Hankali: Tsarin takardar yana da amfani. Da farko ta kafa tattaunawa ta hanyar bambanta chatbots na kayayyaki da ainihin PA na AI—wani bayani mai mahimmanci a cikin kasuwa mai cike da tashin hankali. Sa'an nan kuma ta gina tsarin kimantawa daga tushe, farawa da gogewar mai amfani na zahiri (ƙarshen ƙarshe), motsawa zuwa aikin haƙiƙa (IR, Harshe), kuma ya ƙare a cikin ikon injin da ke ƙasa. Mayar da hankali na gaba akan keɓancewa ga mutum ya biyo baya a matsayin babban tsarin don ɗaga makin UX da IR fiye da ma'auni na gabaɗaya.
Ƙarfi & Kurakurai: Babban ƙarfin tsarin shine aiki mai yawa mai aiki, yana ba da lissafin dubawa ga manajan samfur da masu bincike. Duk da haka, babban laifinsa shine rashin aiki. Ya gano "abin da" amma ba shi da cikakken bayani kan "yadda". Ta yaya kuke haɗa makin UX na zahiri na 4.5/5 tare da makin F1 na 0.87 da yawa? Menene lanƙwasa ciniki? Takardar ta yi ishara ga kalubale kamar ma'auni na kimantawa amma ba ta shiga cikin babban aiki kamar ma'auni na "Bayan Wasa na Kwaikwayo" (BIG-bench) ko ƙaƙƙarfan ka'idojin kimantawa na ɗan adam da masu bincike suka tattauna a Cibiyar AI ta Allen. Bugu da ƙari, yayin da aka haskaka keɓancewa, matsanancun kalubalen kiyaye sirri da yuwuwar haɓaka son zuciya—batutuwa masu mahimmanci ga bincike na yanzu a cikin koyo na tarayya da ML na gaskiya—an taɓa su kawai a sauƙaƙe.
Fahimta Mai Aiki: Ga masu aiki: Dakatar da bayar da rahoton ma'auni guda ɗaya. Karɓi wannan dashboard mai fuskoki huɗu. Idan OKRs na ƙungiyar ku suna kawai game da rage ƙimar kuskuren kalma (AI/Harshe), kuna inganta don takarda bincike, ba samfur ba. Ga masu bincike: Mataki na gaba mai mahimmanci shine ƙirƙirar bayanan da aka haɗa, mai fuskoki da yawa da ƙalubale. Muna buƙatar daidai na ImageNet ko MS MARCO don AI mai tattaunawa wanda ke buƙatar tsarin don yin maki mai kyau akan duk fuskoki huɗu lokaci guda, watakila an yi wahayi ta falsafar kimantawa mai ayyuka da yawa da aka gani a cikin ayyuka kamar CycleGAN, inda nasara ta buƙaci gamsar da ƙuntatawa da yawa, masu gasa (daidaiton zagayowar, kiyaye ainihi, asarar adawa). Makomar kimantawar AI mai tattaunawa ba ta cikin nemo ma'auni na azurfa ba, amma a cikin injiniyance, ayyukan asara masu nauyi waɗanda ke nuna wannan gaskiyar mai fuskoki da yawa.
8. Nassoshi
- Jadeja, M., & Varia, N. (2017). Ra'ayoyin Kimantawa na AI Mai Tattaunawa. SCAI' 2017 Workshop a ICTIR'17. arXiv:1709.04734.
- Radford, A., et al. (2019). Samfuran Harshe Masu Koyo Ba tare da Kulawa ba. OpenAI Blog.
- Shuster, K., et al. (2022). Iyakokin Kimantawar Dan Adam da Bukatar Ma'auni na Kanta a cikin Tattaunawar Buɗe Yanki. Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics.
- Zhu, J.-Y., et al. (2017). Fassarar Hoton zuwa Hoton da ba a Haɗa su ba ta amfani da Cibiyoyin Adawa masu Daidaituwa na Zagayowar. IEEE International Conference on Computer Vision (ICCV). (CycleGAN)
- Sheng, E., et al. (2021). Matar ta yi aiki a matsayin mai renon yara: A kan Son Kai a cikin Samar da Harshe. Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP).
- Google AI. (n.d.). Ayyukan AI Masu Alhaki. An dawo daga https://ai.google/responsibilities/responsible-ai-practices/