Context

Key  Performance  Indicators :
Understanding  Prompt  ranking  
Understanding  prompt  matching  
Ability  to  reuse  previous  knowledge
Wednesday
RAG  Evaluation  Data  Generation  (Rehmet)
Understanding  of  prompt  matching  and  ranking  (Emtinan)
Thursday
RAG  evaluation  metrics  (Emtinan)
RAGOps  - DevOps  of  RAG  development  and  production  deployment  (Rehmet)  
DELIVERABLES
NOTE :  Document  should  be  a  PDF  stored  in  google  drive  or  published  blog  link . DO  NOT
SUBMIT  A  LINK  as  PDF!  If  you  want  to  submit  pdf  document , it  should  be  the  content  of
your  report  not  a  link . 
Interim  Submission  - Wednesday  8pm  UTC
Link  to  your  code  in  GitHub
Repository  where  you  will  be  using  to  complete  the  tasks  in  this  week ' s
challenge . A  minimum  requirement  is  that  you  have  a  well  structured
repository  and  some  coding  progress  is  made .
A  review  report  of  your  reading  and  understanding  of  Task  1  and  any  progress  you
made  in  other  tasks . 
Feedback
You  may  not  receive  detailed  comments  on  your  interim  submission  but  will  receive  a
grade .
Final  Submission  - Saturday  8pm  UTC
Link  to  your  code  in  GitHub  
Complete  work   for  Automatic  prompt  generation
Complete  work   for  Automatic  evaluation  
Complete  work  for  Evaluation  Data  Generation
 
6/6/24, 11:51 PM 10 Academy Cohort B Weekly Challenge: Week 7
https://tenx.10academy.org/trainee/challenge/10 Academy Cohort B Weekly Challenge: Week 7-lwythrp7---75 9/1010  Academy  Cohort  B  Weekly
Challenge : Week  7
Precision  RAG : Prompt  Tuning  For  Building
Enterprise  Grade  RAG  Systems
BUSINESS OBJECTIVE 
  PromptlyTech  is  an  innovative  e - business  specializing  in  providing  AI - driven  solutions
for  optimizing  the  use  of  Language  Models  (LLMs)  in  various  industries . The  company
aims  to  revolutionize  how  businesses  interact  with  LLMs , making  the  technology  more
accessible , efficient , and  effective . By  addressing  the  challenges  of  prompt  engineering ,
the  company  plays  a  pivotal  role  in  enhancing  decision - making , operational  efficiency ,
and  customer  experience  across  various  industries . PromptlyTech ' s  solutions  are
designed  to  cater  to  the  evolving  needs  of  a  digitally - driven  business  landscape , where
speed  and  accuracy  are  key  to  staying  competitive .
The  company  focuses  on  key  services : Automatic  Prompt  Generation , Automatic
Evaluation  Data  Generation , and  Prompt  Testing  and  Ranking .
1 . Automatic  Prompt  Generation  Service :
This  service  streamlines  the  process  of  creating  effective  prompts , enabling
businesses  to  efficiently  utilize  LLMs  for  generating  high - quality , relevant  content .
It  significantly  reduces  the  time  and  expertise  required  in  crafting  prompts
manually .
2 . Automatic  Evaluation  Data  Generation  Service :
PromptlyTech ’ s  service  automates  the  generation  of  diverse  test  cases , ensuring
comprehensive  coverage  and  identifying  potential  issues . This  enhances  the
reliability  and  performance  of  LLM  applications , saving  significant  time  in  the
QA(Quality  Assurance)  process .
3 . Prompt  Testing  and  Ranking  Service :
6/6/24, 11:51 PM 10 Academy Cohort B Weekly Challenge: Week 7
https://tenx.10academy.org/trainee/challenge/10 Academy Cohort B Weekly Challenge: Week 7-lwythrp7---75 1/10Semantic  Similarity  Matching: Using  NLP  techniques  to  match  prompts  based  on
semantic  content , ideal  for  understanding  nuanced  differences  in  prompt
effectiveness .
You  should  adopt  an  innovative  approach  to  prompt  evaluation  by  utilizing  Monte  Carlo
matchmaking and    ELO  rating  systems , or  any  alternative  method  to  match  and  rank .
Task  5 : User  Interface  Development
Develop  a  user - friendly  interface  for  interacting  with  the  prompt  engineering  system .
UI  Design : Plan  and  design  a  user  interface  that  allows  users  to  easily  input  data ,
receive  prompts , and  view  evaluation  results .
UI  Implementation : Develop  and  integrate  the  user  interface  with  the  backend
prompt  engineering  system .
Task  6 : System  Integration  and  Testing
Integrate  all  components  of  the  system  and  conduct  comprehensive  testing .
Integrate  the  prompt  generation , Evaluation  Data  Generation , evaluation , and  user
interface  components .
Test  the  entire  system  for  functionality , usability , and  performance . Refine  based
on  feedback  and  test  results .
TUTORIALS SCHEDULE
In  the  following , the  Bold indicates  morning  sessions , and   Italic indicates  afternoon
sessions .
Monday : Understanding  Prompt  engineering  
Here  the  trainees  will  understand  the  week ’ s  challenge .
Introduction  to  Week  Challenge  (Yabebal)
Introduction  and  challenges  to  prompt  engineering  (Emtinan)
Key  Performance  Indicators :
Understanding  week ’ s  challenge
Understanding  the  prompt  engineering
Ability  to  reuse  previous  knowledge
Tuesday
RAG  components  (Rehmet)
Techniques  to  improve  R  (Retrievers)  in  RAG  (Emtinan)
6/6/24, 11:51 PM 10 Academy Cohort B Weekly Challenge: Week 7
https://tenx.10academy.org/trainee/challenge/10 Academy Cohort B Weekly Challenge: Week 7-lwythrp7---75 8/10