Unit 9: Prototype evaluation

Activity 1: Discussion

Work in pairs. Discuss the answers to the following questions in simple English.

1. What is the primary purpose of your system?
2. Who is the intended user?
3. What is the knowledge base?
4. What inferences does the system make?
5. Is the system fully automated? Explain.

Activity 2: Essential and desirable features

Read the list of features for the expert system which show what the system needs to do.

All features

The expert system will be able to:

1. count, list and order the frequency of words essential
2. count, list and order the frequency of keywords essential
3. allow user to select reference corpus essential
4. allow user to select statistical formular for keyness
5. display the first 20 words/keywords of each dataset


6. order the known datasets by number of shared words/keywords with questionned dataset
7. display the shared words/keywords in the first 20 words/keywords of each dataset essential
8. identify the most similiar dataset based on the highest number of shared top 20 words/keywords of the questionned dataset
9. identify the least similiar dataset based on the lowest number of shared top 20 words/keywords of the questionned dataset essential
10. suggest to rule out the least similiar dataset essential
11. allow user to confirm or reject ruling out
12. continue to identify the most/least similiar dataset and rule out the least for the top 40, then 60 keywords


13. tag the part-of-speech (POS) of each word essential
14. allow the user to select any word or string and display the word or string in context essential
15. show 6 words before the target and 6 after the target word for each instance of the word in each dataset
16. allow the user to search for POS patterns following the target word, e.g. absolutely + JJ essential
17. count the number of identical POS patterns in each dataset essential
18. order the known datasets by number of identical POS patterns with the questionned dataset
19. identify the most similiar dataset based on the number of identical POS patterns with questionned dataset
20. identify the least similiar dataset based on the number of identical POS patterns with questionned dataset essential
21. suggest ruling out the least similiar dataset essential
22. allow user to confirm or reject ruling out

Activity 3: Evaluation criteria

Read.

The evaluation criteria for the expert system are directly extracted from the all features list. Step one is to determine whether a feature is incorporated into the program. This is best shown using a table, for example with 0 to show absence and 1 to show the presence of a feature. Step two is to evaluate the success of the feature. If the program is able to achieve the required result with sufficient accuracy and without unreasonable delay, then your evaluation will be positive. However, if there is a problem, describe the problem briefly and suggest a solution if possible.