Artificial Intelligence (CO3519): Coursework Assessment

UCLan "brief" version of the assessment specification

AI demonstrator implementation

Summary

The AI demonstrator implementation is a tool or software component that meaningfully applies content from the AI module to an interesting use case. It is supported by brief documents summarizing the use case, the software architecture, and how the material from the lecture and/or the suggested reading are related to your work.

The level of development expected from this work is at least that of a proof of concept, demonstrating that further development could make it ready for a real-life application scenario.

Submission: By 2^nd April 2022, end of day (24.00 hours, BST time zone), via Blackboard. The whole work (software and other documents) should be submitted as one archive file, e.g., in zip, tar.gz, or tar.bz2 format. It should contain one folder src/ with all source files (e.g., py or ipynb files) and additionally, one folder component-X/ per component, with X being the number of the respective component as given below.

Mandatory components:

1. A use-case documentation and summary of requirements deduced from the use case.
2. A demonstrator architecture specification component, documenting the module learning outcomes no. 1 to 3:
   1. Explain the theoretical underpinnings of algorithms and techniques specific to artificial intelligence.
   2. Critically evaluate the principles and algorithms of artificial intelligence.
   3. Analyse and evaluate the theoretical foundations of artificial intelligence and computing.

Elective components:

3. An optimization component.
4. A decision making and/or decision support and/or agent function component.
5. A data-driven modelling component.
6. A game-related component.
7. A knowledge representation and/or knowledge-base component.

Out of the five elective components mentioned above, at least two need to be included. It is recommended that you start working first, and then decide toward the end in what respects your work is strongest, presenting these as your two elective components. While it is permitted to submit more than two elective components, this will not in general tend to improve the grade (see the grading scheme below).

Group work by up to four people is welcome. If you work as a group, specify exactly one main caretaker for each component, such that each group member is mentioned as the main caretaker for at least one component.

Components and deliverables

1. Use-case documentation and summary

The use case should be of interest to you. Ideally, it is related to other work that you have been doing or plan to do outside the context of this module, for example, in one of the other modules, as a third-year project, or possibly plans for future research and development, within an academic context or outside it.

Recommendations for selecting a use case include:

• It is of interest to others. There should be an identifiable community or group of potential end users. This does not mean that you will be required within the present module to actually lead your demonstrator to the level where that community can already be addressed.
• You have access to data. Either you select a use case for which the actual data are accessible to the public, e.g., in domains such as finance, public administration, games and sports, or the COVID pandemic, among many others; or you specifically have access to data and the right to use them; or you have have sufficient dedicated domain knowledge to generate realistic data for development and demonstration purposes.

Deliverable D1: Describe a) the use case and b) software requirements deduced from the use case in a two-page A4 document. Literature references do not count toward the page limit. This document should be included in the component-1/ folder of the submitted archive; in group work, it must explicitly identify the main caretaker for component no. 1.

2. Architecture specification

The architecture specification should describe the interaction between the system and a potential user and make it clear how its parts interact and are integrated into one coherent system. Documentation should be provided to a sufficient level such that a developer or user with a disciplinary background in computing can validate, test, employ, and further develop your code.

If you begin your work from scratch, building on nothing (or only on teaching material provided within the module), say so explicitly. If you build on pre-existing work by yourself and others, which is very welcome, include an appropriate statement in the architecture specification. In that case, make it very clear what part of the submission constitutes the present work, since only this will be taken into account for grading.

Deliverable D2: Submit a three-page A4 document summarizing the architecture, relating to the use-case requirements identified above (see component no. 1). The architecture specification should in all brevity explain the employed techniques, and it should critically evaluate the involved design choices (against possible alternatives). Literature references do not count toward the page limit. This document should be included in the component-2/ folder of the submitted archive; in group work, it must explicitly identify the main caretaker for component no. 2. new part developed within the present coursework; only that part will be taken into account for grading.

Any required technical documentation should also be included in the component-2/ folder. This also does not count toward the page limit mentioned above.

3. to 7. Elective components

The submission should make it clear which of the major topics from the lecture are addressed by your work, including at least two out of:

3. Optimization (module part 1, corresponding to elective component no. 3)
4. Agents and Decisions (module part 2, corresponding to elective component no. 4)
5. Modelling (module part 3, corresponding to elective component no. 5)
6. Game Theory (module part 4, corresponding to elective component no. 6)
7. Knowledge Representation (module part 5, corresponding to elective component no. 7)

By the assessment release date, only parts 1 to 3 have been covered in the lectures; as specified above, you are nonetheless permitted (but not required) to include the topics to be discussed further onward, e.g., based on the suggested literature and the upcoming lectures and tutorial sessions (since the deadline is in early April 2022).

In line with the two or more included elective components, two or more deliverables DX and DY are to be submitted, with X and Y being any of the numbers 3 to 7 as mentioned above. They should consist of a one-pager that makes it clear in what way your work relates to the respective part of the CO3519 module, based on content discussed in the lecture or the tutorials, the recommended literature, or any independent research that you may have conducted on related topics. The main point of the one-pager is to establish that your work is on topic. In group work, the main caretaker for each component must be identified. Literature references do not count toward the page limit. Additionally, it should be stated (in a separate document or an appendix, or in any other intelligible way) what parts of the code are to be graded as part of this component.

Submit the documents mentioned above in the component-X/ folder of the submitted archive, where X is the component number.

Overall grading scheme

The components contribute to the grade as follows:

• 20% from component no. 1, use-case documentation and summary;
• 30% from component no. 2, architecture specification;
• 30% from elective components no. 3 to 7, where each included component has equal weight;
• 20% from the components for which the participant is the main caretaker; if there are multiple such components, each contributes equally to this part. (This is in addition to the contribution of these components via the three bullet points above).

It may be tactically advisable to submit the two strongest aspects of your work as elective components. Note that based on the scheme above, including additional components that receive a lower grade will lead to a lower overall grade.

Grading criteria

1^st criteria (70% to 100%)

• The demonstrator provides evidence of thorough independent research and development. Algorithms and techniques are tailored to the use case, and some explanation of how and why this was done is provided. The submitted material makes it easy for a developer or user with a disciplinary background in computing to validate, test, employ, and further develop the code.
• Use case and community or group of potential end users are identified plausibly and clearly; data source and/or method of data generation described plausibly and clearly, including any relevant citations, and establishing the justified expectation that data will be reliable and/or realistic. Specific requirements for the demonstrator architecture are deduced from an analysis of the use case and the needs of the addressed community or group.
• The arguments in favour of the selected algorithms and techniques are correct and convincing, and it is documented in detail how the design choices relate to the use-case requirements. The employed algorithms and techniques and potential alternative algorithms and techniques are summarized concisely, with a recognizable focus on aspects that are relevant to the present work, referencing the original sources and/or more recent relevant academic literature.

2.1 criteria (60% to 69%)

• The demonstrator provides evidence of independent development, and of tailoring the methodology to the use case. The submitted material makes it easy for a developer or user with a disciplinary background in computing to validate, test, and further develop the code.
• Use case identified clearly; data source and/or method of data generation described plausibly and clearly, including any relevant citations. Specific requirements for the demonstrator architecture deduced from an analysis of the use case.
• Some intelligible reasoning is provided to justify the design choices; it is outlined in what way the design choices are grounded in the use-case requirements. The employed algorithms and techniques are introduced jointly with potential alternatives, properly identifying them by correct technical terms in line with disciplinary good practice and including relevant literature references.

2.2 criteria (50% to 59%)

• The demonstrator provides evidence of the competency to implement artificial intelligence algorithms concerning all the relevant aspects of the module. It provides evidence of tailoring the methodology to the use case. The implementation is sufficiently well-documented such that a developer with a disciplinary background in computing is enabled to validate, test, and further develop the code.
• Discussion of a use case and a data source and/or method of data generation; use-case requirements are deduced from this discussion and formulated clearly.
• The design choices are discussed, including an intelligible statement on the relation between the architecture and the use case. The employed algorithms and techniques are introduced jointly with potential alternatives, properly identifying them by correct technical terms in line with disciplinary good practice.

Pass criteria (40% to 49%)

• The demonstrator provides evidence of the competency to implement artificial intelligence algorithms concerning some relevant aspect of the module, addressing the use case in a meaningful and intelligible way. The implementation is sufficiently documented to permit validation by a developer with a disciplinary background in computing.
• Intelligible remarks on a use case, a data source and/or method of data generation, and requirements for the demonstrator.
• An intelligible statement is included concerning the relation between the architecture and the use case. It is stated what the employed algorithms and techniques are, properly identifying them by correct technical terms in line with disciplinary good practice.