This site is not ready yet! The updated version will be available soon.
CS2103/T 2020 Jan-Apr
  • Full Timeline
  • Week 1 [Aug 12]
  • Week 2 [Aug 19]
  • Week 3 [Aug 26]
  • Week 4 [Sep 2]
  • Week 5 [Sep 9]
  • Week 6 [Sep 16]
  • Week 7 [Sep 30]
  • Week 8 [Oct 7]
  • Week 9 [Oct 14]
  • Week 10 [Oct 21]
  • Week 11 [Oct 28]
  • Week 12 [Nov 4]
  • Week 13 [Nov 11]
  • Textbook
  • Admin Info
  • Report Bugs
  • Forum
  • Instructors
  • Announcements
  • File Submissions
  • Tutorial Schedule
  • Java Coding Standard
  • Participation Marks List

  •  Individual Project (iP):
  • Individual Project Info
  • Duke Upstream Repo
  • iP Code Dashboard
  • iP Showcase

  •  Team Project (tP):
  • Team Project Info
  • Team IDs
  • Addressbook-level3
  • Addressbook-level 1,2,4
  • tP Code Dashboard
  • tP Showcase
  • tP: v1.4 [week 13] tP: Grading


    tP: Deliverables

    Here is a list of main deliverables of the project; their details are given in the subsequent sections.

    Deliverable: Executable

    • Should be an executable jar file.
    • Should be releasable. While some features may be scheduled for later versions, the features in v1.4 should be good enough to make it usable by at least some of the target users.

    Deliverable: Source code

    • Should match v1.4 deliverables i.e., executable, docs, website, etc.
    • To be delivered as a Git repo. Ensure your GitHub team repo is updated to match the executable.

    Deliverable: User Guide (UG)

    • Should cover all v1.4 features.
      Ensure those descriptions match the product precisely, as it will be used by peer testers (inaccuracies will be considered bugs).
    • Optionally, can also cover future features. Mark those as Coming in v2.0.

    Deliverable: Developer Guide (DG)

    • Should match the v1.4 implementation.
    • Optionally, can include proposed implementations of future features.
    • Include an appendix named Instructions for Manual Testing, to give some guidance to the tester to chart a path through the features, and provide some important test inputs the tester can copy-paste into the app.
      • Cover all user-testable features but no need to cover existing AB3 features if you did not touch them.
      • No need to give a long list of test cases including all possible variations. It is upto the tester to come up with those variations.
      • Inaccurate instructions will be considered bugs.

    Deliverable: Product Website

    • Should match the submitted deliverables e.g., the UG on the website should match the submitted PDF file.

    Website README

    • Ensure the Ui.png matches the current product

    Some common sense tips for a good product screenshot

    Ui.png represents your product in its full glory.

    • Before taking the screenshot, populate the product with data that makes the product look good. For example,
    • If the product is supposed to show photos, use real photos instead of dummy placeholders.
    • If the product doesn't have nice line wrapping for long inputs/outputs, don't use such inputs/outputs for the screenshot.
    • It should show a state in which the product is well-populated i.e., don't leave data panels largely blank
    • Choose a state that showcase the main features of the product i.e., the login screen is not usually a good choice
    • Take a clean screenshot with a decent resolution. Some screenshot tools can capture a specified window only. If your tool cannot do that, make sure you crop away the extraneous parts captured by the screenshot.
    • Avoid annotations (arrows, callouts, explanatory text etc.); it should look like the product is in use for real.

    Reason: Distracting annotations.

    Reason: Not enough data.

    Reason: screenshot not cropped cleanly (contains extra background details)

    Website AboutUs Page

    • Use a suitable profile photo.

    The purpose of the profile photo is for the teaching team to identify you. Therefore, choose a recent individual photo showing your face clearly (i.e., not too small) -- somewhat similar to a passport photo. Some examples can be seen in the 'Teaching team' page. Given below are some examples of good and bad profile photos.

    If you are uncomfortable posting your photo due to security reasons, you can post a lower resolution image so that it is hard for someone to misuse that image for fraudulent purposes. If you are concerned about privacy, you may use a placeholder image in place of the photo in module-related documents that are publicly visible.

    • Include a link to each person's PPP page.
    • Team member names match full names used by LumiNUS.

    Deliverable: Project Portfolio Page (PPP)

    At the end of the project each student is required to submit a Project Portfolio Page.

    PPP Objectives

    • For you to use (e.g. in your resume) as a well-documented data point of your SE experience
    • For evaluators to use as a data point to evaluate your project contributions

    PPP Sections to include

    • Overview: A short overview of your product to provide some context to the reader. The opening 1-2 sentences may be reused by all team members. If your product overview extends beyond 1-2 sentences, the remainder should be written by yourself.
    • Summary of Contributions --Suggested items to include:
      • Code contributed: Give a link to your code on tP Code Dashboard. The link is available in the Project List Page -- linked to the icon under your photo.
      • Features implemented: A summary of the features you implemented. If you implemented multiple features, you are recommended to indicate which one is the biggest feature.
      • Other contributions:
        • Contributions to project management e.g., setting up project tools, managing releases, managing issue tracker etc.
        • Evidence of helping others e.g. responses you posted in our forum, bugs you reported in other team's products,
        • Evidence of technical leadership e.g. sharing useful information in the forum

    Keep in mind that your feature(s) will be evaluated for depth, completeness, and effort. Use the PPP to convince evaluator how good those aspects of your features are.
    It is fine if you want to directly explain each of those aspects of your features in the PPP i.e., how deep the feature is, why it is complete, how hard it was to implement.

    • [Optional] Contributions to the User Guide: Reproduce the parts in the User Guide that you wrote. This can include features you implemented as well as features you propose to implement.
      The purpose of allowing you to include proposed features is to provide you more flexibility to show your documentation skills. e.g. you can bring in a proposed feature just to give you an opportunity to use a UML diagram type not used by the actual features.

    • [Optional] Contributions to the Developer Guide: Reproduce the parts in the Developer Guide that you wrote. Ensure there is enough content to evaluate your technical documentation skills and UML modelling skills. You can include descriptions of your design/implementations, possible alternatives, pros and cons of alternatives, etc.

    • [Optional] If you plan to use the PPP in your Resume, you can also include your SE work outside of the module (will not be graded).

    PPP Format

    PPP Page Limit

    Content Recommended Hard Limit
    Overview + Summary of contributions 0.5-1 2
    [Optional] Contributions to the User Guide 1-3
    [Optional] Contributions to the Developer Guide 3-6
    • The page limits given above are after converting to PDF format. The actual amount of content you require is actually less than what these numbers suggest because the HTML → PDF conversion adds a lot of spacing around content.

    Deliverable: Demo

    Demo Duration

    • Strictly 18 minutes for a 5-person team, 15 minutes for a 4-person team, 21 minutes for a 6-person team. Exceeding this limit will be penalized.
    • An additional 5 minutes will be given to set up i.e., you will be given access to the demo station 5 minutes before your allocated start time.

    Demo Target audience

    • Assume you are giving a demo to a higher-level manager of your company, to brief him/her on the current capabilities of the product. This is the first time they are seeing the new product you developed but they are familiar with the AddressBook-level3 (AB3) product. The actual audience are the evaluators (the team supervisor and another tutor).

    Demo Scope

    • Start by giving an overview of the product so that the evaluators get a sense of the full picture early. Include the following:
      • What is it? e.g., FooBar is a product to ensure the user takes frequent standing-breaks while working.
      • Who is it for? e.g., It is for someone who works at a PC, prefers typing, and wants to avoid prolonged periods of sitting.
      • How does it help? Give an overview of how the product's features help to solve the target problem for the target user

    Here is an example:

    Hi, welcome to the demo of our product FooBar. It is a product to ensure the user takes
    frequent standing-breaks while working.
    It is for someone who works at a PC, prefers typing, and wants to avoid prolonged periods
    of sitting.
    The user first sets the parameters such as frequency and targets, and then enters a
    command to record the start of the sitting time, ... The app shows the length of the
    sitting periods, and alerts the user if ...
    ...
    
    • There is no need to introduce team members or explain who did what. Reason: to save time.
    • Each person should do a fair share of the demo. However, it's OK for one member to do all the typing.
    • There is no need for you to demo your own work.
    • Present the features in a reasonable order: Organize the demo to present a cohesive picture of the product as a whole, presented in a logical order.
    • No need to cover design/implementation details as the manager is not interested in those details.
    • No need to demo features inherited from AB3 unless they are needed to explain your new features. Reason: existing features will not earn you marks, and the audience is already familiar with AB3 features.

    Demo Structure

    • Demo the product using the same executable you submitted, on your own laptop, using the TV.
    • It can be a sitting down demo. You'll be demonstrating the features using the TV while sitting down. But you may stand around the TV if you prefer that way.
    • It will be an uninterrupted demo. The audience members will not interrupt you during the demo. That means you should finish within the given time.
    • Use a sufficient amount of realistic demo data. e.g at least 20 contacts. Trying to demo a product using just 1-2 sample data creates a bad impression.
    • Dress code: The level of formality is up to you, but it is recommended that the whole team dress at the same level.

    Demo Tips

    • Plan the demo to be in sync with the impression you want to create. For example, if you are trying to convince that the product is easy to use, show the easiest way to perform a task before you show the full command with all the bells and whistles.
    • Spend as much time as possible on demonstrating the actual product. Not recommended to use slides (if you do, use them sparingly) or videos or lengthy narrations. Avoid skits, re-enactments, dramatizations etc. This is not a sales pitch or an informercial. While you need to show how a user use the product to get value, but you don’t need to act like an imaginary user. For example,
      [Instead of this] Jim get’s a call from boss. "Ring ring", "hello", "oh hi Jim, can we postpone the meeting?" "Sure". Jim hang up and curses the boss under his breath. Now he starts typing ..etc.
      [do this] If Jim needs to postpone the meeting, he can type …
      It’s not that dramatization is bad or we don’t like it. We simply don’t have enough time for it. Note that CS2101 demo requirements may differ. Different context → Different requirements.
    • Showcase how the feature improves the user’s life rather than simply describe each feature.
    • Rehearse the steps well and ensure you can do a smooth demo. Find a golden path and stick to it. Poor quality demos can affect your grade.
    • Don’t waste time repeating things the target audience already knows. e.g. no need to say things like "We are students from NUS, SoC".
    • No need to introduce next presenter at the end of your part Reason: to save time.

    Demo Special circumstances

    • If a significant feature was not merged on time: inform the tutor and get permission to show the unmerged feature using your own version of the code. Obviously, unmerged features earn much less marks than a merged equivalent but something is better than nothing.
    • If you are unable to come to the demo due to a valid reason, submit the evidence of your excuse e.g., MC to prof. The demo is part of module grading and absence without a valid reason will cause you to lose marks.

    Deliverable: Practical Exam Dry Run (PE-D)

    PE-D Overview

    What: The v1.3 is subjected to a round of peer acceptance/system testing, also called the Practical Exam (PE) Dry Run as this round of testing will be similar to the graded Practical Exam that will be done at v1.4.

    When, where: uses a 40 minute slot at the start of week 11 lecture

    PE Overview

    Objectives:

    • The primary objective of the PE is to increase the rigor of project grading. Assessing most aspects of the project involves an element subjectivity. As the project counts for 45% of the final grade, it is not prudent to rely on evaluations of tutors alone as there can be significant variations between how different tutors assess projects. That is why we collect more data points via the PE so as to minimize the chance of your project being affected by evaluator-bias.
    • PE is also used to evaluate your manual testing skills, product evaluation skills, effort estimation skills etc.
    • Note that significant project components are not graded solely based on peer ratings. Rather, PE data are mostly used to cross-validate tutors' grades and identify cases that need further investigation. When peer inputs are used for grading, they are usually combined with tutor evaluations with appropriate weight for each. In some cases ratings from team members are given a higher weight compared to ratings from other peers, if that is appropriate.
    • Note that the PE is not a means of pitting you against each other. Developers and testers play for the same side; they need to push each other to improve the quality of their work -- not bring down each other.

    Grading:

    • Your performance in the practical exam will affect your final grade and your peers', as explained in Admin: Project Grading section.
    • As such, we have put in measures to identify and penalize insincere/random evaluations.
    • Also see:

    Relevant: [Admin tP Grading → Notes on how marks are calculated for PE ]

    Notes on how marks are calculated based on PE product testing
    • Of 3A and 3B above, the one you do better will be given a 70% weight and the other a 30% weight so that your total score is driven by your strengths rather than weaknesses.
    • Bugs rejected by the dev team, if the rejection is approved by the teaching team, will not be affect marks of the tester or the developer.
    • The penalty/credit for a bug varies based on,
      • The severity of the bug: severity.High > severity.Medium > severity.Low > severity.VeryLow
      • The type of the bug: type.FunctionalityBug > type.DocumentationBug > type.FeatureFlaw
    • The penalty for a bug is divided equally among assignees.
    • The developers are not penalized for the duplicate bug reports they received but the testers earn credit for the duplicate bug reports they submitted as long as the duplicates are not submitted by the same tester.
    • Obvious bugs earn less credit for the tester and slightly more penalty for the developer.
    • If the team you tested has a low bug count i.e., total bugs found by all testers is low, we will fall back on other means (e.g., performance in PE dry run) to calculate your marks for system/acceptance testing.
    • Your marks for developer testing depends on the bug density rather than total bug count. Here's an example:
      • n bugs found in your feature; it is a difficult feature consisting of lot of code → 4/5 marks
      • n bugs found in your feature; it is a small feature with a small amount of code → 1/5 marks
    • You don't need to find all bugs in the product to get full marks. For example, finding half of the bugs of that product or 4 bugs, whichever the lower, could earn you full marks.
    • Excessive incorrect downgrading/rejecting/ duplicate-flagging, if deemed an unethical attempt to game the system, may be penalized.

    PE Preparation

    • It's similar to,
    • Ensure that you have accepted the invitation to join the GitHub org used by the module. Go to https://github.com/nus-cs2103-AY1920S2 to accept the invitation.

    • Ensure you have access to a computer that is able to run module projects e.g. has the right Java version.

    • Download the latest CATcher and ensure you can run it on your computer.

    Issues created for PED and PE need to be in a precise format for our grading scripts to work. Incorrectly-formatted responses will have to discarded. Therefore, you are strongly recommended to use CATcher for PED and PE activities. If you want to give your response via GitHub instead, please get our permission first.

    • Create a public repo in your GitHub account with the following name:
      • PE Dry Run: ped
      • PE: pe
    • Enable its issue tracker and add the following labels to it (the label names should be precisely as given).

    Bug Severity labels:

    • severity.VeryLow : A flaw that is purely cosmetic and does not affect usage e.g., a typo/spacing/layout/color/font issues in the docs or the UI that doesn't affect usage.
    • severity.Low : A flaw that is unlikely to affect normal operations of the product. Appears only in very rare situations and causes a minor inconvenience only.
    • severity.Medium : A flaw that causes occasional inconvenience to some users but they can continue to use the product.
    • severity.High : A flaw that affects most users and causes major problems for users. i.e., makes the product almost unusable for most users.

    Type labels:

    • type.FunctionalityBug: A functionality does not work as specified/expected.
    • type.FeatureFlaw: Some functionality missing from a feature delivered in v1.4 in a way that the feature becomes less useful to the intended target user for normal usage. i.e., the feature is not 'complete'. In other words, an acceptance testing bug that falls within the scope of v1.4 features. These issues are counted against the 'depth and completeness' of the feature.
    • type.DocumentationBug: A flaw in the documentation that can potentially affect the user e.g., a missing step, a wrong instruction, typos that affect users
    • Have a good screen grab tool with annotation features so that you can quickly take a screenshot of a bug, annotate it, and post in the issue tracker.

      • You can use Ctrl+V to paste a picture from the clipboard into a text box in bug report.
    • Download the product to be tested after you have been notified of which team you have been allocated to test.

    • Charge your computer before coming to the session. The testing venue might not have enough charging points.

    PE Phase 1: Bug Reporting

    • When, where: Week 13 lecture
    PE Phase 1 - Part I Product Testing [60 minutes]
    • Test the product and report bugs as described below. You may report both product bugs and documentation bugs during this period.
    Testing instructions for PE and PE-D
    a) Launching the JAR file
    • Put the jar file in an empty folder.
    • Open a command window. Run the java -version command to ensure you are using Java 11.
    • Launch the jar file using the java -jar command (do not use double-clicking).
    • If the product doesn't work at all: If the product fails catastrophically e.g., cannot even launch, you can test the fallback team allocated to you. But in this case you must inform us immediately after the session so that we can send your bug reports to the correct team.
      • PE-D: Download the JAR file from their GitHub page
      • PE: 1. Download from LumiNUS all files submitted by the team (i.e. jar file, User Guide, Developer Guide, and Project Portfolio Pages) into an empty folder.
    b) What to test
    • PE Dry Run (at v1.3):
      • Test the product based on the User Guide (the UG is most likely accessible using the help command).
      • Do system testing first i.e., does the product work as specified by the documentation?. If there is time left, you can do acceptance testing as well i.e., does the product solve the problem it claims to solve?.
    • PE (at v1.4):
      • Test based on the Developer Guide (Appendix named Instructions for Manual Testing) and the User Guide. The testing instructions in the Developer Guide can provide you some guidance but if you follow those instructions strictly, you are unlikely to find many bugs. You can deviate from the instructions to probe areas that are more likely to have bugs.
      • As before, do both system testing and acceptance testing but give priority to system testing as system-testing bugs can earn you more credit.
    c) What bugs to report?
    • PE Dry Run (at v1.3): You may report functionality bugs, UG bugs, and feature flaws.

    Relevant: [Admin tP Grading → Functionality Bugs ]

    These are considered functionality bugs:
    Behavior differs from the User Guide
    A legitimate user behavior is not handled e.g. incorrect commands, extra parameters
    Behavior is not specified and differs from normal expectations e.g. error message does not match the error

    Relevant: [Admin tP Grading → Functionality Bugs ]

    These will be considered feature flaws:
    The feature does not solve the stated problem of the intended user i.e., the feature is 'incomplete'
    Hard-to-test features
    Features that don't fit well with the product
    Features that are not optimized enough for fast-typists or target users

    Relevant: [Admin tP Grading → Possible UG Bugs ]

    These are considered UG bugs (if they hinder the reader):
    Not enough visuals e.g., screenshots/diagrams
    The visuals are not well integrated to the explanation.
    The visuals are unnecessarily repetitive e.g., same visual repeated with minor changes.
    Not enough examples e.g., sample inputs/outputs.
    The explanation is too brief or unnecessarily long.
    The information is hard to understand for the target audience. e.g., using terms the reader might not know
    The document looks messy, or not well-formatted.

    • PE (at v1.4): In addition to the reporting functionality bugs, UG bugs, and feature flaws, there will be an extra time given specifically for reporting documentation bugs (i.e., UG and DG bugs).

    Relevant: [Admin tP Grading → Possible DG Bugs ]

    These are considered DG bugs (if they hinder the reader):

    These are considered UG bugs (if they hinder the reader):
    Not enough visuals e.g., screenshots/diagrams
    The visuals are not well integrated to the explanation.
    The visuals are unnecessarily repetitive e.g., same visual repeated with minor changes.
    Not enough examples e.g., sample inputs/outputs.
    The explanation is too brief or unnecessarily long.
    The information is hard to understand for the target audience. e.g., using terms the reader might not know
    The document looks messy, or not well-formatted.

    UML notation incorrect or not compliant with the notation covered in the module.
    Some other type of diagram used when a UML diagram would have worked just as well.
    The diagram used is not suitable for the purpose it is used.
    The diagram is too complicated.
    Excessive use of code e.g., a large chunk of code is cited when a smaller extract of would have sufficed.

    • About posting suggestions:
      • PE Dry Run (at v1.3): You can also post suggestions on how to improve the product. Be diplomatic when reporting bugs or suggesting improvements. For example, instead of criticising the current behavior, simply suggest alternatives to consider.
      • PE (at v1.4): Do not post suggestions. But if a feature is missing a critical functionality that makes the feature less useful to the intended user, it can be reported as a bug of type Type.FeatureFlaw.
    d) How to report bugs
    • Post bugs as you find them (i.e., do not wait to post all bugs at the end) because bug reports created/modified after the allocated time will not count.
    • Launch CATcher, and login to the correct profile:
      • PE Dry Run: CS2103/T PE Dry run
      • PE: CS2103/T PE
    • Post bugs using CATcher.
    • Post bug reports in the following repo you created earlier:
      • PE Dry Run: ped
      • PE: pe
    • The whole description of the bug should be in the issue description i.e., do not add comments to the issue.
    e) Bug report format
    • Each bug should be a separate issue.
    • Write good quality bug reports; poor quality or incorrect bug reports will not earn credit.
    • Use a descriptive title.
    • Give a good description of the bug with steps to reproduce and screenshots. If the receiving team cannot reproduce the bug, you will not be able to get credit for it.
    • Assign exactly one severity.* label to the bug report. Bug report without a severity label are considered severity.Low (lower severity bugs earn lower credit)

    Bug Severity labels:

    • severity.VeryLow : A flaw that is purely cosmetic and does not affect usage e.g., a typo/spacing/layout/color/font issues in the docs or the UI that doesn't affect usage.
    • severity.Low : A flaw that is unlikely to affect normal operations of the product. Appears only in very rare situations and causes a minor inconvenience only.
    • severity.Medium : A flaw that causes occasional inconvenience to some users but they can continue to use the product.
    • severity.High : A flaw that affects most users and causes major problems for users. i.e., makes the product almost unusable for most users.
    • Assign exactly one type.* label to the issue.

    Type labels:

    • type.FunctionalityBug: A functionality does not work as specified/expected.
    • type.FeatureFlaw: Some functionality missing from a feature delivered in v1.4 in a way that the feature becomes less useful to the intended target user for normal usage. i.e., the feature is not 'complete'. In other words, an acceptance testing bug that falls within the scope of v1.4 features. These issues are counted against the 'depth and completeness' of the feature.
    • type.DocumentationBug: A flaw in the documentation that can potentially affect the user e.g., a missing step, a wrong instruction, typos that affect users
    PE Phase 1 - Part II Evaluating Documents [20 minutes]
    • This slot is for reporting documentation bugs only. You may report bugs related to UG and DG.
    • For each bug reported, cite evidence and justify. For example, if you think the explanation of a feature is too brief, explain what information is missing and why the omission hinders the reader.

    Relevant: [Admin tP Grading → Possible UG Bugs ]

    These are considered UG bugs (if they hinder the reader):
    Not enough visuals e.g., screenshots/diagrams
    The visuals are not well integrated to the explanation.
    The visuals are unnecessarily repetitive e.g., same visual repeated with minor changes.
    Not enough examples e.g., sample inputs/outputs.
    The explanation is too brief or unnecessarily long.
    The information is hard to understand for the target audience. e.g., using terms the reader might not know
    The document looks messy, or not well-formatted.

    Relevant: [Admin tP Grading → Possible DG Bugs ]

    These are considered DG bugs (if they hinder the reader):

    These are considered UG bugs (if they hinder the reader):
    Not enough visuals e.g., screenshots/diagrams
    The visuals are not well integrated to the explanation.
    The visuals are unnecessarily repetitive e.g., same visual repeated with minor changes.
    Not enough examples e.g., sample inputs/outputs.
    The explanation is too brief or unnecessarily long.
    The information is hard to understand for the target audience. e.g., using terms the reader might not know
    The document looks messy, or not well-formatted.

    UML notation incorrect or not compliant with the notation covered in the module.
    Some other type of diagram used when a UML diagram would have worked just as well.
    The diagram used is not suitable for the purpose it is used.
    The diagram is too complicated.
    Excessive use of code e.g., a large chunk of code is cited when a smaller extract of would have sufficed.

    PE Phase 1 - Part III Overall Evaluation [10 minutes]
    • Evaluate the following aspects. Submit via TEAMMATES. You are recommended to complete this during the PE session itself, but you have until the end of the day to submit (or revise) your submissions.

    A. Product Design []:

    Evaluate the product design based on the User Guide and the actual product behavior.

    Target user:

    • target user specified and appropriate: The target user is clearly specified, prefers typing over other modes of input, and not too general (should be narrowed to a specific user group with certain characteristics).
    • value specified and matching: The value offered by the product is clearly specified and matches the target user.
    • optimized for the target user: It feels like a fast typist can be more productive with the app, compared to an equivalent GUI app without a CLI.

    Value to the target user:

    B. Quality of user docs []:

    UG: Compared to AB3, the quality of this UG is,

    C. Quality of developer docs []:

    DG: similar to UG

    D. Feature Quality []:

    Quality: Compared to AB3, the quality of this product is,

    E. Amount of work []:

    Effort: Assume the effort required to create AB3 from scratch is 10 in a scale of 0 to 30. How much effort do you estimate the team put in for this project?

    • Do not give a high value just to be nice. Your responses will be used to evaluate your effort estimation skills.

    PE Phase 2: Developer Response

    This phase is for you to respond to the bug reports you received.

    Duration: The review period will start around 1 day after the PE (exact time to be announced) and will last until the following Tuesday midnight. However, you are recommended to finish this task ASAP, to minimize cutting into your exam preparation work.

    Bug reviewing is recommended to be done as a team as some of the decisions need team consensus.

    Instructions for Reviewing Bug Reports

    • Don't freak out if there are lot of bug reports. Many can be duplicates and some can be false positives. In any case, we anticipate that all of these products will have some bugs and our penalty for bugs is not harsh. Furthermore, it depends on the severity of the bug. Some bug may not even be penalized.
    • CATcher does not come with a UG, but the UI is fairly intuitive (there are tool tips too). Do post in the forum if you need any guidance with its usage.
    • Also note that CATcher hasn't been battle-tested for this phase, in particular, w.r.t. multiple team members editing the same issue concurrently. It is ideal if the team members get together and work through the issues together. If you think others might be editing the same issues at the same time, use the Sync button at the top to force-sync your view with the latest data from GitHub.
  • Launch CATcher, and login to the profile CS2103/T PE. It will show all the bugs assigned to your team, divided into three sections:
    1. Issues Pending Responses - Issues that your team has not processed yet.
    2. Issues Responded - Your job is to get all issues to the second category.
    3. Faulty Issues - e.g., Bugs marked as duplicates of each other, or causing circular duplicate relationships. Fix the problem given so that no issues remain in this category.
  • Respond to the bug reports shown.
  • Issues created for PE-D and PE need to be in a precise format for our grading scripts to work. Incorrectly-formatted responses will have to discarded. Therefore, you are strongly recommended to use CATcher for PE-D and PE activities. If you want to give your response via GitHub instead, please get our permission first.

    • Go to the dev-response issue tracker
    • Use tutorial.* and team.* labels to filter bug reports your team received.
    • Do not edit the subject or the description. Your response (if any) should be added as a comment.
    • Add a comment using the following exact template.
    # Team's Response
    {replace this with your response}
    

    ## Duplicate status (if any):

    Here is an example:

    # Team's Response
    

    Yes this is a bug. But it is a duplicate. * Changed the bug type because this is just a bug in the UG. * Lowered the severity because users can still use the feature.

    ## Duplicate status (if any): Duplicate of #67

    • Do not close the bug report after you are done processing it.
    • Use the exact Duplicate of #123 format to indicate duplicates.
    • There should be exactly one comment per issue. If there are multiple comments, the last one will be taken for processing.
    • If a bug seems to be for a different product (i.e. wrongly assigned to your team), let us know ASAP.
    • If the bug is reported multiple times,
      • Mark all copies EXCEPT one as duplicates of the one left out (let's call that one the original) using the duplicate tag.
      • If the duplicates have different severity levels, you should keep the one with the highest severity as the original. But you can downgrade the severity of the original or the duplicates.
      • For each group of duplicates, all duplicates should point to one original i.e., no multiple levels of duplicates, and no cyclical duplication relationships.
      • If the duplication status is eventually accepted, all duplicates will be assumed to have inherited the type.* and severity.* from the original.

    • Apply exactly one of these labels (if missing, we assign: response.Accepted)

    Response Labels:

    • response.Accepted: You accept it as a bug.
    • response.NotInScope: It is a valid issue but not something the team should be penalized for e.g., it was not related to features delivered in v1.4.
    • response.Rejected: What tester treated as a bug is in fact the expected behavior. You can reject bugs that you inherited from AB3.
    • response.CannotReproduce: You are unable to reproduce the behavior reported in the bug after multiple tries.
    • response.IssueUnclear: The issue description is not clear. Don't post comments asking the tester to give more info. The tester will not be able to see those comments because the bug reports are anonymous.
    • Apply exactly one of these labels (if missing, we assign: type.FunctionalityBug)

    Type labels:

    • type.FunctionalityBug: A functionality does not work as specified/expected.
    • type.FeatureFlaw: Some functionality missing from a feature delivered in v1.4 in a way that the feature becomes less useful to the intended target user for normal usage. i.e., the feature is not 'complete'. In other words, an acceptance testing bug that falls within the scope of v1.4 features. These issues are counted against the 'depth and completeness' of the feature.
    • type.DocumentationBug: A flaw in the documentation that can potentially affect the user e.g., a missing step, a wrong instruction, typos that affect users
    • If you disagree with the original severity assigned to the bug, you may change it to the correct level.

    Bug Severity labels:

    • severity.VeryLow : A flaw that is purely cosmetic and does not affect usage e.g., a typo/spacing/layout/color/font issues in the docs or the UI that doesn't affect usage.
    • severity.Low : A flaw that is unlikely to affect normal operations of the product. Appears only in very rare situations and causes a minor inconvenience only.
    • severity.Medium : A flaw that causes occasional inconvenience to some users but they can continue to use the product.
    • severity.High : A flaw that affects most users and causes major problems for users. i.e., makes the product almost unusable for most users.
    • Decide who should fix the bug. Use the Assignees field to assign the issue to that person(s). There is no need to actually fix the bug though. It's simply an indication/acceptance of responsibility. If there is no assignee, we will distribute the penalty for that bug (if any) among all team members.
      • If it is not easy to decide the assignee(s), we recommend (but not enforce) that the feature owner should be assigned bugs related to the feature, Reason: The feature owner should have defended the feature against bugs using automated tests and defensive coding techniques.

    • As far as possible, choose the correct type.*, severity.*, and assignees even for bugs you are not accepting or for bugs that are marked as duplicates. Reason: your non-acceptance or duplication status may be rejected in a later phase, in which case we need to grade it as an accepted/non-duplicate bug.

    • Justify your response. For all of the following cases, you must add a comment justifying your stance. Testers will get to respond to all those cases and will be double-checked by the teaching team in later phases. Indiscriminate/unreasonable dev/tester responses, if deemed as a case of trying to game the system, will be penalized.

      • downgrading severity
      • non-acceptance of a bug
      • changing the bug type
      • non-obvious duplicate

    PE Phase 3: Tester Response

    • In this phase you will get to state whether you agree or disagree with the dev response to the bugs you reported. If a bug reported has been subjected to any of the below by the receiving team, you can record your objections and the reason for the objection.
      • not accepted
      • severity downgraded
      • bug type changed
    • As before, consider carefully before you object to a team's response. If many of your objections were overruled by the teaching team later, you will lose marks for not being able to evaluate a bug report properly.
    • This phase is optional. If you do not respond to a dev response, we'll assume that you agree with it.
    • Deadline: Reading week Thursday 2359
    • Procedure:
    • When the phase has been announced as open, login to CATcher as usual (profile: CS2103/T PE).
    • For the issues listed, go to the details, and read the team's response.
    • If you disagree with any of the items listed, tick on the I disagree tick box and enter your justification for the disagreement, and click Save.

    Issues created for PE-D and PE need to be in a precise format for our grading scripts to work. Incorrectly-formatted responses will have to discarded. Therefore, you are strongly recommended to use CATcher for PE-D and PE activities. If you want to give your response via GitHub instead, please get our permission first.

    • When the phase has been announced as open, go to your pe repo.
    • For the issues listed, go to the details, and read the bot's comment containing the team's response.
    • If you disagree with any of the items listed in that comment, tick on the [ ] I disagree tick box. Edit the comment (do not add a new comment) to replace the [replace this with your reason] with your reasoning. Here is an example:
    Team chose ['response.IssueUnclear']
    
    - [x] I disagree
    
    **Reason for disagreement:** I think one can easily reproduce the problem by following the steps I gave.
    

    PE Phase 4: Tutor Moderation

    • In this phase tutors will look through all dev responses you objected to in the previous phase and decide on a final outcome.
    • In the unlikely we need your inputs, the tutor will contact you.

    Grading: Taking part in the PE dry run is strongly encouraged as it can affect your grade in the following ways.

    • If the product you are allocated to test in the Practical Exam (at v1.4) had a very low bug count, we will consider your performance in PE dry run as well when grading the PE.
    • PE dry run will help you practice for the actual PE.
    • Taking part in the PE dry run will earn you participation points.
    • There is no penalty for bugs reported in your product. Every bug you find is a win-win for you and the team whose product you are testing.

    Why:

    • To train you to do manual testing, bug reporting, bug triaging, bug fixing, communicating with users/testers/developers, evaluating products etc.
    • To help you improve your product before the final submission.

    PE-D Preparation

    • Ensure that you have accepted the invitation to join the GitHub org used by the module. Go to https://github.com/nus-cs2103-AY1920S2 to accept the invitation.

    • Ensure you have access to a computer that is able to run module projects e.g. has the right Java version.

    • Download the latest CATcher and ensure you can run it on your computer.

    Issues created for PED and PE need to be in a precise format for our grading scripts to work. Incorrectly-formatted responses will have to discarded. Therefore, you are strongly recommended to use CATcher for PED and PE activities. If you want to give your response via GitHub instead, please get our permission first.

    • Create a public repo in your GitHub account with the following name:
      • PE Dry Run: ped
      • PE: pe
    • Enable its issue tracker and add the following labels to it (the label names should be precisely as given).

    Bug Severity labels:

    • severity.VeryLow : A flaw that is purely cosmetic and does not affect usage e.g., a typo/spacing/layout/color/font issues in the docs or the UI that doesn't affect usage.
    • severity.Low : A flaw that is unlikely to affect normal operations of the product. Appears only in very rare situations and causes a minor inconvenience only.
    • severity.Medium : A flaw that causes occasional inconvenience to some users but they can continue to use the product.
    • severity.High : A flaw that affects most users and causes major problems for users. i.e., makes the product almost unusable for most users.

    Type labels:

    • type.FunctionalityBug: A functionality does not work as specified/expected.
    • type.FeatureFlaw: Some functionality missing from a feature delivered in v1.4 in a way that the feature becomes less useful to the intended target user for normal usage. i.e., the feature is not 'complete'. In other words, an acceptance testing bug that falls within the scope of v1.4 features. These issues are counted against the 'depth and completeness' of the feature.
    • type.DocumentationBug: A flaw in the documentation that can potentially affect the user e.g., a missing step, a wrong instruction, typos that affect users
    • Have a good screen grab tool with annotation features so that you can quickly take a screenshot of a bug, annotate it, and post in the issue tracker.

      • You can use Ctrl+V to paste a picture from the clipboard into a text box in bug report.
    • Download the product to be tested after you have been notified of which team you have been allocated to test.

    • Charge your computer before coming to the session. The testing venue might not have enough charging points.

    PE-D During the session

    Testing instructions for PE and PE-D
    a) Launching the JAR file
    • Put the jar file in an empty folder.
    • Open a command window. Run the java -version command to ensure you are using Java 11.
    • Launch the jar file using the java -jar command (do not use double-clicking).
    • If the product doesn't work at all: If the product fails catastrophically e.g., cannot even launch, you can test the fallback team allocated to you. But in this case you must inform us immediately after the session so that we can send your bug reports to the correct team.
      • PE-D: Download the JAR file from their GitHub page
      • PE: 1. Download from LumiNUS all files submitted by the team (i.e. jar file, User Guide, Developer Guide, and Project Portfolio Pages) into an empty folder.
    b) What to test
    • PE Dry Run (at v1.3):
      • Test the product based on the User Guide (the UG is most likely accessible using the help command).
      • Do system testing first i.e., does the product work as specified by the documentation?. If there is time left, you can do acceptance testing as well i.e., does the product solve the problem it claims to solve?.
    • PE (at v1.4):
      • Test based on the Developer Guide (Appendix named Instructions for Manual Testing) and the User Guide. The testing instructions in the Developer Guide can provide you some guidance but if you follow those instructions strictly, you are unlikely to find many bugs. You can deviate from the instructions to probe areas that are more likely to have bugs.
      • As before, do both system testing and acceptance testing but give priority to system testing as system-testing bugs can earn you more credit.
    c) What bugs to report?
    • PE Dry Run (at v1.3): You may report functionality bugs, UG bugs, and feature flaws.

    Relevant: [Admin tP Grading → Functionality Bugs ]

    These are considered functionality bugs:
    Behavior differs from the User Guide
    A legitimate user behavior is not handled e.g. incorrect commands, extra parameters
    Behavior is not specified and differs from normal expectations e.g. error message does not match the error

    Relevant: [Admin tP Grading → Functionality Bugs ]

    These will be considered feature flaws:
    The feature does not solve the stated problem of the intended user i.e., the feature is 'incomplete'
    Hard-to-test features
    Features that don't fit well with the product
    Features that are not optimized enough for fast-typists or target users

    Relevant: [Admin tP Grading → Possible UG Bugs ]

    These are considered UG bugs (if they hinder the reader):
    Not enough visuals e.g., screenshots/diagrams
    The visuals are not well integrated to the explanation.
    The visuals are unnecessarily repetitive e.g., same visual repeated with minor changes.
    Not enough examples e.g., sample inputs/outputs.
    The explanation is too brief or unnecessarily long.
    The information is hard to understand for the target audience. e.g., using terms the reader might not know
    The document looks messy, or not well-formatted.

    • PE (at v1.4): In addition to the reporting functionality bugs, UG bugs, and feature flaws, there will be an extra time given specifically for reporting documentation bugs (i.e., UG and DG bugs).

    Relevant: [Admin tP Grading → Possible DG Bugs ]

    These are considered DG bugs (if they hinder the reader):

    These are considered UG bugs (if they hinder the reader):
    Not enough visuals e.g., screenshots/diagrams
    The visuals are not well integrated to the explanation.
    The visuals are unnecessarily repetitive e.g., same visual repeated with minor changes.
    Not enough examples e.g., sample inputs/outputs.
    The explanation is too brief or unnecessarily long.
    The information is hard to understand for the target audience. e.g., using terms the reader might not know
    The document looks messy, or not well-formatted.

    UML notation incorrect or not compliant with the notation covered in the module.
    Some other type of diagram used when a UML diagram would have worked just as well.
    The diagram used is not suitable for the purpose it is used.
    The diagram is too complicated.
    Excessive use of code e.g., a large chunk of code is cited when a smaller extract of would have sufficed.

    • About posting suggestions:
      • PE Dry Run (at v1.3): You can also post suggestions on how to improve the product. Be diplomatic when reporting bugs or suggesting improvements. For example, instead of criticising the current behavior, simply suggest alternatives to consider.
      • PE (at v1.4): Do not post suggestions. But if a feature is missing a critical functionality that makes the feature less useful to the intended user, it can be reported as a bug of type Type.FeatureFlaw.
    d) How to report bugs
    • Post bugs as you find them (i.e., do not wait to post all bugs at the end) because bug reports created/modified after the allocated time will not count.
    • Launch CATcher, and login to the correct profile:
      • PE Dry Run: CS2103/T PE Dry run
      • PE: CS2103/T PE
    • Post bugs using CATcher.
    • Post bug reports in the following repo you created earlier:
      • PE Dry Run: ped
      • PE: pe
    • The whole description of the bug should be in the issue description i.e., do not add comments to the issue.
    e) Bug report format
    • Each bug should be a separate issue.
    • Write good quality bug reports; poor quality or incorrect bug reports will not earn credit.
    • Use a descriptive title.
    • Give a good description of the bug with steps to reproduce and screenshots. If the receiving team cannot reproduce the bug, you will not be able to get credit for it.
    • Assign exactly one severity.* label to the bug report. Bug report without a severity label are considered severity.Low (lower severity bugs earn lower credit)

    Bug Severity labels:

    • severity.VeryLow : A flaw that is purely cosmetic and does not affect usage e.g., a typo/spacing/layout/color/font issues in the docs or the UI that doesn't affect usage.
    • severity.Low : A flaw that is unlikely to affect normal operations of the product. Appears only in very rare situations and causes a minor inconvenience only.
    • severity.Medium : A flaw that causes occasional inconvenience to some users but they can continue to use the product.
    • severity.High : A flaw that affects most users and causes major problems for users. i.e., makes the product almost unusable for most users.
    • Assign exactly one type.* label to the issue.

    Type labels:

    • type.FunctionalityBug: A functionality does not work as specified/expected.
    • type.FeatureFlaw: Some functionality missing from a feature delivered in v1.4 in a way that the feature becomes less useful to the intended target user for normal usage. i.e., the feature is not 'complete'. In other words, an acceptance testing bug that falls within the scope of v1.4 features. These issues are counted against the 'depth and completeness' of the feature.
    • type.DocumentationBug: A flaw in the documentation that can potentially affect the user e.g., a missing step, a wrong instruction, typos that affect users

    At the end of the project each student is required to submit a Project Portfolio Page.

    PPP Objectives

    • For you to use (e.g. in your resume) as a well-documented data point of your SE experience
    • For evaluators to use as a data point to evaluate your project contributions

    PPP Sections to include

    • Overview: A short overview of your product to provide some context to the reader. The opening 1-2 sentences may be reused by all team members. If your product overview extends beyond 1-2 sentences, the remainder should be written by yourself.
    • Summary of Contributions --Suggested items to include:
      • Code contributed: Give a link to your code on tP Code Dashboard. The link is available in the Project List Page -- linked to the icon under your photo.
      • Features implemented: A summary of the features you implemented. If you implemented multiple features, you are recommended to indicate which one is the biggest feature.
      • Other contributions:
        • Contributions to project management e.g., setting up project tools, managing releases, managing issue tracker etc.
        • Evidence of helping others e.g. responses you posted in our forum, bugs you reported in other team's products,
        • Evidence of technical leadership e.g. sharing useful information in the forum

    Keep in mind that your feature(s) will be evaluated for depth, completeness, and effort. Use the PPP to convince evaluator how good those aspects of your features are.
    It is fine if you want to directly explain each of those aspects of your features in the PPP i.e., how deep the feature is, why it is complete, how hard it was to implement.

    • [Optional] Contributions to the User Guide: Reproduce the parts in the User Guide that you wrote. This can include features you implemented as well as features you propose to implement.
      The purpose of allowing you to include proposed features is to provide you more flexibility to show your documentation skills. e.g. you can bring in a proposed feature just to give you an opportunity to use a UML diagram type not used by the actual features.

    • [Optional] Contributions to the Developer Guide: Reproduce the parts in the Developer Guide that you wrote. Ensure there is enough content to evaluate your technical documentation skills and UML modelling skills. You can include descriptions of your design/implementations, possible alternatives, pros and cons of alternatives, etc.

    • [Optional] If you plan to use the PPP in your Resume, you can also include your SE work outside of the module (will not be graded).

    PPP Format

    PPP Page Limit

    Content Recommended Hard Limit
    Overview + Summary of contributions 0.5-1 2
    [Optional] Contributions to the User Guide 1-3
    [Optional] Contributions to the Developer Guide 3-6
    • The page limits given above are after converting to PDF format. The actual amount of content you require is actually less than what these numbers suggest because the HTML → PDF conversion adds a lot of spacing around content.

    PE-D After the session

    • We'll transfer the relevant bug reports to your repo over the weekend. Once you have received the bug reports for your product, it is up to you to decide whether you will act on reported issues before the final submission v1.4. For some issues, the correct decision could be to reject or postpone to a version beyond v1.4.
    • If you have received bug reports that don't seem to be about your project, do let us know ASAP (email the prof).
    • You can navigate to the original bug report (via the back-link provided in the bug report given to you) and post in that issue thread to communicate with the tester who reported the bug e.g. to ask for more info, etc. However, the tester is not obliged to respond.
      • Do not argue with the bug reporter to try to convince that person that your way is correct/better. If at all, you can gently explain the rationale for the current behavior but do not waste time getting involved in long arguments. If you think the suggestion/bug is unreasonable, just thank the reporter for their view and discontinue to discussion.

    Deliverable: Practical Exam (PE)

    PE Overview

    Objectives:

    • The primary objective of the PE is to increase the rigor of project grading. Assessing most aspects of the project involves an element subjectivity. As the project counts for 45% of the final grade, it is not prudent to rely on evaluations of tutors alone as there can be significant variations between how different tutors assess projects. That is why we collect more data points via the PE so as to minimize the chance of your project being affected by evaluator-bias.
    • PE is also used to evaluate your manual testing skills, product evaluation skills, effort estimation skills etc.
    • Note that significant project components are not graded solely based on peer ratings. Rather, PE data are mostly used to cross-validate tutors' grades and identify cases that need further investigation. When peer inputs are used for grading, they are usually combined with tutor evaluations with appropriate weight for each. In some cases ratings from team members are given a higher weight compared to ratings from other peers, if that is appropriate.
    • Note that the PE is not a means of pitting you against each other. Developers and testers play for the same side; they need to push each other to improve the quality of their work -- not bring down each other.

    Grading:

    • Your performance in the practical exam will affect your final grade and your peers', as explained in Admin: Project Grading section.
    • As such, we have put in measures to identify and penalize insincere/random evaluations.
    • Also see:

    Relevant: [Admin tP Grading → Notes on how marks are calculated for PE ]

    Notes on how marks are calculated based on PE product testing
    • Of 3A and 3B above, the one you do better will be given a 70% weight and the other a 30% weight so that your total score is driven by your strengths rather than weaknesses.
    • Bugs rejected by the dev team, if the rejection is approved by the teaching team, will not be affect marks of the tester or the developer.
    • The penalty/credit for a bug varies based on,
      • The severity of the bug: severity.High > severity.Medium > severity.Low > severity.VeryLow
      • The type of the bug: type.FunctionalityBug > type.DocumentationBug > type.FeatureFlaw
    • The penalty for a bug is divided equally among assignees.
    • The developers are not penalized for the duplicate bug reports they received but the testers earn credit for the duplicate bug reports they submitted as long as the duplicates are not submitted by the same tester.
    • Obvious bugs earn less credit for the tester and slightly more penalty for the developer.
    • If the team you tested has a low bug count i.e., total bugs found by all testers is low, we will fall back on other means (e.g., performance in PE dry run) to calculate your marks for system/acceptance testing.
    • Your marks for developer testing depends on the bug density rather than total bug count. Here's an example:
      • n bugs found in your feature; it is a difficult feature consisting of lot of code → 4/5 marks
      • n bugs found in your feature; it is a small feature with a small amount of code → 1/5 marks
    • You don't need to find all bugs in the product to get full marks. For example, finding half of the bugs of that product or 4 bugs, whichever the lower, could earn you full marks.
    • Excessive incorrect downgrading/rejecting/ duplicate-flagging, if deemed an unethical attempt to game the system, may be penalized.

    PE Preparation

    • It's similar to,
    • Ensure that you have accepted the invitation to join the GitHub org used by the module. Go to https://github.com/nus-cs2103-AY1920S2 to accept the invitation.

    • Ensure you have access to a computer that is able to run module projects e.g. has the right Java version.

    • Download the latest CATcher and ensure you can run it on your computer.

    Issues created for PED and PE need to be in a precise format for our grading scripts to work. Incorrectly-formatted responses will have to discarded. Therefore, you are strongly recommended to use CATcher for PED and PE activities. If you want to give your response via GitHub instead, please get our permission first.

    • Create a public repo in your GitHub account with the following name:
      • PE Dry Run: ped
      • PE: pe
    • Enable its issue tracker and add the following labels to it (the label names should be precisely as given).

    Bug Severity labels:

    • severity.VeryLow : A flaw that is purely cosmetic and does not affect usage e.g., a typo/spacing/layout/color/font issues in the docs or the UI that doesn't affect usage.
    • severity.Low : A flaw that is unlikely to affect normal operations of the product. Appears only in very rare situations and causes a minor inconvenience only.
    • severity.Medium : A flaw that causes occasional inconvenience to some users but they can continue to use the product.
    • severity.High : A flaw that affects most users and causes major problems for users. i.e., makes the product almost unusable for most users.

    Type labels:

    • type.FunctionalityBug: A functionality does not work as specified/expected.
    • type.FeatureFlaw: Some functionality missing from a feature delivered in v1.4 in a way that the feature becomes less useful to the intended target user for normal usage. i.e., the feature is not 'complete'. In other words, an acceptance testing bug that falls within the scope of v1.4 features. These issues are counted against the 'depth and completeness' of the feature.
    • type.DocumentationBug: A flaw in the documentation that can potentially affect the user e.g., a missing step, a wrong instruction, typos that affect users
    • Have a good screen grab tool with annotation features so that you can quickly take a screenshot of a bug, annotate it, and post in the issue tracker.

      • You can use Ctrl+V to paste a picture from the clipboard into a text box in bug report.
    • Download the product to be tested after you have been notified of which team you have been allocated to test.

    • Charge your computer before coming to the session. The testing venue might not have enough charging points.

    PE Phase 1: Bug Reporting

    • When, where: Week 13 lecture
    PE Phase 1 - Part I Product Testing [60 minutes]
    • Test the product and report bugs as described below. You may report both product bugs and documentation bugs during this period.
    Testing instructions for PE and PE-D
    a) Launching the JAR file
    • Put the jar file in an empty folder.
    • Open a command window. Run the java -version command to ensure you are using Java 11.
    • Launch the jar file using the java -jar command (do not use double-clicking).
    • If the product doesn't work at all: If the product fails catastrophically e.g., cannot even launch, you can test the fallback team allocated to you. But in this case you must inform us immediately after the session so that we can send your bug reports to the correct team.
      • PE-D: Download the JAR file from their GitHub page
      • PE: 1. Download from LumiNUS all files submitted by the team (i.e. jar file, User Guide, Developer Guide, and Project Portfolio Pages) into an empty folder.
    b) What to test
    • PE Dry Run (at v1.3):
      • Test the product based on the User Guide (the UG is most likely accessible using the help command).
      • Do system testing first i.e., does the product work as specified by the documentation?. If there is time left, you can do acceptance testing as well i.e., does the product solve the problem it claims to solve?.
    • PE (at v1.4):
      • Test based on the Developer Guide (Appendix named Instructions for Manual Testing) and the User Guide. The testing instructions in the Developer Guide can provide you some guidance but if you follow those instructions strictly, you are unlikely to find many bugs. You can deviate from the instructions to probe areas that are more likely to have bugs.
      • As before, do both system testing and acceptance testing but give priority to system testing as system-testing bugs can earn you more credit.
    c) What bugs to report?
    • PE Dry Run (at v1.3): You may report functionality bugs, UG bugs, and feature flaws.

    Relevant: [Admin tP Grading → Functionality Bugs ]

    These are considered functionality bugs:
    Behavior differs from the User Guide
    A legitimate user behavior is not handled e.g. incorrect commands, extra parameters
    Behavior is not specified and differs from normal expectations e.g. error message does not match the error

    Relevant: [Admin tP Grading → Functionality Bugs ]

    These will be considered feature flaws:
    The feature does not solve the stated problem of the intended user i.e., the feature is 'incomplete'
    Hard-to-test features
    Features that don't fit well with the product
    Features that are not optimized enough for fast-typists or target users

    Relevant: [Admin tP Grading → Possible UG Bugs ]

    These are considered UG bugs (if they hinder the reader):
    Not enough visuals e.g., screenshots/diagrams
    The visuals are not well integrated to the explanation.
    The visuals are unnecessarily repetitive e.g., same visual repeated with minor changes.
    Not enough examples e.g., sample inputs/outputs.
    The explanation is too brief or unnecessarily long.
    The information is hard to understand for the target audience. e.g., using terms the reader might not know
    The document looks messy, or not well-formatted.

    • PE (at v1.4): In addition to the reporting functionality bugs, UG bugs, and feature flaws, there will be an extra time given specifically for reporting documentation bugs (i.e., UG and DG bugs).

    Relevant: [Admin tP Grading → Possible DG Bugs ]

    These are considered DG bugs (if they hinder the reader):

    These are considered UG bugs (if they hinder the reader):
    Not enough visuals e.g., screenshots/diagrams
    The visuals are not well integrated to the explanation.
    The visuals are unnecessarily repetitive e.g., same visual repeated with minor changes.
    Not enough examples e.g., sample inputs/outputs.
    The explanation is too brief or unnecessarily long.
    The information is hard to understand for the target audience. e.g., using terms the reader might not know
    The document looks messy, or not well-formatted.

    UML notation incorrect or not compliant with the notation covered in the module.
    Some other type of diagram used when a UML diagram would have worked just as well.
    The diagram used is not suitable for the purpose it is used.
    The diagram is too complicated.
    Excessive use of code e.g., a large chunk of code is cited when a smaller extract of would have sufficed.

    • About posting suggestions:
      • PE Dry Run (at v1.3): You can also post suggestions on how to improve the product. Be diplomatic when reporting bugs or suggesting improvements. For example, instead of criticising the current behavior, simply suggest alternatives to consider.
      • PE (at v1.4): Do not post suggestions. But if a feature is missing a critical functionality that makes the feature less useful to the intended user, it can be reported as a bug of type Type.FeatureFlaw.
    d) How to report bugs
    • Post bugs as you find them (i.e., do not wait to post all bugs at the end) because bug reports created/modified after the allocated time will not count.
    • Launch CATcher, and login to the correct profile:
      • PE Dry Run: CS2103/T PE Dry run
      • PE: CS2103/T PE
    • Post bugs using CATcher.
    • Post bug reports in the following repo you created earlier:
      • PE Dry Run: ped
      • PE: pe
    • The whole description of the bug should be in the issue description i.e., do not add comments to the issue.
    e) Bug report format
    • Each bug should be a separate issue.
    • Write good quality bug reports; poor quality or incorrect bug reports will not earn credit.
    • Use a descriptive title.
    • Give a good description of the bug with steps to reproduce and screenshots. If the receiving team cannot reproduce the bug, you will not be able to get credit for it.
    • Assign exactly one severity.* label to the bug report. Bug report without a severity label are considered severity.Low (lower severity bugs earn lower credit)

    Bug Severity labels:

    • severity.VeryLow : A flaw that is purely cosmetic and does not affect usage e.g., a typo/spacing/layout/color/font issues in the docs or the UI that doesn't affect usage.
    • severity.Low : A flaw that is unlikely to affect normal operations of the product. Appears only in very rare situations and causes a minor inconvenience only.
    • severity.Medium : A flaw that causes occasional inconvenience to some users but they can continue to use the product.
    • severity.High : A flaw that affects most users and causes major problems for users. i.e., makes the product almost unusable for most users.
    • Assign exactly one type.* label to the issue.

    Type labels:

    • type.FunctionalityBug: A functionality does not work as specified/expected.
    • type.FeatureFlaw: Some functionality missing from a feature delivered in v1.4 in a way that the feature becomes less useful to the intended target user for normal usage. i.e., the feature is not 'complete'. In other words, an acceptance testing bug that falls within the scope of v1.4 features. These issues are counted against the 'depth and completeness' of the feature.
    • type.DocumentationBug: A flaw in the documentation that can potentially affect the user e.g., a missing step, a wrong instruction, typos that affect users
    PE Phase 1 - Part II Evaluating Documents [20 minutes]
    • This slot is for reporting documentation bugs only. You may report bugs related to UG and DG.
    • For each bug reported, cite evidence and justify. For example, if you think the explanation of a feature is too brief, explain what information is missing and why the omission hinders the reader.

    Relevant: [Admin tP Grading → Possible UG Bugs ]

    These are considered UG bugs (if they hinder the reader):
    Not enough visuals e.g., screenshots/diagrams
    The visuals are not well integrated to the explanation.
    The visuals are unnecessarily repetitive e.g., same visual repeated with minor changes.
    Not enough examples e.g., sample inputs/outputs.
    The explanation is too brief or unnecessarily long.
    The information is hard to understand for the target audience. e.g., using terms the reader might not know
    The document looks messy, or not well-formatted.

    Relevant: [Admin tP Grading → Possible DG Bugs ]

    These are considered DG bugs (if they hinder the reader):

    These are considered UG bugs (if they hinder the reader):
    Not enough visuals e.g., screenshots/diagrams
    The visuals are not well integrated to the explanation.
    The visuals are unnecessarily repetitive e.g., same visual repeated with minor changes.
    Not enough examples e.g., sample inputs/outputs.
    The explanation is too brief or unnecessarily long.
    The information is hard to understand for the target audience. e.g., using terms the reader might not know
    The document looks messy, or not well-formatted.

    UML notation incorrect or not compliant with the notation covered in the module.
    Some other type of diagram used when a UML diagram would have worked just as well.
    The diagram used is not suitable for the purpose it is used.
    The diagram is too complicated.
    Excessive use of code e.g., a large chunk of code is cited when a smaller extract of would have sufficed.

    PE Phase 1 - Part III Overall Evaluation [10 minutes]
    • Evaluate the following aspects. Submit via TEAMMATES. You are recommended to complete this during the PE session itself, but you have until the end of the day to submit (or revise) your submissions.

    A. Product Design []:

    Evaluate the product design based on the User Guide and the actual product behavior.

    Target user:

    • target user specified and appropriate: The target user is clearly specified, prefers typing over other modes of input, and not too general (should be narrowed to a specific user group with certain characteristics).
    • value specified and matching: The value offered by the product is clearly specified and matches the target user.
    • optimized for the target user: It feels like a fast typist can be more productive with the app, compared to an equivalent GUI app without a CLI.

    Value to the target user:

    B. Quality of user docs []:

    UG: Compared to AB3, the quality of this UG is,

    C. Quality of developer docs []:

    DG: similar to UG

    D. Feature Quality []:

    Quality: Compared to AB3, the quality of this product is,

    E. Amount of work []:

    Effort: Assume the effort required to create AB3 from scratch is 10 in a scale of 0 to 30. How much effort do you estimate the team put in for this project?

    • Do not give a high value just to be nice. Your responses will be used to evaluate your effort estimation skills.

    PE Phase 2: Developer Response

    This phase is for you to respond to the bug reports you received.

    Duration: The review period will start around 1 day after the PE (exact time to be announced) and will last until the following Tuesday midnight. However, you are recommended to finish this task ASAP, to minimize cutting into your exam preparation work.

    Bug reviewing is recommended to be done as a team as some of the decisions need team consensus.

    Instructions for Reviewing Bug Reports

    • Don't freak out if there are lot of bug reports. Many can be duplicates and some can be false positives. In any case, we anticipate that all of these products will have some bugs and our penalty for bugs is not harsh. Furthermore, it depends on the severity of the bug. Some bug may not even be penalized.
    • CATcher does not come with a UG, but the UI is fairly intuitive (there are tool tips too). Do post in the forum if you need any guidance with its usage.
    • Also note that CATcher hasn't been battle-tested for this phase, in particular, w.r.t. multiple team members editing the same issue concurrently. It is ideal if the team members get together and work through the issues together. If you think others might be editing the same issues at the same time, use the Sync button at the top to force-sync your view with the latest data from GitHub.
  • Launch CATcher, and login to the profile CS2103/T PE. It will show all the bugs assigned to your team, divided into three sections:
    1. Issues Pending Responses - Issues that your team has not processed yet.
    2. Issues Responded - Your job is to get all issues to the second category.
    3. Faulty Issues - e.g., Bugs marked as duplicates of each other, or causing circular duplicate relationships. Fix the problem given so that no issues remain in this category.
  • Respond to the bug reports shown.
  • Issues created for PE-D and PE need to be in a precise format for our grading scripts to work. Incorrectly-formatted responses will have to discarded. Therefore, you are strongly recommended to use CATcher for PE-D and PE activities. If you want to give your response via GitHub instead, please get our permission first.

    • Go to the dev-response issue tracker
    • Use tutorial.* and team.* labels to filter bug reports your team received.
    • Do not edit the subject or the description. Your response (if any) should be added as a comment.
    • Add a comment using the following exact template.
    # Team's Response
    {replace this with your response}
    

    ## Duplicate status (if any):

    Here is an example:

    # Team's Response
    

    Yes this is a bug. But it is a duplicate. * Changed the bug type because this is just a bug in the UG. * Lowered the severity because users can still use the feature.

    ## Duplicate status (if any): Duplicate of #67

    • Do not close the bug report after you are done processing it.
    • Use the exact Duplicate of #123 format to indicate duplicates.
    • There should be exactly one comment per issue. If there are multiple comments, the last one will be taken for processing.
    • If a bug seems to be for a different product (i.e. wrongly assigned to your team), let us know ASAP.
    • If the bug is reported multiple times,
      • Mark all copies EXCEPT one as duplicates of the one left out (let's call that one the original) using the duplicate tag.
      • If the duplicates have different severity levels, you should keep the one with the highest severity as the original. But you can downgrade the severity of the original or the duplicates.
      • For each group of duplicates, all duplicates should point to one original i.e., no multiple levels of duplicates, and no cyclical duplication relationships.
      • If the duplication status is eventually accepted, all duplicates will be assumed to have inherited the type.* and severity.* from the original.

    • Apply exactly one of these labels (if missing, we assign: response.Accepted)

    Response Labels:

    • response.Accepted: You accept it as a bug.
    • response.NotInScope: It is a valid issue but not something the team should be penalized for e.g., it was not related to features delivered in v1.4.
    • response.Rejected: What tester treated as a bug is in fact the expected behavior. You can reject bugs that you inherited from AB3.
    • response.CannotReproduce: You are unable to reproduce the behavior reported in the bug after multiple tries.
    • response.IssueUnclear: The issue description is not clear. Don't post comments asking the tester to give more info. The tester will not be able to see those comments because the bug reports are anonymous.
    • Apply exactly one of these labels (if missing, we assign: type.FunctionalityBug)

    Type labels:

    • type.FunctionalityBug: A functionality does not work as specified/expected.
    • type.FeatureFlaw: Some functionality missing from a feature delivered in v1.4 in a way that the feature becomes less useful to the intended target user for normal usage. i.e., the feature is not 'complete'. In other words, an acceptance testing bug that falls within the scope of v1.4 features. These issues are counted against the 'depth and completeness' of the feature.
    • type.DocumentationBug: A flaw in the documentation that can potentially affect the user e.g., a missing step, a wrong instruction, typos that affect users
    • If you disagree with the original severity assigned to the bug, you may change it to the correct level.

    Bug Severity labels:

    • severity.VeryLow : A flaw that is purely cosmetic and does not affect usage e.g., a typo/spacing/layout/color/font issues in the docs or the UI that doesn't affect usage.
    • severity.Low : A flaw that is unlikely to affect normal operations of the product. Appears only in very rare situations and causes a minor inconvenience only.
    • severity.Medium : A flaw that causes occasional inconvenience to some users but they can continue to use the product.
    • severity.High : A flaw that affects most users and causes major problems for users. i.e., makes the product almost unusable for most users.
    • Decide who should fix the bug. Use the Assignees field to assign the issue to that person(s). There is no need to actually fix the bug though. It's simply an indication/acceptance of responsibility. If there is no assignee, we will distribute the penalty for that bug (if any) among all team members.
      • If it is not easy to decide the assignee(s), we recommend (but not enforce) that the feature owner should be assigned bugs related to the feature, Reason: The feature owner should have defended the feature against bugs using automated tests and defensive coding techniques.

    • As far as possible, choose the correct type.*, severity.*, and assignees even for bugs you are not accepting or for bugs that are marked as duplicates. Reason: your non-acceptance or duplication status may be rejected in a later phase, in which case we need to grade it as an accepted/non-duplicate bug.

    • Justify your response. For all of the following cases, you must add a comment justifying your stance. Testers will get to respond to all those cases and will be double-checked by the teaching team in later phases. Indiscriminate/unreasonable dev/tester responses, if deemed as a case of trying to game the system, will be penalized.

      • downgrading severity
      • non-acceptance of a bug
      • changing the bug type
      • non-obvious duplicate

    PE Phase 3: Tester Response

    • In this phase you will get to state whether you agree or disagree with the dev response to the bugs you reported. If a bug reported has been subjected to any of the below by the receiving team, you can record your objections and the reason for the objection.
      • not accepted
      • severity downgraded
      • bug type changed
    • As before, consider carefully before you object to a team's response. If many of your objections were overruled by the teaching team later, you will lose marks for not being able to evaluate a bug report properly.
    • This phase is optional. If you do not respond to a dev response, we'll assume that you agree with it.
    • Deadline: Reading week Thursday 2359
    • Procedure:
    • When the phase has been announced as open, login to CATcher as usual (profile: CS2103/T PE).
    • For the issues listed, go to the details, and read the team's response.
    • If you disagree with any of the items listed, tick on the I disagree tick box and enter your justification for the disagreement, and click Save.

    Issues created for PE-D and PE need to be in a precise format for our grading scripts to work. Incorrectly-formatted responses will have to discarded. Therefore, you are strongly recommended to use CATcher for PE-D and PE activities. If you want to give your response via GitHub instead, please get our permission first.

    • When the phase has been announced as open, go to your pe repo.
    • For the issues listed, go to the details, and read the bot's comment containing the team's response.
    • If you disagree with any of the items listed in that comment, tick on the [ ] I disagree tick box. Edit the comment (do not add a new comment) to replace the [replace this with your reason] with your reasoning. Here is an example:
    Team chose ['response.IssueUnclear']
    
    - [x] I disagree
    
    **Reason for disagreement:** I think one can easily reproduce the problem by following the steps I gave.
    

    PE Phase 4: Tutor Moderation

    • In this phase tutors will look through all dev responses you objected to in the previous phase and decide on a final outcome.
    • In the unlikely we need your inputs, the tutor will contact you.

    tP: v1.4 [week 13] tP: Grading