- Course Prefix and Number: CS 480A7
- Course Title: Principles of Empirical Software Engineering
- Catalog Description:
Conducting empirical research on topics related to the software development process. Through a combination of lectures, research paper readings, and collaborative projects, students will learn how to plan and execute a software engineering research study, derive actionable conclusions for industry and researchers, and present their findings effectively.
- Prerequisites:
The course can be taken by both graduates and undergraduates interested in research. Prerequisite courses for undergraduates: (CS314 with a C or better) AND (STAT 301 with a C or better or STAT 315 with a C or better), as students need a certain level of maturity in software engineering processes and statistics concepts to understand the research content and methods.
- Course Learning Objectives (CLOs):
Formulate research questions in Software Engineering.
Design and conduct their own empirical studies in Software Engineering.
Use quantitative and qualitative research methods to analyze data related to the Software Development lifecycle.
Critically assess research methodologies and outcomes in Software Engineering.
Write and present a research paper on Empirical Software Engineering.
- Grading Scale and Policies:
- Research Project: 60%
Groups of 4, self-assigned.
Throughout the course, you will be provided with the description of the topic for designing an Empirical Software Engineering study and writing a short paper (maximum 4 pages) based on it.
All papers must be written using LaTeX, with Overleaf as the platform for collaboration.
The instructor will create Ovearleaf projects for each team and share them with GTAs and team members.
Each team member’s grade will be the same as the team grade, but it may be adjusted based on feedback from peers and the instructor.
If you plan to present or publish the data obtained from this study, it is important to consult your major professor (and possibly the IRB) to determine if IRB approval is necessary.
The projects and progress will be evaluated based on subjective criteria used by reviewers from Software Engineering conferences. The main factor for grading will be the extent to which the project meets its stated goals and addresses its research questions.
Details and due dates will be posted on Canvas as the semester progresses.
Deliveries will include a PDF generated from Overleaf, and a short report detailing each member’s contribution to the delivery.
Deliveries will be presented in class – presentation agenda randomly assigned by instructor.
Changes in the presentation agenda must be requested from TAs at least one week before and are not guaranteed to be done. The group is responsible for finding another group that would accept the change.
More details in class.
- Individual quizzes and assignments (15%)
All quizzes will be announced (no pop quizzes) and will be done in Canvas.
Assignments will have a due date to be delivered in Canvas.
The quiz with the lowest score will be dropped and not counted for the final grade (quiz 0 cannot be dropped and is not included in the group for dropping)
Cover the topics in the course content.
Quizzes will include multiple choice, true/false, fill-in-the-blanks, and short essay questions, among other types.
No Late turn-ins will be allowed.
- Exams (25%)
2 in-class mid-term exams and 1 final exam (Paper-based)
The mid-term exam will be done within the class time.
The mid-term exam with the lowest score will be dropped and not counted for the final grade. Final exam cannot be dropped.
There are no make-up exams.
Students who need Testing accommodation will need to use the SDC Accessible Testing Center, students will need to schedule their exams at least seven (7) days in advance
Mandatory final exam will be given during the scheduled final exam time.
Should be turned in within the final exam time.
Students who need Testing accommodation will need to use the SDC Accessible Testing Center, students will need to schedule their exams at least seven (7) days in advance
2 Mid-term Exams (10%)
Final Exam (15%)
A (90-100%), B+ (86-89%), B (82-85%), B- (78-81%), C (70-77%), D (60-69%), F (below 60%)
- Course Materials:
The course is based on content from multiple sources. Textbook purchases are not required, but the main references for the course content are the following books and research papers. Additional content will be provided from various sources, including more research papers, blog posts, videos, podcasts, slide decks, etc.
- Late assignments:
The due dates will always be at 11:59 pm MDT time. Between the due date and 48 hours after the due date, assignments will be penalized with 10%. Electronic submission is closed 48 hours after assignments are due; students who have not submitted programs receive an automatic zero on the assignment. “Emergencies” require that YOU contact the instructor as soon as possible. Students’ requests for late submissions after the due date may not be granted.
- Academic Integrity:
Adherence to the university’s academic integrity policy is expected. Academic dishonesty, including plagiarism, cheating, and misrepresentation, will not be tolerated.
- Accessibility:
Students requiring accommodations should contact the Student Disability Center to ensure that all needs are met.
- Class Participation:
Active participation in discussions and group activities is expected. Online discussions will be conducted via Canvas, and students are expected to engage respectfully and constructively.
- Communication Policy:
I use email for all communication and will respond as quickly as possible, usually within 48 hours. You can reach me via Teams, but I cannot guarantee a synchronous response. Put URGENT in the subject if you need me to expedite the request.
- Additional Resources:
Canvas for course materials and grades.
Access to R or Python and Overleaf (LaTeX) for quantitative analysis and research projects.
- CSU policies:
The page https://col.st/2FA2gLinks to an external site.Links to an external site. provides policies and resources to help with various challenges you may encounter. including, Canvas Information and Technical Support, Universal Design for Learning/Accommodation of Needs, Undocumented Student Support, Food Insecurity, Title IX/Interpersonal Violence, Religious Observances, CSU Principles of Community, Diversity and Inclusion, Student Parents/Guardians/Caregivers, Student Case Management, Mental Health and Wellness.
- Use of Generative AI:
We will run studies and write scientific papers in this course, and we want them to be accepted. Conferences, journals, and magazines now require that any content generated by Generative AI be referenced appropriately or acknowledged.
- We follow IEEE submission policy (linkLinks to an external site.) and the ACM authorship policy (linkLinks to an external site.).
- When using Generative AI to improve the quality of your text, be very careful. Even when using it for grammar enhancements, always double-check the output before incorporating it.
- While Generative AI is a powerful tool, it’s essential to critically evaluate the content it produces. AI-generated data can carry biases, results may be hallucinated, and references are many times incorrect. Moreover, submitting a paper with vague or unsound text greatly increases the risk of rejection.
- Whether you aim to publish to enhance your CV or improve your skills, I am here to help you refine your skills. Professionals who effectively use large language models (LLMs) as tools while critically assessing their outputs will thrive in their careers. You can only critique AI-generated content if you have the foundational knowledge to discern accuracy.
- So, keep in mind:
- References hallucination will be a reason to downgrade, as you used LLMs without checking.
- Critical thinking is key.
- Overreliance on LLMs is not the solution.
- You must fully understand the content you present. This includes being able to explain and defend it during discussions.
- In this course, I will guide you in leveraging LLMs for Software Engineering Research (yes, as a tool), particularly during experiments, while teaching you how to critically evaluate their outputs. It means that in some cases, we will deliberately use Generative AI for experiments and discuss these scenarios in class.
- Online students
Instead of group presentations, online students will send a recorded video following the same time restriction as in-person students (5min for delivery1, 8min for delivery2, and 10min for delivery3).
Exams will be done on a locked browser (Canvas) with both hands appearing on the video. Closed-book (and of course, closed-LLM) exam.
- Academic Integrity:
Adherence to the university’s academic integrity policy is expected. Academic dishonesty, including plagiarism, cheating, and misrepresentation, will not be tolerated.
- Accessibility:
Students requiring accommodations should contact the Student Disability Center to ensure that all needs are met.
- Class Participation:
Active participation in discussions and group activities is expected. Online discussions will be conducted via Canvas, and students are expected to engage respectfully and constructively.
- Communication Policy:
I use email for all communication and will respond as quickly as possible, usually within 48 hours. You can reach me via Teams, but I cannot guarantee a synchronous response. Put URGENT in the subject if you need me to expedite the request.
- Additional Resources:
Canvas for course materials and grades.
Access to R or Python and Overleaf (LaTeX) for quantitative analysis and research projects.
- CSU policies:
The page https://col.st/2FA2gLinks to an external site.Links to an external site. provides policies and resources to help with various challenges you may encounter. including, Canvas Information and Technical Support, Universal Design for Learning/Accommodation of Needs, Undocumented Student Support, Food Insecurity, Title IX/Interpersonal Violence, Religious Observances, CSU Principles of Community, Diversity and Inclusion, Student Parents/Guardians/Caregivers, Student Case Management, Mental Health and Wellness.
- Use of Generative AI:
We will run studies and write scientific papers in this course, and we want them to be accepted. Conferences, journals, and magazines now require that any content generated by Generative AI be referenced appropriately or acknowledged. We follow IEEE submission and the ACM authorship policies:
- When using Generative AI to improve the quality of your text, be very careful. Even when using it for grammar enhancements, always double-check the output before incorporating it.
- While Generative AI is a powerful tool, it’s essential to critically evaluate the content it produces. AI-generated data can carry biases, results may be hallucinated, and references are many times incorrect. Moreover, submitting a paper with vague or unsound text greatly increases the risk of rejection.
- Whether you aim to publish to enhance your CV or improve your skills, I am here to help you refine your skills. Professionals who effectively use large language models (LLMs) as tools while critically assessing their outputs will thrive in their careers. You can only critique AI-generated content if you have the foundational knowledge to discern accuracy.
- So, keep in mind:
- References hallucination will be a reason to downgrade, as you used LLMs without checking.
- Critical thinking is key, and overreliance on LLMs is not the solution for you to learn.
- You must fully understand the content you present. This includes being able to explain and defend it during discussions.
- In this course, I will guide you in leveraging LLMs for Software Engineering Research (yes, as a tool), particularly during experiments, while teaching you how to critically evaluate their outputs. It means that in some cases, we will deliberately use Generative AI for experiments and discuss these scenarios in class.
- Online students
- Instead of group presentations, online students will send a recorded video following the same time restriction as in-person students (5min for delivery1, 8min for delivery2, and 10min for delivery3).
- Exams will be done on a locked browser (Canvas) with both hands appearing on the video. Closed-book (and of course, closed-LLM) exam.
