LLM-Based Reporting Assistant for Agile Project Management

Publication Type:

Conference Paper

Source:

ACM-W Greek Chapter 8th Summit on Gender Equality in Computing, Patras, Greece (2026)

URL:

http://gec26.ceid.upatras.gr/papers/21.Plexida.pdf

Keywords:

Agile project management, entity-level LLM evaluation, Jira, Large Language Models, reference-free evaluation

Abstract:

We present an AI assistant that processes Jira data to generate concise sprint summaries, improving communication across Agile teams. The system is based on a hybrid two-layer architecture in which Layer 1 extracts verified metrics from Jira repositories to establish computable ground truth, and Layer 2 generates role- specific reports. We further introduce an evaluation framework based on text statistics and regular expressions (regex) to assess each report across seven independent dimensions, enabling comparison across different LLMs and sprint-size clusters. Experiments on real-world projects show that ticket count is the primary factor affecting report quality. Beyond summary generation, the proposed approach enables validation of report fidelity and can be integrated as a monitoring mechanism within Agile workflows, supporting both technical and business communication.