Hear Me Out: Solving unmet legal needs through conversational AI

We kicked the project off by conducting a comprehensive technical investigation to explore the capabilities of artificial intelligence in simplifying the legal complaint process. This crucial first step helped us define the software requirements needed to build an effective solution. We designed a prototype to assess the feasibility of integrating AI and performed initial user testing with people who had experienced access to justice issues to gather valuable feedback. This prototype served as a proof of concept, demonstrating the potential impact of AI on streamlining complaint navigation.

At the same time, we evaluated the feasibility of these new Gen AI solutions against more traditional approaches, such as decision tree pathways, to ensure we chose the most effective approach for guiding users through complex complaint scenarios.

Using this research as a foundation, we conducted a feature mapping workshop with NJP to transform the prototype into a plan for an AI product, road-mapping the development of the minimum viable product (MVP). This allowed us to prioritise features that deliver the most value to users and set the stage for the next phase.

To ensure the reliability and effectiveness of "Hear Me Out," we developed a custom testing framework capable of running bulk tests across various scenarios. This framework utilised automated and "human-in-the-loop" testing, allowing us to make real-time adjustments to prompts and Retrieval-Augmented Generation (RAG) settings and observe the outcomes. Through technical discovery, we decided on a largely RAG-based approach. By providing the AI agent with relevant documents to search through and extract context from, we achieved greater flexibility than extensive retraining and fine-tuning.

A number of other tasks were undertaken to deliver the solution:

• Refined Input Data: To enhance the AI's focus, non-essential sections of user factsheets were removed, minimising distraction and irrelevant information for the AI.
• Vector Database Creation: A specialised database indexed real-life examples from factsheets, enabling the AI to match user queries more accurately with relevant information.
• Summarised Content: ChatGPT was used to summarise factsheets, allowing the AI to process more comprehensive data without exceeding its context limits. Summarised content was reviewed and updated by experts as needed.
• Prompt Optimisation: The development of effective prompts required a careful balance between detail and adaptability, with continuous testing to ensure clear, context-aware AI responses.
• Ethical Safeguards: An offramp system directed users with complex needs or those at risk to appropriate human support services, ensuring ethical use and trust.
• User-Centric Development: A minimum viable product (MVP) content site was built using Webflow and a chat interface integrated with OpenAI, refined through user testing to align with users’ needs.

Through iterative refinement and direct end-user testing, we continuously adjusted the information architecture and content of Hear Me Out. This user-centric approach ensured that the product evolved in alignment with its users' needs and expectations.

We initiated a soft launch of Hear Me Out to monitor its performance in a real-world setting and gather additional user feedback. This approach allowed us to make any necessary adjustments before a full-scale rollout. We planned to gradually scale our marketing strategy to increase user engagement and collect valuable insights for future enhancements. By carefully evaluating the tool's impact and user reception, we positioned Hear Me Out for long-term success in empowering Australians in their pursuit of justice.