Whilst working on the main app, I began wondering about the effectiveness of the prompts I was writing for the AI actions. Sure, I have a bit of experience with prompting coming from developer tools like Cursor but it’s all very finger in the air type stuff. That and because the interface hides the output of the prompts, I never directly interact with the LLM output itself, either.
So I wrote a little tool in a few hours to help with some prompt development. You have some input (denoted by variables in the tool) and want to test some different prompts out to see what gives the best results. It runs multiple prompts and then allows you to compare the output of them with each other, as well as comparing with the input variables. It’s probably aimed more towards people who are writing pre-baked prompts within apps than anything related to (the more common) AI chat type apps.
Obviously, none of this stuff is an exact science, so it’s kind of hard to home in on a “perfect” prompt. However, it is useful for finding out what prompts NOT to use, as well as micro-optimising token usage (note: I should add token counts to the UI).
Not sure if there are any better tools around already which do something similar - I didn’t bother to look because this was so quick to whip together. As a side note, I’ll say that the packaging situation for Dotnet MAUI has spooked me a little and I’m vaguely concerned about how releasing the bigger app will go. It caused me to spend some time looking into alternative GUI libraries though (also because the allure of a full rewrite is ever-present).
I’ll stick with MAUI for Aixaminator too (at least for now) because I read something on lobste.rs or hn about the power of persistence (or focus or something). I am pretty aware of the fact that MAUI doesn’t support Linux though. This is a problem (of unknown proportion) - especially because I am somewhat concerned with the fact that Aixaminator should be highly available (accessible?) as a byproduct of education needing to be accessible to as many people as possible.
Screenshots
Home screen
Configuring a prompt
Comparing outputs in the diff view