HN Explorer

Show HN: Prompt Engineering Made Easy

23 points by MikahDang ↗ HN
Hey HN,

We've been hard at work on a tool that we believe will change the game for developers, data scientists, and anyone working with models that rely on textual prompts. I'm excited to introduce our new tool: Automated Prompt Engineering (APE).

Problem: As many of you know, how you phrase a prompt can significantly impact the results you get from models, especially with sophisticated language models. It often requires numerous iterations to hone in on the right prompt to obtain the desired response.

Solution: APE is designed to tackle this exact problem. With APE, you can: - Iterative Testing: Input your desired outcome, and APE will iteratively rephrase and test multiple prompts to achieve that outcome. - Optimization: APE can integrate with popular models and optimize prompts based on the model's feedback, ensuring the highest quality responses.

Features: Customization: Tailor the tool according to your domain-specific requirements. Model Integration: Seamless integration with popular NLP models and platforms.

Try it out: We've opened a limited beta for HN users. Get early access and let us know your feedback. Your insights will be invaluable in shaping the next iterations of APE.

Link to the beta: https://app.astadeus.com/write

21 comments

[ 2.6 ms ] story [ 48.3 ms ] thread
gus_massa
I tried

> How to make a banana salad.

I got a long text, most of it makes sense. But using bananas in a salad confused the AI.

> Begin by listing the essential ingredients for the banana salad, such as ripe bananas, diced avocado, sliced bell peppers, and crunchy nuts like almonds or cashews. Mention using organic produce and high-quality oils to ensure optimal taste and nutrition.

I think asking explicitly for the list of ingredients is useful, I should have thought about that. But some of the ingredients are strange, but I never ate a banana salad anyway. (sorry for that, but I always use banana for test.)

My question is how are you making this? Is this AI generated? How are you sure that the long "optimized" version is better?

MikahDang
Thank you for trying. Yes, this is AI-generated. We fine-tuned a special LLM with a huge dataset of prompts with a Monte Carlo algorithm to explore the solution space. We have evaluated every single prompt in our datasets before fine-tuning them with LLM metrics.
Ancapistani
So, you've created an LLM specifically for generating prompts for other LLMs? A meta-LLM?

That tickles my engineering receptors in a pleasing way (), but my financial side has questions. The most pressing: if this is successful and useful, why wouldn't the LLMs that you're targeting just replicate your creation and transparently put it between users and their own LLM?

In other words, what is your moat here?

MikahDang
This is a great question. Actually, I have two opinions about this: - It is not economically useful for foundational model providers to build this, or at least integrate it with their products, because it will cost them double. They would prefer RLHF and something intrinsic to their models. - Our dataset is proprietary. We have to collect and write a huge number of prompts to fine-tune it well. And our model is not 100% LLM, we use a version of the evolutionary algorithm to control the quality.

I would say we have a tiny moat of data and algorithm. But at the end of the day, it is about delivering something people find useful. If we could be replaced, let it be. But it works for now.

meiraleal
> The most pressing: if this is successful and useful, why wouldn't the LLMs that you're targeting just replicate your creation and transparently put it between users and their own LLM?

> In other words, what is your moat here?

He is using other people platforms to create a business, layers on top of LLMs won't have a moat, not even training the LLM is guaranteeing a moat. Why on earth everybody asks this on Show HN? The world doesn't need more big techs with moats, we need a healthy ecosystem where multiple players can work together without killing each other and become the only option. No moat, please.

RecycledEle
If you can make a better banana salad than the computer suggested, then and only then so you have the right to criticize the computer's output.
TMWNN
Related question: We are approaching one year since news articles began appearing on "prompt engineer" as a job title. How viable has "prompt engineering" been as a job description? I am not asking about the breathless articles discussing $300-400K compensation, per se, but on whether it seems like a viable career path going forward.
soultrees
From what I’ve seen there is definitely value in knowing how to get what to you want from these machines and especially how to not get what you don’t want, and every company using AI will need someone in charge of all the prompts the company uses, especially as they get more granular and quantitative, keeping a backlog of previous prompts used to write any internal or external comments, and being the main chopping head on the block for when it’s used in a way that gets the company sued etc. But the old way of prompt engineering has now evolved into a more scientific, testing multiple times in multiple models so I’d say the job is here to stay.
MikahDang
I think that is true. And I also think the future would be automated by the algorithm but the human will still be a significant part in deciding the outputs. Humans cannot be replaced 100% unless big corps create something truly AGI.
wodenokoto
We need to backpropagate from output string back through the network to generate the input string.
MikahDang
Thank you for your reply. Could you give us more details on that idea?
wodenokoto
It’s how the original image recognition obfuscation techniques worked. You’d update an image by backpropagating from the desired output category.

LLMs are much more complicated structures than imagenets, so I’m sure there are a lot of challenges involved.

Also I imagine the input (output?) might be complete ghibberish.

meiraleal
It seems there are some CORS issue? Using firefox:

Cross-Origin Request Blocked: The Same Origin Policy disallows reading the remote resource at https://apis.astadeus.com/autotext. (Reason: CORS header ‘Access-Control-Allow-Origin’ missing)

MikahDang
We would check this soon. Thank you for your feedback.