OpenHermesPreferences: Dataset of ~1M AI preferences from teknium/OpenHermes-2.5 (huggingface.co)

The dataset can be used for training preference models or aligning language models through techniques like Direct Preference Optimization (DPO).