This is expected behavior. QnA works on the 'one question (and related, similar questions)' to one answer idea, and expects unique questions for the queries. The QnA documentation states:
A knowledge base consists of question and answer (QnA) sets. Each set has one answer and a set contains all the information associated with that answer. An answer can loosely resemble a database row or a data structure instance.
The required settings in a question-and-answer (QnA) set are:
- a question - text of user query, used to QnA Maker's machine-learning, to align with text of user's question with different wording but the same answer
- the answer - the set's answer is the response that's returned when a user query is matched with the associated question
Each set is represented by an ID.
The optional settings for a set include:
- Alternate forms of the question - this helps QnA Maker return the correct answer for a wider variety of question phrasings
- Metadata: Metadata are tags associated with a QnA pair and are represented as key-value pairs. Metadata tags are used to filter QnA pairs and limit the set over which query matching is performed.
- Multi-turn prompts, used to continue a multi-turn conversation
QnA Maker doesn't differentiate between the two questions because it isn't two questions. It's literally the same question with two different answers.
This particular case would be a good use of QnAMaker's multi-turn prompt feature, where, after the customer has put in the query 'The machine won't start', QnA can follow up with a prompt that says "Which machine did you mean? Machine A or Machine B", and whichever they choose leads to the correct answer. I would look into Multi-Turn Conversations for your knowledgebase.
