Well, you can check if the message starts with your prefix or has been sent by a bot: if none of those, it means it's not a command nor a command response. All of that assuming the message is in that channel.
// ASSUMPTIONS:
// channel = your channel as a TextChannel
// prefix = your prefix as a string
// owner = you as a User
client.on('message', msg => {
if (msg.channel != channel || msg.author.bot || msg.content.startsWith(prefix)) return;
else msg.delete();
});
// if you want your messages to be ignored too:
client.on('message', msg => {
if (msg.channel != channel || msg.author.bot || msg.content.startsWith(prefix) || msg.author == owner) return;
else msg.delete();
});