If you are just looking for the simple answer, I think this is it where MyApp is your application name and where that's the root namespace name for your application. You have to use the pack URI syntax, but it doesn't have to be that complicated to pull an icon out of your embedded resources.
<Window x:Class="MyApp.MainWindow"
xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
xmlns:d="http://schemas.microsoft.com/expression/blend/2008"
xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
mc:Ignorable="d"
Height="100"
Width="200"
Icon="pack://application:,,,/MyApp;component/Resources/small_icon.ico">