Question 1
No it is not required.
On server side you should add a behavior like this
<behavior name="SecureBehavior">
<serviceMetadata httpGetEnabled="true" />
<serviceCredentials>
<!--
The serviceCredentials behavior allows one to specify a custom validator for username/password combinations.
-->
<userNameAuthentication userNamePasswordValidationMode="Custom"
customUserNamePasswordValidatorType="[Your.Custom.WCFUserValidator], [AssemblyName]"/>
<!--
The serviceCredentials behavior allows one to define a service certificate.
A service certificate is used by a client to authenticate the service and provide message protection.
This configuration references the "localhost" certificate installed during the setup instructions.
-->
<serviceCertificate findValue="[certificateName]" storeLocation="LocalMachine" storeName="My" x509FindType="FindBySubjectName"/>
</serviceCredentials>
</behavior>
Then add the behavior to the server endPoint
<service name="[serviceName]" behaviorConfiguration="SecureBehavior">
<endpoint address="" binding="wsHttpBinding" bindingConfiguration="wsSecureConfig"
contract="[ContractName]" />
<endpoint address="/MEX" binding="mexHttpBinding" contract="IMetadataExchange" />
</service>
On client side you can set public portion of the service's certificate in config file something like this:
<endpoint address="http://..."
binding="wsHttpBinding"
contract="..."
name="serviceName">
<identity>
<certificate encodedValue="[Encoded Value]" />
</identity>
</endpoint>
The easy way to get the client configuration is by adding a service reference to your service in your client project through Visual Studio (Add Service Reference context menu). This will add a config file with a client end point ready to use.
Question 2
If you use Custom Authentication, the client credential type must by set to UserName. This enables the user name and password to be submitted to the service to preform authentication. And Yes you must to use a certificate.