I'm calculating a vision cone for my enemy and, if the player is in it, I try to check if there is a collider between them with Physics2D.Raycast, so as to make sure the enemy does not see the player through walls.
The walls that can't be seen through are from a mesh, and have a mesh collider attached - see this pic. They are also on a specific layer, to which I'm trying to limit my Raycast.
My code is pretty straight forward. if the enemy is facing the player, and the player is close enough, fire the raycast.
if (distanceToPlayer < viewDistance && dirToPlayer == dir) {
// player is in cone, check for collision
LayerMask lm = LayerMask.NameToLayer ("WallCollisions");
RaycastHit2D hit1 = Physics2D.Raycast(transform.position, (player.transform.position - transform.position).normalized, 1000, lm.value );
RaycastHit2D hit2 = Physics2D.Linecast (transform.position, player.transform.position);
return true;
}
When breaking on the return statement to inspect the hit, it always looks like this:
{UnityEngine.RaycastHit2D}
centroid: {(0.0, 0.0)}
collider: (null)
distance: 0
fraction: 0
normal: {(0.0, 0.0)}
point: {(0.0, 0.0)}
rigidbody: (null)
transform: (null)
Non-public members:
Which means nothing was found.
Also, note that in the pic, the guard is surrounded by the mesh, meaning that regardless of direction, there should be a hit within the distance specified..
I have seen a lot of people do byteshifting to their LayerMasks, but mimicing theirs doesn't help, and frankly I don't understand how it would.