A Dictionary can support multiple readers concurrently, as long as the collection is not modified. Even so, enumerating through a collection is intrinsically not a thread-safe procedure. In the rare case where an enumeration contends with write accesses, the collection must be locked during the entire enumeration. To allow the collection to be accessed by multiple threads for reading and writing, you must implement your own synchronization.
This is what MSDN says.
I don't want to use ConcurrentDictionary if it is not necessary
I assume, if there is only one thread performing writer operations on Dictionary<T,K>, it is safe for other threads to perform simple reading operations(like TryGetValue not enumeration) at the same time without acquiring a lock, am I correct?