The direct answer would be, yes, it might be fine to rely on OAuth 2.0, but the most honest answer would be so dependent, and there would be more than one way to do this.
Scenario 1 - Web and API are independent
If your web application is completely independent and does not act as an API client, my personal desire would be to use the OAuth 2.0 / OpenID Connect stream to get user authentication, get a token confirming their identity, and then create a regular cookie session. When I talk about a regular session, I mean a session that is stored on the server side, and the cookie contains only a unique identifier for this session.
You can also go and save the actual token in a cookie and go stateless, but I honestly don’t see the benefits. In most cases, the web application will want to store enough information in the session so that the idea of storing all this in a cookie is simply bad. Yes, you can save the access token, and then still have the server-side session state associated with that token, but then what will you get from saving the actual token.
Scenario 2 - The Web is also an API Client
In this case, everything can happen differently. Judging by your question, your web application is based on ASP.NET MVC, so I assume that the API calls will be made on the server side.
I would still work on creating a regular session identifier in a cookie during authentication instead of saving the actual token, but more important than you have to decide what to do when the token expires , you can have a session as large as the token allows, and then force the user to repeat the entire authentication process in order to get a new token, but if your access tokens have a very long life (which is usually not approved and also not recommended), this is really bad for users.
My tendency to use the session identifier instead of the token itself is only with the tokens, if its functional requirement should not have a token available to the client, then do not do this, otherwise you will simply increase the impact in case of violation.
Personally, in this case, I would resort to OAuth support to update the token in order to be able to update the access token for the web application during a user session is active. Thus, you can easily implement a sliding session that will be active as long as the user at the same time uses access tokens with shorter lifetimes, which reduce the impact of the missed access token.
Here are some resources for further reading about cookies / sessions and token authentication that can help you get a few different sides to support your decision-making process: