Hi!
Yes, you do not have to provide your tenants with access to the Quick Sight dashboard itself, and you are able to give users access to display specific dashboards to users.
The first question here is to understand if you have chosen anonymous or user-based embedding. The main difference here is that with anonymous embedding you will not have to add users to Quicksight, but they will not be able to create or own assets. However, RLS is supported for both of these use cases.
If you are using anonymous embeddings, you can set up tag-based rules to restrict access for anonymous users. For further information on how to set this up, you can look at this guide.
On the other hand, if you are using user-based embeddings (which it seems might be the case, since you mentioned that users are signing in to your portal) then you can map your sign-ins to a user in Quicksight. You can use STS to assume a role and grant access to the specified dashboard/analyses. Then you can use the GenerateEmbedUrlForRegisteredUser to get the URL for the dashboard that the user should see and present this. For more information on this method, you can refer to this Community Answer.
Hope this helps!