如何在多个其他表中使用时删除记录

I'm making an application in C# and I'm using the EF Code First for my database-creation (for a SQL-Server database). I have a class "Address" which is used in several other classes. So several records can relate to the same Address-record. Is there an option in the EF where I can delete the Address-record when it is nowhere used anymore? Unless if I'm wrong, the CascadeOnDelete-option will remove the record once a certain record is deleted while others still relate to the Address-record. Also, it wouldn't be very useful to create a new Address-record for each record that relates to it, because most Address-record would be exactly the same (for example, a lot of Address-record would just contain the name of the same country or city).I'm making an application in C# and I'm using t