I believe the prominence of the site, the amount of text copied, the subject content, and the T&C of the site are some important factors you have to take into consideration.

Referencing is always good, even if you modified the text and/or just use it sparingly.