Configuration files can be moved from package to package and we should
be able to handle that. Previous implementation had limitation that it
handled only configuration files from package that were removed. This
commit removes this restriction.

Also note that there should be no way that we could steal configuration
from package requiring it as that would be collision between packages
and as such we would already failed at this point.