You're right that it depends whether or not you're coming from a place of theism or not. I've studied this at the graduate level, and I'm well within the "normal range" for Christian ethics (and I'd argue, while still being mainstream, that morality apart from theism is a crapshoot).
That being said, I also don't buy into the doctrine of Original Sin.
In your example, if you flipped the switch, not knowing you were hurting people, but then you found out, wouldn't you feel bad (unless you're still sociopath)? That's the sign of immorality in this situation. But again, I'm appealing to a universal (theistic) sense of morality, so it may be that we'll always be talking past each other.