This question often comes up with people moving into the area. Your post office address is Irvington and your school system is Irvington but it is not part of the Village of Irvington so you do not have to pay Village taxes.
I recently listed a home at 33 Shaw Lane Irvington, NY which is considered part of in East Irvington, part of the unincorporated Town of Greenburgh. The benefits of this area are lower taxes because you don’t pay the Village tax. You also have access to the Town of Greenburgh pools and recreational facilities and which you would not have access to if you are part of the Village of Irvington. There are also great expanses of green space for you enjoyment.
East Irvington Nature Preserve is a great place to hike. It is just off Taxter Rd, here are a couple of links that give you a sense of the area and trails. There is also the Taxter Ridge Park Preserve in the area as well. It is over 200 acres of preserved open space.
This is a map of the Town of Greenburgh, you can see the unincorporated area is marked.
For your reference, here is a link for Town of Greenburgh parks, facilities and directions