Many researchers use infant and child mortality data compiled by the World Bank's World Development Indicators, by UNICEF's State of the World's Children, or, for child mortality rates only, by Ahmad, Lopez, and Inoue (2000). According to Ross (2006), the most transparent is UNICEF's (see page 866).
These international organizations now coordinate in producing infant and child mortality statistics, under the name of The UN Inter-agency Group for Child Mortality Estimation (IGME). See their collection of papers on the child mortality estimation.
Abouharb and Kimball (2007) introduce a dataset on annual infant mortality rates in each country for 1816-2002, by filling as many country-year cells as possible with infant mortality data from a variety of sources (I am not sure if this does not sacrifice the comparability across countries and years). The dataset and the codebook are available at www.prio.no/jpr/datasets (look for the last link for 2007 (vol. 44), no. 6). They avoid using the UN Demographic Yearbooks (which actually do provide annual data in its printed version, but not online) as much as possible. They keep the record on which data source is used, for each country-year observation. It turns out that 41 percent of observations after 1950, mainly developing countries, come from US Census Bureau's International Data Base. I am not sure why we should trust US Census Bureau more than the United Nations.
For poor countries, however, these data may be created by the interpolation of very few data points. See Qian (2015: 303).