When the NA folded after the 1875 season, Borden signed a three-year contract with the Boston Red Caps. On April 22, 1876, Borden and the Red Caps were victorious in the first National League (NL) game ever played. Later that season, on May 23, he pitched a shutout, which some historians claim was the first no-hitter in Major League Baseball. Known for having an eccentric personality, he played under different surnames, such as Josephs and Nedrob, so as to disguise his involvement in baseball; his prominent family would have disapproved had they known. After he was released from the Red Caps as a player during the first season of his contract, he worked for a short period of time as their groundskeeper until he and the owner agreed to a buyout of the remainder of his contract.