Boycott of literary and arts festivals is attempt to ‘gag society’, says culture secretary Lisa Nandy