This article is an update of the previous article "Debug a smart card application on Yosemite".

New logging scheme

Apple made some changes in the way system, and application, logs are managed.
Now to get the log messages from a process use the command:

$ log stream

You will get a lot of messages. It is possible to restrict the messages from a particular process using predicates.

Logs from com.apple.ifdreader

To get only the log messages from the process com.apple.ifdreader use:

$ log stream --predicate 'process == "com.apple.ifdreader"'

For example I get something like:

$ log stream --predicate 'process == "com.apple.ifdreader"'
Filtering the log data using "process == "com.apple.ifdreader""
Timestamp                       Thread     Type        Activity             PID    
2017-11-06 16:11:21.092392+0100 0x751b1    Error       0x0                  260    com.apple.ifdreader: (CryptoTokenKit) [com.apple.CryptoTokenKit.token] <private>: failed to transmit APDU
2017-11-06 16:11:22.173066+0100 0x74b9d    Error       0x800000000000af43   260    com.apple.ifdreader: (CryptoTokenKit) [com.apple.CryptoTokenKit.token] <private>: failed to transmit APDU
2017-11-06 16:11:23.253418+0100 0x734eb    Error       0x0                  260    com.apple.ifdreader: (CryptoTokenKit) [com.apple.CryptoTokenKit.token] <private>: failed to transmit APDU

APDU logging

It is still possible to log the APDU sent to the card and the response received. Use the same command as before:

$ sudo defaults write /Library/Preferences/com.apple.security.smartcard Logging -bool yes

You need to connect the reader after executing the above command to get the results.

For example I now get something like:

$ log stream --predicate 'process == "com.apple.ifdreader"'
Filtering the log data using "process == "com.apple.ifdreader""
Timestamp                       Thread     Type        Activity             PID    
2017-11-06 16:13:58.514542+0100 0x7868a    Activity    0x8000000000000511   260    com.apple.ifdreader: (CoreFoundation) Loading Preferences From System CFPrefsD
2017-11-06 16:13:58.514929+0100 0x7868a    Activity    0x8000000000000512   260    com.apple.ifdreader: (CoreFoundation) Sending Updated Preferences to System CFPrefsD
2017-11-06 16:13:58.516120+0100 0x7868a    Default     0x0                  260    com.apple.ifdreader: (CryptoTokenKit) [com.apple.CryptoTokenKit.APDULog] logging slot Gemalto PC Twin Reader
2017-11-06 16:14:06.444894+0100 0x734eb    Default     0x0                  260    com.apple.ifdreader: (CryptoTokenKit) [com.apple.CryptoTokenKit.APDULog] card in
2017-11-06 16:14:06.508793+0100 0x734eb    Default     0x0                  260    com.apple.ifdreader: (CryptoTokenKit) [com.apple.CryptoTokenKit.APDULog] ATR: 3b 7f 96 00 00 80 31 80 65 b0 85 03 00 ef 12 0f fe 82 90 00
2017-11-06 16:14:06.541594+0100 0x76ff0    Default     0x800000000000afe6   260    com.apple.ifdreader: (CryptoTokenKit) [com.apple.CryptoTokenKit.APDULog] T=0
2017-11-06 16:14:06.542311+0100 0x76752    Default     0x800000000000afe6   260    com.apple.ifdreader: (CryptoTokenKit) [com.apple.CryptoTokenKit.APDULog] APDU -> 00 a4 04 00 0b a0 00 00 03 08 00 00 10 00 01 00
2017-11-06 16:14:06.560122+0100 0x76ff0    Default     0x800000000000afe6   260    com.apple.ifdreader: (CryptoTokenKit) [com.apple.CryptoTokenKit.APDULog] APDU <- 61 13
2017-11-06 16:14:06.568852+0100 0x76752    Default     0x0                  260    com.apple.ifdreader: (CryptoTokenKit) [com.apple.CryptoTokenKit.APDULog] APDU -> 00 a6 00 00 15
2017-11-06 16:14:07.649227+0100 0x734eb    Default     0x0                  260    com.apple.ifdreader: (CryptoTokenKit) [com.apple.CryptoTokenKit.APDULog] APDU FAILED
2017-11-06 16:14:07.649267+0100 0x734eb    Error       0x0                  260    com.apple.ifdreader: (CryptoTokenKit) [com.apple.CryptoTokenKit.token] <private>: failed to transmit APDU
2017-11-06 16:14:07.650895+0100 0x76ff0    Default     0x800000000000b133   260    com.apple.ifdreader: (CryptoTokenKit) [com.apple.CryptoTokenKit.APDULog] APDU -> 00 a4 04 00 0b a0 00 00 03 08 00 00 10 00 01 00
2017-11-06 16:14:08.729874+0100 0x734eb    Default     0x800000000000b133   260    com.apple.ifdreader: (CryptoTokenKit) [com.apple.CryptoTokenKit.APDULog] APDU FAILED
2017-11-06 16:14:08.729915+0100 0x734eb    Error       0x800000000000b133   260    com.apple.ifdreader: (CryptoTokenKit) [com.apple.CryptoTokenKit.token] <private>: failed to transmit APDU
2017-11-06 16:14:08.731128+0100 0x74b9d    Default     0x0                  260    com.apple.ifdreader: (CryptoTokenKit) [com.apple.CryptoTokenKit.APDULog] APDU -> 00 a6 00 00 15
2017-11-06 16:14:09.809998+0100 0x734eb    Default     0x0                  260    com.apple.ifdreader: (CryptoTokenKit) [com.apple.CryptoTokenKit.APDULog] APDU FAILED
2017-11-06 16:14:09.810069+0100 0x734eb    Error       0x0                  260    com.apple.ifdreader: (CryptoTokenKit) [com.apple.CryptoTokenKit.token] <private>: failed to transmit APDU
2017-11-06 16:14:11.670245+0100 0x7868a    Default     0x0                  260    com.apple.ifdreader: (CryptoTokenKit) [com.apple.CryptoTokenKit.APDULog] unpower

Private fields

Some fields are shown as <private> in the log. To see the real value you must use:

$ sudo log config --mode "private_data:on"

For example I now get something like:

$ log stream --predicate 'process == "com.apple.ifdreader"'
Filtering the log data using "process == "com.apple.ifdreader""
Timestamp                       Thread     Type        Activity             PID    
2017-11-06 16:28:06.270662+0100 0x8689a    Activity    0x8000000000000514   260    com.apple.ifdreader: (CoreFoundation) Loading Preferences From System CFPrefsD
2017-11-06 16:28:06.271009+0100 0x8689a    Activity    0x8000000000000515   260    com.apple.ifdreader: (CoreFoundation) Sending Updated Preferences to System CFPrefsD
2017-11-06 16:28:06.280244+0100 0x8689a    Default     0x0                  260    com.apple.ifdreader: (CryptoTokenKit) [com.apple.CryptoTokenKit.APDULog] logging slot Gemalto PC Twin Reader
2017-11-06 16:28:08.106965+0100 0x85613    Default     0x0                  260    com.apple.ifdreader: (CryptoTokenKit) [com.apple.CryptoTokenKit.APDULog] card in
2017-11-06 16:28:08.171131+0100 0x85613    Default     0x0                  260    com.apple.ifdreader: (CryptoTokenKit) [com.apple.CryptoTokenKit.APDULog] ATR: 3b 7f 96 00 00 80 31 80 65 b0 85 03 00 ef 12 0f fe 82 90 00
2017-11-06 16:28:08.197556+0100 0x86c91    Default     0x800000000000afef   260    com.apple.ifdreader: (CryptoTokenKit) [com.apple.CryptoTokenKit.APDULog] T=0
2017-11-06 16:28:08.198083+0100 0x86c91    Default     0x800000000000afef   260    com.apple.ifdreader: (CryptoTokenKit) [com.apple.CryptoTokenKit.APDULog] APDU -> 00 a4 04 00 0b a0 00 00 03 08 00 00 10 00 01 00
2017-11-06 16:28:08.214878+0100 0x86c92    Default     0x800000000000afef   260    com.apple.ifdreader: (CryptoTokenKit) [com.apple.CryptoTokenKit.APDULog] APDU <- 61 13
2017-11-06 16:28:08.220709+0100 0x86c92    Default     0x0                  260    com.apple.ifdreader: (CryptoTokenKit) [com.apple.CryptoTokenKit.APDULog] APDU -> 00 a6 00 00 15
2017-11-06 16:28:09.301451+0100 0x7c9d3    Default     0x0                  260    com.apple.ifdreader: (CryptoTokenKit) [com.apple.CryptoTokenKit.APDULog] APDU FAILED
2017-11-06 16:28:09.301494+0100 0x7c9d3    Error       0x0                  260    com.apple.ifdreader: (CryptoTokenKit) [com.apple.CryptoTokenKit.token] Gemalto PC Twin Reader: failed to transmit APDU
2017-11-06 16:28:09.302846+0100 0x86c91    Default     0x800000000000b693   260    com.apple.ifdreader: (CryptoTokenKit) [com.apple.CryptoTokenKit.APDULog] APDU -> 00 a4 04 00 0b a0 00 00 03 08 00 00 10 00 01 00
2017-11-06 16:28:10.381888+0100 0x82669    Default     0x800000000000b693   260    com.apple.ifdreader: (CryptoTokenKit) [com.apple.CryptoTokenKit.APDULog] APDU FAILED
2017-11-06 16:28:10.381933+0100 0x82669    Error       0x800000000000b693   260    com.apple.ifdreader: (CryptoTokenKit) [com.apple.CryptoTokenKit.token] Gemalto PC Twin Reader: failed to transmit APDU
2017-11-06 16:28:10.383481+0100 0x7c9d3    Default     0x0                  260    com.apple.ifdreader: (CryptoTokenKit) [com.apple.CryptoTokenKit.APDULog] APDU -> 00 a6 00 00 15
2017-11-06 16:28:11.462582+0100 0x82669    Default     0x0                  260    com.apple.ifdreader: (CryptoTokenKit) [com.apple.CryptoTokenKit.APDULog] APDU FAILED
2017-11-06 16:28:11.462616+0100 0x82669    Error       0x0                  260    com.apple.ifdreader: (CryptoTokenKit) [com.apple.CryptoTokenKit.token] Gemalto PC Twin Reader: failed to transmit APDU
2017-11-06 16:28:13.333227+0100 0x85613    Default     0x0                  260    com.apple.ifdreader: (CryptoTokenKit) [com.apple.CryptoTokenKit.APDULog] unpower

More debug

You can get even more details by adding the --debug argument.

For example, just by connecting the smart card reader, I now get something like:

$ log stream --predicate 'process == "com.apple.ifdreader"' --debug
Filtering the log data using "process == "com.apple.ifdreader""
Timestamp                       Thread     Type        Activity             PID    
2017-11-06 16:39:11.456379+0100 0x91fc6    Debug       0x0                  11944  com.apple.ifdreader: [com.apple.CryptoTokenKit.smartcard] deviceRemovalHandler invoked (entryId=4294970878)
2017-11-06 16:39:11.456411+0100 0x91fc6    Debug       0x0                  11944  com.apple.ifdreader: [com.apple.CryptoTokenKit.smartcard] installed device removal notification
2017-11-06 16:39:11.456533+0100 0x91fc6    Debug       0x0                  11944  com.apple.ifdreader: [com.apple.CryptoTokenKit.smartcard] new device arrival: 08e6:3437 14400000 (entryId=4294970878)
2017-11-06 16:39:11.470462+0100 0x91fc6    Debug       0x0                  11944  com.apple.ifdreader: [com.apple.CryptoTokenKit.smartcard] bundle loaded: /usr/libexec/SmartCardServices/drivers/ifd-ccid.bundle
2017-11-06 16:39:11.470519+0100 0x91fc6    Debug       0x0                  11944  com.apple.ifdreader: [com.apple.CryptoTokenKit.smartcard] found bundle for device, resolved entryId=4294970878 to deviceName='Gemalto PC Twin Reader'
2017-11-06 16:39:11.470708+0100 0x92267    Debug       0x0                  11944  com.apple.ifdreader: [com.apple.CryptoTokenKit.smartcard] -> IFDHCreateChannelByName(00000000, 'Gemalto PC Twin Reader')
2017-11-06 16:39:11.778142+0100 0x92267    Debug       0x0                  11944  com.apple.ifdreader: [com.apple.CryptoTokenKit.smartcard] <- IFDHCreateChannelByName() = 0
2017-11-06 16:39:11.778188+0100 0x92267    Debug       0x0                  11944  com.apple.ifdreader: [com.apple.CryptoTokenKit.smartcard] -> IFDHGetCapabilities(00000000, TAG_IFD_THREAD_SAFE)
2017-11-06 16:39:11.778203+0100 0x92267    Debug       0x0                  11944  com.apple.ifdreader: [com.apple.CryptoTokenKit.smartcard] <- IFDHGetCapabilities() = 0 (0)
2017-11-06 16:39:11.778297+0100 0x92267    Debug       0x0                  11944  com.apple.ifdreader: [com.apple.CryptoTokenKit.smartcard] -> IFDHGetCapabilities(00000000, SCARD_ATTR_MAXINPUT)
2017-11-06 16:39:11.778323+0100 0x92267    Debug       0x0                  11944  com.apple.ifdreader: [com.apple.CryptoTokenKit.smartcard] <- IFDHGetCapabilities() = 0 (261)
2017-11-06 16:39:11.778495+0100 0x92267    Debug       0x0                  11944  com.apple.ifdreader: [com.apple.CryptoTokenKit.smartcard] -> IFDHControl_v3(00000000, code=1107299656, in=(null))
2017-11-06 16:39:11.778570+0100 0x92267    Debug       0x0                  11944  com.apple.ifdreader: [com.apple.CryptoTokenKit.smartcard] <- IFDHControl(out=<12044233 0012>) = 0
2017-11-06 16:39:11.778590+0100 0x92267    Debug       0x0                  11944  com.apple.ifdreader: [com.apple.CryptoTokenKit.smartcard] -> IFDHGetCapabilities(00000000, TAG_IFD_SLOTS_NUMBER)
2017-11-06 16:39:11.778605+0100 0x92267    Debug       0x0                  11944  com.apple.ifdreader: [com.apple.CryptoTokenKit.smartcard] <- IFDHGetCapabilities() = 0, (1)
2017-11-06 16:39:11.778657+0100 0x92267    Debug       0x0                  11944  com.apple.ifdreader: (CryptoTokenKit) [com.apple.CryptoTokenKit.token] setupWithName:'Gemalto PC Twin Reader'
2017-11-06 16:39:11.779628+0100 0x92267    Debug       0x0                  11944  com.apple.ifdreader: (CryptoTokenKit) [com.apple.CryptoTokenKit.token] port for pm notifications registered
2017-11-06 16:39:11.781182+0100 0x92267    Debug       0x0                  11944  com.apple.ifdreader: (CryptoTokenKit) [com.apple.CryptoTokenKit.token] initWithName:'Gemalto PC Twin Reader' successfully registered
2017-11-06 16:39:11.781648+0100 0x92267    Activity    0x800000000000bb00   11944  com.apple.ifdreader: (CoreFoundation) Loading Preferences From System CFPrefsD
2017-11-06 16:39:11.781704+0100 0x92284    Debug       0x0                  11944  com.apple.ifdreader: (CryptoTokenKit) [com.apple.CryptoTokenKit.token] Gemalto PC Twin Reader: new client connection established
2017-11-06 16:39:11.781847+0100 0x9228d    Debug       0x0                  11944  com.apple.ifdreader: (CryptoTokenKit) [com.apple.CryptoTokenKit.token] Gemalto PC Twin Reader: slot was set up
2017-11-06 16:39:11.781997+0100 0x92267    Debug       0x800000000000bb00   11944  com.apple.ifdreader: (CoreFoundation) [com.apple.defaults.User Defaults] CFPrefsPlistSource<0x7fd450c0d910> (Domain: com.apple.security.smartcard, User: kCFPreferencesAnyUser, ByHost: Yes, Container: (null)) loaded: a new base plist and no additional changes from the base plist
2017-11-06 16:39:11.782060+0100 0x92267    Activity    0x800000000000bb01   11944  com.apple.ifdreader: (CoreFoundation) Sending Updated Preferences to System CFPrefsD
2017-11-06 16:39:11.782193+0100 0x92267    Debug       0x0                  11944  com.apple.ifdreader: (CoreFoundation) [com.apple.defaults.User Defaults] CFPrefsPlistSource<0x7fd450c0d910> (Domain: com.apple.security.smartcard, User: kCFPreferencesAnyUser, ByHost: Yes, Container: (null)) is waiting for writes to complete
2017-11-06 16:39:11.782204+0100 0x9228d    Debug       0x0                  11944  com.apple.ifdreader: (CryptoTokenKit) [com.apple.CryptoTokenKit.token] Gemalto PC Twin Reader: new client connection established
2017-11-06 16:39:11.782291+0100 0x9228d    Debug       0x0                  11944  com.apple.ifdreader: (CryptoTokenKit) [com.apple.CryptoTokenKit.token] Gemalto PC Twin Reader: slot was set up
2017-11-06 16:39:11.783202+0100 0x91fce    Debug       0x0                  11944  com.apple.ifdreader: (CryptoTokenKit) [com.apple.CryptoTokenKit.token] Gemalto PC Twin Reader: new client connection established
2017-11-06 16:39:11.783284+0100 0x9228d    Debug       0x0                  11944  com.apple.ifdreader: (CryptoTokenKit) [com.apple.CryptoTokenKit.token] Gemalto PC Twin Reader: slot was set up
2017-11-06 16:39:11.790768+0100 0x92267    Default     0x0                  11944  com.apple.ifdreader: (CryptoTokenKit) [com.apple.CryptoTokenKit.APDULog] logging slot Gemalto PC Twin Reader
2017-11-06 16:39:12.306587+0100 0x9228d    Debug       0x0                  11944  com.apple.ifdreader: (CryptoTokenKit) [com.apple.CryptoTokenKit.token] Gemalto PC Twin Reader: new client connection established
2017-11-06 16:39:12.306724+0100 0x92284    Debug       0x0                  11944  com.apple.ifdreader: (CryptoTokenKit) [com.apple.CryptoTokenKit.token] Gemalto PC Twin Reader: slot was set up
2017-11-06 16:39:12.307550+0100 0x91fce    Debug       0x0                  11944  com.apple.ifdreader: (CryptoTokenKit) [com.apple.CryptoTokenKit.token] Gemalto PC Twin Reader: new client connection established
2017-11-06 16:39:12.307663+0100 0x92267    Debug       0x0                  11944  com.apple.ifdreader: (CryptoTokenKit) [com.apple.CryptoTokenKit.token] Gemalto PC Twin Reader: slot was set up
[...]

Resize your terminal

The logs lines are very long. I suggest you to resize your Terminal window to be very large, at least as large as the log lines.

Console application

You can also use the Console application (/Applications/Utilities/Console.app). It may be easier to use if you prefer a graphical application instead of the Terminal and the command line.

I have not yet found a way to have the equivalent of --debug with the Console application.

Conclusion

Apple provides nice tools to get useful debug messages. Since some applications can be very verbose logs messages are not all stored in a file like /var/log/system.log. You need to use the log stream command.

The log command has many other options that you can explore.

I just released a new version of PyKCS11, a Python wrapper above the PKCS#11 API.

Changes:
1.4.4 - October 2017, Ludovic Rousseau

• getAttributeValue(): handle CKR_ARGUMENTS_BAD error
• seedRandom: fix the seed conversion
• Add vendor errors support to PyKCS11Error
• samples/getinfo & dumpit: list only slots with a token present by default
• run_test: add support of OpenSC PKCS#11 spy
• ckbytelist: update __repr__()
• include tests files in the archive
• dumpit: display the error if getAttributeValue() fails
• some minor improvements

Source code available on:

• github https://github.com/LudovicRousseau/PyKCS11
• pypi https://pypi.python.org/pypi/pykcs11
• sourceforge.net https://sourceforge.net/projects/pkcs11wrap/files/pykcs11/pykcs11-1.4.4/

I just released a version 1.4.28 of libccid the Free Software CCID class smart card reader driver.

Changes:
1.4.28 - 11 October 2017, Ludovic Rousseau

• Add support of
• Athena IDProtect Flash
• Elatec TWN4/B1.06/CPF3.05/S1SC1.32/P (Beta 3)
• HID Global OMNIKEY 5122 Dual
• HID Global OMNIKEY 5122 Smartcard Reader
• IIT E.Key Crystal-1
• KRONEGGER Micro Core Platform
• KRONEGGER NFC blue Reader Platform
• Ledger Nano S
• REINER SCT cyberJack RFID standard
• REINER SCT cyberJack one
• SAFETRUST SABRE SCR
• SafeNet eToken 5300
• Unicept GmbH AirID USB Dongle
• Watchdata USB Key
• mCore SCard-Reader
• Disabled readers
• Jinmuyu Electronics Co., Ltd. MR800
• Fix non-pinpad HID global devices
• udev rules:
• allow rule overwrite
• Disable USB autosuspend on C3PO LTC31 v1 reader
• Some minor improvements

In "PCSC sample in Java" I presented the javax.smartcardio wrapper to access smart cards from Java.

Problem

One big problem with the Oracle implementation is that only one PC/SC context is created and used for all the javax.smartcardio calls.

The pcsc-lite documentation says for SCardEstablishContext():

Each thread of an application shall use its own SCARDCONTEXT, unless calling SCardCancel(), which MUST be called with the same context as the context used to call SCardGetStatusChange().

The effect is that you can get dead locks in your Java application if you uses smartcardio in different threads.

Solution

One solution can be to use an alternative implementation of javax.smartcardio. One is available at intarsys smartcard-io. It uses the BSD 3-clauses license.

The differences compared to the Oracle implementation are documented in javax.smartcardio Provider:

The library comes with an alternative javax.smartcardio provider. There are a couple of things to consider:

• Intended differences
  • dedicated PCSC context for terminals, terminal and card
  • waitForChange(timeout) semantics improved(?), state change is reset even in case of timeout
  • reader insertion is handled, too
  • no finalizer for card!

Conclusion

I wrote this blog article to document the problem and so that people can find the solution by themselves.

macOS High Sierra (macOS 10.13) is now available since 25th September, 2017.

API Differences between 10.12 and 10.13

The differences are listed in the developer page macOS Sierra 10.13. The page only documents big changes. No changes related to smart card are listed.

PC/SC

Since Yosemite (10.10) the PC/SC layer is no more a fork of pcsc-lite. So comparing versions with pcsc-lite is useless.

$ cat /System/Library/Frameworks/PCSC.framework/Versions/A/Resources/version.plist
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
 <key>BuildAliasOf</key>
 <string>CryptoTokenKit</string>
 <key>BuildVersion</key>
 <string>3</string>
 <key>CFBundleShortVersionString</key>
 <string>8.0</string>
 <key>CFBundleVersion</key>
 <string>1</string>
 <key>ProjectName</key>
 <string>SmartCardServices</string>
 <key>SourceVersion</key>
 <string>281001001000000</string>
</dict>
</plist>

The BuildVersion moved from 65 in Sierra 10.12.0 to 3 in High Sierra 10.13.0. I guess this number is not a good indicator since it is decreasing.

The SourceVersion moved from 196001003000000 in Sierra 10.12.0 to 281001001000000 in High Sierra 10.13.0. I have no idea how to parse or use this information.

PC/SC Bugs fixed

These bugs were found in El Capitan or latter and are now fixed in Sierra:

1. SCARD_W_RESET_CARD not returned by SCardTransmit()
2. PC/SC SCardTransmit() silently truncates the smart card response (bug #30868184)
3. Command "security smartcards token -d" command does not work as documented? (bug #31010575)

Some (minor) bugs reported on El Capitan are still present in High Sierra. I updated the page "OS X El Capitan and smart cards: known bugs".

CryptoTokenKit

CryptoTokenKit is the native smart card API since the complete rewrite in macOS Yosemite 10.10 (OS X Yosemite BETA and smart cards status).

$ strings /System/Library/Frameworks/CryptoTokenKit.framework/CryptoTokenKit | grep BuildRoot
/BuildRoot/Library/Caches/com.apple.xbs/Sources/CryptoTokenKit/CryptoTokenKit-281.1.1/CryptoTokenKit/TKToken.m
/BuildRoot/Library/Caches/com.apple.xbs/Sources/CryptoTokenKit/CryptoTokenKit-281.1.1/CryptoTokenKit/TKSmartCard.m
/BuildRoot/Library/Caches/com.apple.xbs/Sources/CryptoTokenKit/CryptoTokenKit-281.1.1/CryptoTokenKit/TKTokenSession.m

In High Sierra CryptoTokenKit source code is at version 281.1.1. In Sierra it was at version 196.60.1. Since the source code is not available I can't write much more than that.

Card events

It is not easy to compare the state of CyptoTokenKit since the source code is not public. Nonetheless I could find changes in the com.apple.ifdreader process (this process loads and use the IFDHandler, i.e. the smart card reader driver like the CCID driver).

It looks like Apple has worked on moving from active polling to eventing to manage smart card events (card insertion and removal). I reported it as a feature request: OS X El Capitan missing feature: add support of TAG_IFD_POLLING_THREAD_WITH_TIMEOUT.

I compared the result of the strings(1) Unix command on the binary from Sierra and the binary from High Sierra. In High Sierra new symbols are available:

$ strings /System/Library/CryptoTokenKit/com.apple.ifdreader.slotd/Contents/MacOS/com.apple.ifdreader | grep -i Poll
getPollingFunction:
getStopPollingFunction:
isPollingThreadKillable:
setupPolling
getPollingFunction
polling:
stopPolling
_pollingStarted
_pollingThread
PollingTimeout
Failed to create polling thread: %d
'IFDHPolling' failed %ld
%{public}@: got sleep request, stop polling card, close the channel

They are all new strings in High Sierra except the last one that was already present in Sierra.

Since my bug #24009313 was closed as a duplicate of #17534485 I don't know if the bug is fixed, or not, in High Sierra. This will need some debug logs from a driver. The Terminal command "log stream --debug | grep CryptoTokenKit" gives a lot of details regarding the smart card activity but not enough to know how the card events are generated.

CCID driver

Driver version 1.4.27. Sierra had: 1.4.24 in 10.12.0 and 1.4.25 in 10.12.6.

$ grep -A 1 CFBundleShortVersionString /usr/libexec/SmartCardServices/drivers/ifd-ccid.bundle/Contents/Info.plist
 <key>CFBundleShortVersionString</key>
 <string>1.4.27</string>

You can have a look at the CCID README file to know what changes between version 1.4.25 and version 1.4.27.

Note that the CCID driver version 1.4.27 provided in macOS High Sierra is the latest version available (as I write this blog). version 1.4.27 has been released in May 2017 (4 months ago only).

Conclusion

You can compare with the status I made for Sierra in "macOS Sierra and smart cards status".

It looks like High Sierra has not seen many changes regarding smart card.

The API proposed by Apple to use a smart card and a smart card reader is CryptoTokenKit. I already wrote about this API in:

• macOS Sierra and CryptoTokenKit API
• "PC/SC" sample in Objective-C (synchronous)
• PCSC sample in Objective-C

CryptoTokenKit is an equivalent of the PC/SC API defined by the PC/SC workgroup and implemented by Microsoft in Windows and by pcsc-lite for Unixes. The PC/SC API is also known as WinSCard.

Pinpad reader

The idea of a pinpad reader it so submit the user secret PIN code to the card without entering it on the computer. The PIN is entered on the smart card reader and is sent directly to the smart card inserted in the smart card reader. The computer (PC, Mac, whatever) never has access to the PIN code.

A pinpad reader is often used with banking applications. Even if your system is compromised the PIN is safe.

You can have a list of pinpad readers at my Reader selection page.

I will use a Gemalto Ezio Bluetooth reader in USB mode for the demo.

TKSmartCardUserInteractionForPINOperation

Apple provides a way to use a pinpad reader using CryptoTokenKit with the class TKSmartCardUserInteractionForPINOperation.

In the example I will use the subclass TKSmartCardUserInteractionForSecurePINVerification and the method userInteractionForSecurePINVerificationWithPINFormat:APDU:PINByteOffset:.

The parameter PINFormat of class TKSmartCardPINFormat offers services similar to what can be found in PC/SC version 2 Part 10 for using a pinpad reader but with a better interface design.

Source code

#import <Foundation/Foundation.h>
#import <CryptoTokenKit/CryptoTokenKit.h>

int main(int argc, const char * argv[]) {
    TKSmartCardSlotManager * mngr;
    mngr = [TKSmartCardSlotManager defaultManager];

    // Use the first reader/slot found
    if ([mngr.slotNames count] == 0)
    {
        NSLog(@"No reader found");
        return -1;
    }

    NSString *slotName = (NSString *)mngr.slotNames[0];
    NSLog(@"slotName: %@", slotName);

    dispatch_semaphore_t sem = dispatch_semaphore_create(0);

    // connect to the slot
    [mngr getSlotWithName:slotName reply:^(TKSmartCardSlot *slot)
     {
         // connect to the card
         TKSmartCard *card = [slot makeSmartCard];
         if (nil == card)
         {
             NSLog(@"No card found");

             // signals end of getSlotWithName block
             dispatch_semaphore_signal(sem);

             return;
         }

         // begin a session
         [card beginSessionWithReply:^(BOOL success, NSError *error)
          {
              if (success)
              {
                  NSData *response;
                  UInt16 sw;
                  TKSmartCardPINFormat *PINFormat;
                  TKSmartCardUserInteractionForSecurePINVerification *userInter;
                  NSData *APDUTemplate;

                  // explicitly set the CLA byte even if 0 is already the default value
                  card.cla = 0x00;

                  // send select applet APDU
                  uint8_t aid[] = {0xA0, 0x00, 0x00, 0x00, 0x18, 0xFF, 0x01};
                  NSData *data = [NSData dataWithBytes:aid length:sizeof aid];

                  response = [card sendIns:0xA4 p1:0x04 p2:0x00 data:data le:nil sw:&sw error:&error];

                  if (nil == response)
                  {
                      NSLog(@"sendIns error: %@", error);
                      [card endSession];

                      // signals end of beginSessionWithReply block
                      dispatch_semaphore_signal(sem);

                      return;
                  }

                  NSLog(@"Response select applet: %@ 0x%04X", response, sw);

                  const UInt8 template[] = {card.cla, 0x20, 0x00, 0x80, 0x08, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff};
                  APDUTemplate = [NSData dataWithBytes:template length:sizeof(template)];
                  PINFormat = [[TKSmartCardPINFormat alloc] init];
                  PINFormat.PINBitOffset = 0;

                  // VerifyPIN
                  data = [NSData dataWithBytes:template length:sizeof template];
                  userInter = [card userInteractionForSecurePINVerificationWithPINFormat:PINFormat APDU:data PINByteOffset:0];

                  if (nil == userInter)
                  {
                      NSLog(@"userInteractionForSecurePINVerificationWithPINFormat returned nil. Are you using a pinpad reader?");

                      [card endSession];

                      // signals end of beginSessionWithReply block
                      dispatch_semaphore_signal(sem);
                  }
                  else
                  {
                      NSLog(@"Enter the PIN on the pinpad");
                      [userInter runWithReply:^(BOOL success, NSError *error)
                       {
                           if (success)
                           {
                               NSLog(@"success");

                               // give some time to the reader to display a message before the next APDU
                               sleep(1);

                               NSLog(@"resultData: %@", [userInter resultData]);
                               NSLog(@"resultSW: %04X", [userInter resultSW]);

                               UInt16 sw;

                               // send PIN dump
                               uint8_t param[] = {0x09};
                               NSData *data = [NSData dataWithBytes:param length:sizeof param];
                            
                               NSData *response = [card sendIns:0x40 p1:0x00 p2:0x00 data:data le:@0 sw:&sw error:&error];
                            
                               if (nil == response)
                                   NSLog(@"sendIns error: %@", error);
                               else
                                   NSLog(@"Response PIN dump: %@ 0x%04X", response, sw);
                            
                               [card endSession];
                            
                               // signals end of beginSessionWithReply block
                               dispatch_semaphore_signal(sem);
                           }
                           else
                           {
                               NSLog(@"Error: %@", error);
                            
                               [card endSession];
                            
                               // signals end of beginSessionWithReply block
                               dispatch_semaphore_signal(sem);
                           }
                       }];
                  }
              }
              else
              {
                  NSLog(@"Session error: %@", error);
                
                  // signals end of getSlotWithName block
                  dispatch_semaphore_signal(sem);
              }
          }];
     }];

    // wait for the asynchronous blocks to finish
    dispatch_semaphore_wait(sem, DISPATCH_TIME_FOREVER);
}

Output

slotName: Gemalto Ezio Shield
 Response select applet: <> 0x9000
 Enter the PIN on the pinpad
 success
 resultData: <>
 resultSW: 9000
 Response PIN dump: <00200080 08313233 34353637 38> 0x9000

At the beginning

Application started

PIN code entered in the reader

PIN code validated by the card

This last picture is out of focus. I am sorry for that. Note that it is not easy to hold the smartphone, validate on the pinpad and take the picture with only two hands.

Comments

Source code auto documented

I will not comment the code here. Comments are already present in the source code.

Entitlements

Do not forget to create an .entitlements file in Xcode (enable App Sandbox). Then add the com.apple.security.smartcard property and set it to YES.

If you do not do that access to the CryptoTokenKit API will be denied and your output will be something like:

[smartcard] ctk: connecting to slot registration server failed
 No reader found
 [smartcard] connection to slot registration server failed

Without a pinpad reader

If you do not use a reader that support PIN verification using a pinpad then the method userInteractionForSecurePINVerificationWithPINFormat: will return nil instead of a TKSmartCardUserInteractionForSecurePINVerification object.

Be careful to check the returned value.

Test applet

I use a smart card with a specific test applet. The Java applet source code is available.

From the execution Output you can see the line:

Response PIN dump: <00200080 08313233 34353637 38> 0x9000

This is a special debug command I use to check what exactly the reader has sent to the card. The PIN may have many different formats and padding. Here you see the PIN "12345678" (Oops, it is no more a secret) has been sent as the ASCII codes for "1" (0x31), "2" (0x32), etc.

Your may need a different coding or padding. Have a look at the different TKSmartCardPINFormat parameters like charset, minPINLength, maxPINLength, PINJustification, PINBitOffset, etc.

Asynchronous runWithReply:

The runWithReply: method is asynchronous. So be careful to not end the card session before the user has entered its PIN. In the code I use a semaphore but you can use something else, or nothing, depending on your application.

Non-CCID readers?

Because of a bug in, I think, CryptoTokenKit I was not able to use the Gemalto Ezio Bluetooth reader in Bluetooth mode. The reader is not detected as a pinpad reader. This reader works fine with the PC/SC API on macOS and the pinpad feature can be used in Bluetooth mode.

My guess is that CryptoTokenKit detects that a reader is a pinpad reader by reading the bPINSupport byte directly from the CCID USB descriptor instead of using the driver IFDHControl(CM_IOCTL_GET_FEATURE_REQUEST, ...) call to check if FEATURE_VERIFY_PIN_DIRECT is supported. So only CCID pinpad readers may be supported.

I opened a bug at Apple "CryptoTokenKit does not detect my Bluetooth smart card reader as a pinpad reader" #34648641.

Source code and blog licence

If you do plan to reuse (part of) my code please read the blog licence bellow and be sure it is OK for you to conform with it, or contact me. See also My blog messages license.

Conclusion

I do not know many applications that use CryptoTokenKit to interact with a smart card. When I was debugging my code I searched for examples of use of TKSmartCardUserInteractionForPINOperation but could not find any.

I would not be surprised if my code is the first public example of CryptoTokenKit to use a pinpad reader.

Article from the series "ATR statistics".

TA3 - Specific to T after T from 0 to 14 in TDi–1

The ISO 7816-3 specification is not public. So I can't copy/paste part of the text. I will use Wikipedia instead.

For T = 1: maximum block size the card can receive. Encodes IFSC.
If T = 15: supported supply voltages and low power modes

TA3	#	%
	1379	66.55 %
0xFE	388	18.73 %
0xC7	109	5.26 %
0xC3	35	1.69 %
0x80	24	1.16 %
0x20	19	0.92 %
0x43	14	0.68 %
0x3C	9	0.43 %
0x47	8	0.39 %
0x86	7	0.34 %
0xA0	7	0.34 %
0x00	6	0.29 %
0x03	6	0.29 %
0x50	6	0.29 %
0x42	5	0.24 %
0xC6	5	0.24 %
0x40	4	0.19 %
0x60	4	0.19 %
0x70	4	0.19 %
0x66	3	0.14 %
0x90	3	0.14 %
0xFC	3	0.14 %
0xFF	3	0.14 %
0x46	2	0.10 %
0x76	2	0.10 %
0xF0	2	0.10 %
0xFA	2	0.10 %
0xFB	2	0.10 %
0x07	1	0.05 %
0x26	1	0.05 %
0x3A	1	0.05 %
0x3D	1	0.05 %
0x52	1	0.05 %
0x64	1	0.05 %
0x7C	1	0.05 %
0x83	1	0.05 %
0x87	1	0.05 %
0xC2	1	0.05 %
0xEF	1	0.05 %

The interpretation of TA₃ depends on the protocol in use.

19% of ATRs have TA₃ = 0xFE :

• In most case the interpretation is that TA₃ gives the value of IFSC (maximum size of information field of blocks that can be received by the card) for T=1. One example is 3B 97 11 C0 FF B1 FE 35 1F 83 A5 05 01 01 02 A3 01 5F with IFSC = 0xFE or 254 bytes.
• For 2 ATRs AT₃ gives the supported supply voltages and low power modes for T=0. One example is 3B D9 18 00 C0 09 10 FE 54 59 46 4F 4E 45 00 00 00

IFSC

The IFSC repartition is as follows:

IFSC	#	%
254	366	75,78 %
128	21	4,35 %
32	19	3,93 %
60	9	1,86 %
134	7	1,45 %
160	7	1,45 %
80	6	1,24 %
0	5	1,04 %
64	4	0,83 %
96	4	0,83 %
112	4	0,83 %
66	3	0,62 %
102	3	0,62 %
144	3	0,62 %
252	3	0,62 %
255	3	0,62 %
70	2	0,41 %
118	2	0,41 %
240	2	0,41 %
250	2	0,41 %
251	2	0,41 %
38	1	0,21 %
58	1	0,21 %
82	1	0,21 %
100	1	0,21 %
124	1	0,21 %
239	1	0,21 %

With a logarithmic scale:


75% of ATRs with a IFSC defined in TA₃ (so T=1 cards only) defines IFSC = 254.

Supported supply voltages

It is also possible to get the repartition of the class:

Class	#	%
A 5V B 3V C 1.8V	114	49,57 %
A 5V B 3V	106	46,09 %
B 3V C 1.8V	5	2,17 %
B 3V	3	1,30 %
B 3V C 1.8V D RFU E RFU	2	0,87 %

50% of ATRs defining the class supports the 3 classes: A, B and C for 5 Volts, 3 Volts and 1.8 Volt.

46% of ATRs defining the class supports the classes A and B but not C.

2 cards declare the support of classes D and E that are RFU. They are 3B D9 18 00 C0 09 10 FE 54 59 46 4F 4E 45 00 00 00 and 3B DD 96 00 80 10 FE 80 31 80 63 01 FF C0 73 B3 21 1B 81 05. It may be a bug in the parsing since the class is defined only after T=15 protocol. But in the two examples the latest defined protocol is T=0. The ATRs may not be valid.

Article from the series "ATR statistics"

TD2 - Structural, encodes Y3 and T

The ISO 7816-3 specification is not public. So I can't copy/paste part of the text. I will use Wikipedia instead.

Refer to TD1 - Structural, encodes Y2 and T since the definition of TD₂ is identical to TD₁.

TD2	#	%
	1123	54.20 %
0x31	393	18.97 %
0x01	250	12.07 %
0x1F	178	8.59 %
0x71	50	2.41 %
0xB1	48	2.32 %
0x91	10	0.48 %
0x21	6	0.29 %
0x3F	5	0.24 %
0xF1	5	0.24 %
0x10	2	0.10 %
0x11	2	0.10 %

TD₂ (as the other TD_i bytes) is structural and indicates:

• How to interpret the other ATR bytes
• What communication protocol the card wants to use

For 54% of the ATRs no TD₂ is present. So no other TA₃, TB₃, TC₃ or TD₃ is present and no new protocol is defined so the protocol defined by TD₁ (if any) will be used.

For 19% of the ATRs TD₂ = 0x31. The high nibble is 0011b so TA₃, TB₃ are present and T=1 protocol is defined. One such ATR is 3B 82 81 31 76 43 C0 02 C5.

For 12% of ATRs TD₂ = 0x01. The high nibble is 0000b so no other TA₃, TB₃, TC3 or TD₃ is present and T=1 protocol is defined. One such ATR is 3B 84 80 01 00 00 90 00 95.

0.10% of cards have TD₂ = 0x10 and so (re)define the use of T=0. In fact what is important here is the high nibble of TD₂ 0001b that defines that TA₃ is present. One such ATR is 3B DD 96 00 80 10 FE 80 31 80 63 01 FF C0 73 B3 21 1B 81 05.

I will not document all the other cases. I let this exercise to the reader.

This is the second new PC/SC bug I report for macOS Sierra (first one was "macOS Sierra bug: SCardTransmit() silently truncates the card response").
I imagine this bug is also present in El Capitan and Yosemite but I have not checked.

SCardEndTransaction() ends all the transactions

SCardEndTransaction() does not work as it should, at least it does not work as the pcsc-lite version on GNU/Linux.
If you start a new PC/SC transaction inside a PC/SC transaction you expect the card to be available to another application only after all the transactions have ended.

For example if, in one application, you do:
A. SCardBeginTransaction()
  B. SCardBeginTransaction()
  C. SCardEndTransaction()
D. SCardEndTransaction()
You may expect that the card is available again for another application after the transaction is ended in step D.

What happens on macOS Sierra is that the transaction is ended at step C instead. So all the card APDU exchanges between steps C and D are not protected by the PC/SC transaction and can be mixed with APDU exchanges from any other applications.

It looks like PC/SC on macOS does not include a nested transaction counter. The first call to SCardEndTransaction() will end the transaction.

The sample code above is very simple and may seem stupid. But maybe the steps B and C are executed in another function of your application and then more hidden.

The macOS Sierra behaviour may produce very bad side effects. Imagine you use transactions to restrict access to the card private key after verifying the card PIN code. Then another application could also use the private key (between steps C and D) before the PIN is unverified just before step D.

See also

Apple bug report #33940052 "Smart card PC/SC transactions can't be nested"
Closed by Apple as a duplicate of #33752531.

Sample code

#include <stdio.h>
#include <stdlib.h>
#include <string.h>

#ifdef __APPLE__
#include <PCSC/winscard.h>
#include <PCSC/wintypes.h>
#else
#include <winscard.h>
#endif

#define pcsc_error(fct) printf(fct ": %s 0x%08lX\n", pcsc_stringify_error(err), (long)err)

int main(void)
{
    LPSTR mszReaders;
    DWORD err, cchReaders = 0;
    SCARDCONTEXT hContext = 0;
    SCARDHANDLE hCard = 0;

    err = SCardEstablishContext(SCARD_SCOPE_SYSTEM, NULL, NULL, &hContext);
    if (err != SCARD_S_SUCCESS) {
        pcsc_error("ScardEstablishedContext");
        return -1;
    }

    err = SCardListReaders(hContext, NULL, NULL, &cchReaders);
    if (err != 0) {
        pcsc_error("ScardListReaders");
        return -1;
    }
    mszReaders = calloc(cchReaders, sizeof(char));
    if (!mszReaders) {
        printf("calloc\n");
        return -1;
    }
    err = SCardListReaders(hContext,NULL, mszReaders, &cchReaders);
    if (err != SCARD_S_SUCCESS) {
        pcsc_error("ScardListReaders");
        return -1;
    }

    printf("Using reader: %s\n", mszReaders);

    DWORD dwActiveProtocol;
    printf("connect\n");
    err = SCardConnect(hContext, mszReaders, SCARD_SHARE_SHARED, SCARD_PROTOCOL_T0 | SCARD_PROTOCOL_T1, &hCard, &dwActiveProtocol);
    if (err != SCARD_S_SUCCESS) {
        pcsc_error("SCardConnect");
        return 0;
    }

    printf("transaction 1\n");
    err = SCardBeginTransaction(hCard);
    if (err != SCARD_S_SUCCESS)
        pcsc_error("SCardBeginTransaction");

    printf("transaction 2\n");
    err = SCardBeginTransaction(hCard);
    if (err != SCARD_S_SUCCESS)
        pcsc_error("SCardBeginTransaction");

    printf("release transaction 2\n");
    err = SCardEndTransaction(hCard, SCARD_LEAVE_CARD);
    if (err != SCARD_S_SUCCESS)
        pcsc_error("SCardBeginTransaction");

    printf("Start me again and press enter\n");
    getchar();

    printf("release transaction 1\n");
    err = SCardEndTransaction(hCard, SCARD_LEAVE_CARD);
    if (err != SCARD_S_SUCCESS)
        pcsc_error("SCardBeginTransaction");

    err = SCardDisconnect(hCard, SCARD_LEAVE_CARD);
    if (err != SCARD_S_SUCCESS)
        pcsc_error("SCardDisconnect");

    SCardReleaseContext(hContext);

    return 0;
}

Result (on Sierra)

$ CFLAGS="-framework PCSC" make main
cc -framework PCSC    main.c   -o main

First execution:

$ ./main_Mac
Using reader: Gemalto PC Twin Reader
connect
transaction 1
transaction 2
release transaction 2
Start me again and press enter

Second execution in another window:

$ ./main_Mac 
Using reader: Gemalto PC Twin Reader
connect
transaction 1
transaction 2
release transaction 2
Start me again and press enter

Expected result (on Debian)

$ CFLAGS=`pkg-config --cflags libpcsclite` LDFLAGS=`pkg-config --libs libpcsclite` make main
cc -pthread -I/usr/include/PCSC    -lpcsclite    main.c   -o main

First execution:

$ ./main_Linux 
Using reader: Gemalto PC Twin Reader 00 00
connect
transaction 1
transaction 2
release transaction 2
Start me again and press enter

Second execution in another window:

$ ./main_Linux 
Using reader: Gemalto PC Twin Reader 00 00
connect

Here you notice that the second execution is blocked at the "connect" step. Once you press "Enter" for the first execution then the transaction is released and the second execution can continue.

Known workaround

None. Do not use nested PC/SC transactions.

Update: 8th January 2018

The bug is fixed in macOS High Sierra 10.13.2.

I just released a new version of PyKCS11, a Python wrapper above the PKCS#11 API.

Changes:
1.4.3 - June 2017, Ludovic Rousseau

• Add support of CKM_RSA_PKCS_PSS mechanism
• fix CKM_AES_CBC issue with Python 3
• add Unitary Tests (make tests)
• add tox support (automate and standardize testing in Python)
• add coverage support (measuring code coverage of Python programs)
• add Travis-CI configuration (automatic build and tests)
• some minor improvements

Source code available on:

• pypi https://pypi.python.org/pypi/pykcs11
• sourceforge.net https://sourceforge.net/projects/pkcs11wrap/files/pykcs11/pykcs11-1.4.3/

Results available on:

• code coverage https://coveralls.io/github/LudovicRousseau/PyKCS11?branch=master
• Travis-CI https://travis-ci.org/LudovicRousseau/PyKCS11