Availability (Voice) with Sprite Kit

Question

I am trying to add Voice Over accessibility support in a fixed-board puzzle. However, I am unable to get the UIAccessibilityElements .

Now I am redefining accessibilityElementAtIndex , accessibilityElementCount and indexOfAccessibilityElement in my SKScene.

They return an array of available elements as such:

 func loadAccessibleElements() { self.isAccessibilityElement = false let pieces = getAllPieces() accessibleElements.removeAll(keepCapacity: false) for piece in pieces { let element = UIAccessibilityElement(accessibilityContainer: self.usableView!) element.accessibilityFrame = piece.getAccessibilityFrame() element.accessibilityLabel = piece.getText() element.accessibilityTraits = UIAccessibilityTraitButton accessibleElements.append(element) } } 

Where the piece is a subclass of SKSpriteNode and getAccessibilityFrame is defined:

 func getAccessibilityFrame() -> CGRect { return parentView!.convertRect(frame, toView: nil) } 

Currently, one (wrong size) accessibility element seems to appear on the screen in the wrong place.

Can someone point me in the right direction?

Many thanks

EDIT:
I tried working with hack-ish by placing a UIView on an SKView with UIButton elements in the same place as SKSpriteNodes. However, accessibility still does not want to work. The view is loaded as such:

 func loadAccessibilityView() { view.isAccessibilityElement = false view.accessibilityElementsHidden = false skView.accessibilityElementsHidden = false let accessibleSubview = UIView(frame: view.frame) accessibleSubview.userInteractionEnabled = true accessibleSubview.isAccessibilityElement = false view.addSubview(accessibleSubview) view.bringSubviewToFront(accessibleSubview) let pieces = (skView.scene! as! GameScene).getAllPieces() for piece in pieces { let pieceButton = UIButton(frame: piece.getAccessibilityFrame()) pieceButton.isAccessibilityElement = true pieceButton.accessibilityElementsHidden = false pieceButton.accessibilityTraits = UIAccessibilityTraitButton pieceButton.setTitle(piece.getText(), forState: UIControlState.Normal) pieceButton.setBackgroundImage(UIImage(named: "blue-button"), forState: UIControlState.Normal) pieceButton.alpha = 0.2 pieceButton.accessibilityLabel = piece.getText() pieceButton.accessibilityFrame = pieceButton.frame pieceButton.addTarget(self, action: Selector("didTap:"), forControlEvents: UIControlEvents.TouchUpInside) accessibleSubview.addSubview(pieceButton) } UIAccessibilityPostNotification(UIAccessibilityScreenChangedNotification, nil) } 

The buttons are positioned correctly, but accessibility just doesn't work. Something seems to be stopping him from working.

Answer 1

I searched in vain for a description of how to implement VoiceOver in Swift using SpriteKit, so I finally figured out how to do it. Here is some working code that converts SKNode into an accessible button when added to the SKScene class:

 // Add the following code to a scene where you want to make the SKNode variable named "leave" an accessible button // leave must already be initialized and added as a child of the scene, or a child of other SKNodes in the scene // screenHeight must already be defined as the height of the device screen, in points // Accessibility private var accessibleElements: [UIAccessibilityElement] = [] private func nodeToDevicePointsFrame(node: SKNode) -> CGRect { // first convert from frame in SKNode to frame in SKScene coordinates var sceneFrame = node.frame sceneFrame.origin = node.scene!.convertPoint(node.frame.origin, fromNode: node.parent!) // convert frame from SKScene coordinates to device points // sprite kit scene origin is in lower left, accessibility device screen origin is at upper left // assumes scene is initialized using SKSceneScaleMode.Fill using dimensions same as device points var deviceFrame = sceneFrame deviceFrame.origin.y = CGFloat(screenHeight-1) - (sceneFrame.origin.y + sceneFrame.size.height) return deviceFrame } private func initAccessibility() { if accessibleElements.count == 0 { let accessibleLeave = UIAccessibilityElement(accessibilityContainer: self.view!) accessibleLeave.accessibilityFrame = nodeToDevicePointsFrame(leave) accessibleLeave.accessibilityTraits = UIAccessibilityTraitButton accessibleLeave.accessibilityLabel = "leave" // the accessible name of the button accessibleElements.append(accessibleLeave) } } override func didMoveToView(view: SKView) { self.isAccessibilityElement = false leave.isAccessibilityElement = true } override func willMoveFromView(view: SKView) { accessibleElements = [] } override func accessibilityElementCount() -> Int { initAccessibility() return accessibleElements.count } override func accessibilityElementAtIndex(index: Int) -> AnyObject? { initAccessibility() if (index < accessibleElements.count) { return accessibleElements[index] as AnyObject } else { return nil } } override func indexOfAccessibilityElement(element: AnyObject) -> Int { initAccessibility() return accessibleElements.indexOf(element as! UIAccessibilityElement)! } 

Answer 2

Availability frames are defined in the fixed physical coordinates of the screen , and not in the coordinates of the UIView, and the conversion between them is quite complicated.

The beginning of the device is the lower left position of the screen, with X up when the device is in landscape mode.

This is a painful transformation; I don’t know why Apple did it this way.