[DCOS_OSS-2960] Apache Spark - example update for 1.10 Created: 08/Aug/17  Updated: 09/Nov/18  Resolved: 19/Dec/17

Status: Resolved
Project: DC/OS
Component/s: community
Affects Version/s: None
Fix Version/s: DC/OS 1.10.0

Type: Task Priority: Medium
Reporter: Judith Malnick (Inactive) Assignee: Unassigned
Resolution: Done  
Labels: example, project:dcoscommunity
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Epic Link: Update Examples for DC/OS 1.10



Comment by Kevin Sztern [ 13/Aug/17 ]

I'm on it!

In case you need to reach me, I'm "chewiebeardy" on the DC/OS slack.

I just started the fork on https://github.com/Chewie/examples

Comment by Judith Malnick (Inactive) [ 14/Aug/17 ]

Kevin Sztern beautiful thank you! Jörg Schad would you assign a shepherd to this one when you get a chance?

Comment by Chewie [ 16/Aug/17 ]

Update: I started updating the docs (https://github.com/Chewie/examples/tree/spark_1.10) and took the liberty of adding a few more info (like mentionning that the fine-grained mode is deprecated according to the Spark docs).

Unfortunately, the cloud provider we use at work has troubles at the moment and I'm unable to spin a 1.10 beta cluster to update the images and ensure that everything runs smoothly. I'll be able to do that as soon as we manage to spin a new cluster!



Comment by Kevin Sztern [ 18/Aug/17 ]

Update: work is for the most part done (https://github.com/Chewie/examples/tree/spark_1.10/spark/1.10), I improved the prerequisites and updated the screenshots.

There's still one issue that prevents me from issuing the PR: the `dcos spark log` doesn't output the logs for some reason, though they definitely exist when I look for them manually. I'm interested in your input whether that's intended behavior and the command must now be launched differently, or if there's a bug that should be fixed.


Comment by Kevin Sztern [ 11/Sep/17 ]

Update: The final issue was solved with the GA release of 1.10, so I created the PR: https://github.com/dcos/examples/pull/220


Comment by Elizabeth K. Joseph [ 19/Dec/17 ]

Thanks Kevin Sztern! I see this was merged, so I'm closing this ticket.

Comment by Judith Malnick (Inactive) [ 19/Dec/17 ]

Hi Kevin Sztern thanks for your great work on this! Please remember to email community-team@dcos.io with the following info so we can send you your swag!

  • Subject: 1.10 Example Testing - Service Name
  • Body:
    • Your name
    • JIRA task link
    • Your Twitter handle if you have one
    • Community Slack handle if you have one
    • Mailing address (we'll be sending you a little bit of swag )
    • Phone number
Generated at Wed May 18 08:14:03 CDT 2022 using JIRA 7.8.4#78004-sha1:5704c55c9196a87d91490cbb295eb482fa3e65cf.