Updated! Updates are shown in quote text like this. Some scores are updated following app updates.
An Apps Experiment
Cross-posted from https://lemmy.world/post/18159531
Introduction
This is an experiment I performed out of curiosity, and I have a few big disclaimers at the bottom. Basically, I’ve seen a lot of comments recently about one app or another not displaying something right. Lemmy has been around for a while now and can no longer be considered an experimental platform.
Lemmy and the apps that people use to access the platform have become an important part of people’s lives. Whether you are checking the app weekly or daily, and whether you use it to stay up on the news or to stay connected to your hobby, it’s important that it works. I hope that this helps people to see the extent of the challenge, and encourages developers to improve their apps, too.
How I did it
I wanted to investigate objectively how accurately each app displays text of posts and comments using the standard Lemmy markdown. Markdown is a standard part of the Lemmy platform, but not all apps handle it the same. It is basically what gives text useful formatting.
I used the latest release of each app, but did not include pre-releases. I only included apps that have released an update in the last 6 months, which should include most apps in active development. I was unable to test iOS-exclusive apps, so they are not included either. In all, 16 apps met the inclusion criteria.
I also added Eternity, which is in active development, although it has not had a recent update. I was able to include several iOS apps thanks to testing from @[email protected] – Thanks, Jordan! This made for 20 apps that were tested.
Each app was rated in 5 categories: Text, Format, Spoilers, Links, and Images. I chose these mostly based on the wonderful Markdown Guide from @[email protected], which was posted about a year ago in [email protected] (here).
I checked whether each app correctly displayed each category, then took the overall average. Each category was weighted equally. Text includes italic, bold, strong, strikethrough, superscript, and subscript. Format includes block quotes, lists, code (block and inline), tables, and dividers. Spoilers includes display of hidden, expandable spoilers. Links includes external links, username links, and community links. Images included embedded images, image references, and inline images.
Thanks to input from others, I also added a test to see if lemmy hyperlinks opened in-app. There was a problem with using the SFFA Community Guide that caused some apps to be essentially penalized twice because there was formatting inside formatting, so I created this TEST POST to more clearly and fairly measure each app.
In each case, I checked whether the display was correct based on the rules for Lemmy Markdown, and consistent with the author’s intent. In cases where the app recognized the tag correctly but did not display it accurately, that was treated as a fail.
Results
Out of a possible perfect 10, 6 apps displayed all markdown correctly:
Alexandrite - 10.0
Connect - 10.0
Jerboa (Official Android client) - 10.0
Photon - 10.0
Summit - 10.0
Voyager - 10.0
Quiblr - 9.5
Arctic - 9.3
Interstellar - 9.1
Lemmuy-UI - 9.0
Thunder - 8.9
Tesseract - 8.6
mlmym - 8.0
Racoon - 7.6
Boost - 7.3
Eternity - 7.0
Lemmios - 6.9
Sync - 6.9
Lemmynade - 6.1
Avelon - 5.7
Disclaimers
Disclaimers
I Love Lemmy Apps (and their devs)
Lemmy apps devs work very hard, and invest a lot in the platform. Lemmy is better because they are doing the work that they do. Like, a LOT better. Everyone who uses the platform has to access it through one app or another. Apps are the face of the entire platform. Whether an app is a FOSS passion project, underwritten by a grant, or generating income through sales or ads, no one is getting rich by making their app. It is for the benefit of the community.
This is not meant to be a rating of the quality or functionality of any app. An app may have a high rating here but be missing other features that users want, or users may love an app that has a lower rating. This is just about how well apps handle markdown.
This is pretty unscientific
You’ll see my methodology above. I’m not a scientist. There is probably a much better way to do this, and I probably have biases in terms of how I went about it. I think it’s interesting and probably has some valuable information. If you think it’s interesting, let me know. If you think of a better way, PM me and I’d be happy to share what I have so you don’t have to start from scratch.
My only goal is to help the community
I do think that accurately displaying markdown should be a standard expectation of a finished app. I hope that devs use this as an opportunity to shore up the areas that are lagging, and that they have a set of standards to aim for.
I don’t have any Apple things
Sorry. This is just Android and Web review. If someone would like to see how iOS apps are doing, please reach out and I’ll share how we can work together to include them.
You must log in or register to comment.
I started on jerboa, but ended up moving to connect because of the bugs.
What bugs?
This was over a year ago now Jerboa may well be in a better state now. I can’t remember specific bugs but they were frequent and serious enough to frustrate me, a professional software tester, enough to move to a different app.
Is there a list of what each app failed? It would be nice for the devs to be able to see. I use Mlem, and there is about to be a new release rebuilding it from the ground up. Hopefully it will rate higher once that happens.
Yes, I’ve linked it in the post, and you can find the test post and detailed results.
Thanks. Interesting how the apps, even those that have lower scores, perform better than a web browser. Using Safari and Firefox (on a laptop) and both open your links in Lemmy.world instead of that thread on my instance. Neither recognize the user as anything other than text.
Odds are that’s Lemmy-UI. It should behave the same in any browser.
Update: connect, jerboa, boost, and sync all break on the following post with spoiler:
I have to use desktop mode here to make sense of the posts
So what’s the technological story here? I’m guessing lemmy itself uses a particular markdown parser that could probably be extracted and used in other contexts as it’s likely written in rust and should therefore be pretty portable without too much effort.
Are other apps just using whatever markdown parser is convenient to them? Is this something that the lemmy and threadiverse community could converge on? Even the fediverse as a whole where just about every platform other than mastodon supports writing in some for of markdown … feels like a pandoc like utility could go far.
I’m probably not the person to ask, to be honest. Lemmy as I understand it is the protocol that exchanges the information about posts, etc. The post content is stored and shared as plaintext, but Lemmy also has instructions about how a UI should interpret the text and serve it to the user.
Ideally, the same text should appear consistent across any UI. Obviously, some apps will use different fonts and colors and may interpret the style of an element differently.
Ideally, the same text should appear consistent across any UI. Obviously, some apps will use different fonts and colors and may interpret the style of an element differently.
Oh yea styling isn’t the issue here … it’s whether the markdown is correctly interpreted and rendered. AFAIU, lemmy doesn’t have any instructions about how to interpret the text, just some standard that they’ve chosen to use, along with their open source software for doing so (as they’ve built too clients, the default web UI and Jerboa).
Ouch, I use Boost and paid for ads free. Pls bring it up to 10.0.
Sync only got 6.9 but I have no complaints about the app
did
u
know
u
can
nest
spoilers?
dog pic
This displays incorrectly for me on Jerboa
:( It works on lemmy-ui/photon/alexandrite/voyager (maybe others too - these are just ones I’ve tested that work)
Why is your username color highlighted in voyager
I am the voyager dev!
Worth the effort for the good boy or girl.
Whoa that’s cool! It works in Thunder!
I did not know that. Works on Android Thunder.
hello fellow client dev
Not on Jerboa apparently.
Neat! I did not know that.
Awesome
It’s been a month since I’ve been able to post anything from my lemmy.world account using any app.
Weird, I use Boost all the time. Did you turn on 2FA or something? Maybe try removing and re-adding your account?
Yeah, I added 2FA. I will try disabling it to see if anything changes.
I might consider eternity abandonware now
1 year no updates
Eternity is in active development. It was sleeping for a while, but @[email protected] has confirmed that it will see a new release soon.
One note on Jerboa, at least for me gifs don’t seem to play when embedded in comments. Otherwise 10/10 for me.
I love Jerboa, it most closely resembles RiF from the beforetimes.
I did not test different media types - but maybe in the future!
I don’t understand why there isn’t a “markdown library” of some sort that software developers can just use in their app. I haven’t looked too deep into this, but it has always seemed to me that every app must individually implement markdown display. Why?
The problem isn’t that there are no libraries out there that parse Markdown. There are, in fact, plenty for all different languages. The issue is that every site has its own flavor of it. Lemmy does it one way, GitHub another, and something else does it completely differently yet again.
It is, unfortunately, kind of a mess.
Because markdown has committed the worst of old programming sins. It has no standard.
However I’m pretty sure that Lemmy has a standard so there’s not really much excuse.
Lemmy documentation references CommonMark so I’m assuming that is the accepted standard, plus a few Lemmy specific things.
Funny how the competition between charger standards in the alt text was eventually solved with, you guessed it, another standard, called USB-C.
I am fairly sure that the comic isn’t that old. So I think usb-c is what it’s alluding to.
that the comic isn’t that old
That comic recently became a teenager. The first USB-C specifications weren’t published until 2014.
Lol. I wish XKCD showed date published.
Explain xkcd shows the date published: 2011-07-20
The comic is now a teenager.
P.S. Pedantic rule on the capitalization of xkcd:
The preferred form is “xkcd”, all lower-case. In formal contexts where a lowercase word shouldn’t start a sentence, “XKCD” is an okay alternative. “Xkcd” is frowned upon.
Isn’t the base markdown standardized?
It’s just that so many flavors advertise themselves as markdown+ flavor?
only sort of.
this is the original document defining markdown, and you’ll notice it doesn’t really specify a lot of the things that have compatibility issues across different markdown processors, along with allowing arbitrary html which really depends on where you’re showing it. There’s a list of ambiguous syntax here.
CommonMark is as close to a standard as we have.
Thanks for the info. I thought that markdownguide.org was the standard as explained in your link from the creator.
By using what is described in markdownguide, I’ve never encountered any issue with any markdown compatible text editor.
There are Markdown libraries. Many have small differences and many apps have their own custom additions though.
As one of the Thunder devs, I can say there are markdown libraries. Thunder is written in Dart/Flutter and there is a great library that we use.
https://pub.dev/packages/flutter_markdown
That said, and as others have mentioned, markdown is not as well standardized and it seems like just about every site renders it differently, so there are a lot of edge cases to handle. Lemmy also has several unique implementations of things, such as spoilers, superscript/subscript, and the ability to tag users/communities without a hyperlink.
In fact, one of the things Thunder failed on (table alignment) is a known bug in the markdown library we use. :-)
I see. Markdown badly needs a good standard, doesn’t it.
cross-posted
Minor nit pick, but did you know that Lemmy has actual cross posting functionality?
Either way, interesting study. This is the type of content that I
Reder… Lemmy for, so thanks for posting. I use Voyager myself, being an Apollo refugee.Ya I’ve only been able to cross post on the web UI. I’ve seen apps like Jerboa and Voyager at least show cross posting correctly, I just wish they made it easier to cross-post in app.
Boost has a very easy-to-handle implementation of crossposting
Oh nice. I don’t think I’ve tried that one yet now that I think of it.
Thunder allows cross-posting! It should follow the web UI implementation (where the body of the new post has a link to the original, plus the original contents in a quote block).
Oh nice! That’s the one I’ve probably tried the least so I’ll have to give that one another go. (Not for any particular reason, I just got used to the UI of the first couple other apps I tried). Thanks for the good news!
I’ve yet to find an app which uses the same Lemmy crossposting function that is in the web UI.
Yeah, cross posting is another quirky Lemmy thing. AFAIK it just generates a new post with the same content, and also maybe varies by app. That could be wrong though: I’m not sure.
AFAIK it just generates a new post with the same content
Yup, that is exactly what it does. So if the original post is edited, none of the changes propagate to any of the crossposts.
In my opinion, crossposts should embed the original post, not simply copy a snapshot of the content at the time the crosspost is made. That’s a Lemmy issue though, not an app issue.
deleted by creator
There are a lot of image/gif(?) posts that I haven’t been able to view either on the Memmy (Apple) app or in-browser with either Safari (Apple) or Google Chrome. I imagine it comes down to the file types as well as the lack of native hosting to standardize posts of different media types, but I’m not the techiest person to consult on that. One downside of the fediverse is the lack of standards for file hosting/conversion/displaying to ensure that all media can be accessed regardless of the browser/app (or, alternatively, the lack of an all-encompassing app for all devices [Jerboa sounds like the closest to this to me but it is not available for iOS yet]), as well as the self-funded nature of the instances commonly not having the budget to natively host multimedia content such as videos.
If you’re getting that granular then you must’ve had to record the data somewhere. Did I miss where the OP is sharing their data set?
Sure, you can DM me somewhere to share a spreadsheet. Just please keep in mind that DM in Lemmy is not encrypted.
Just add it to the post lmao.
