• 0 Posts
  • 8 Comments
Joined 1 year ago
cake
Cake day: January 17th, 2024

help-circle

  • The 1.5B version that can be run basically on anything. My friend runs it in his shitty laptop with 512MB iGPU and 8GB of RAM (inference takes 30 seconds)

    You don’t even need a GPU with good VRAM, as you can offload it to RAM (slower inference, though)

    I’ve run the 14B version on my AMD 6700XT GPU and it only takes ~9GB of VRAM (inference over 1k tokens takes 20 seconds). The 8B version takes around 5-6GB of VRAM (inference over 1k tokens takes 5 seconds)

    The numbers in your second link are waaaaaay off.



  • It is not my intention to be rude. I’m from Colombia, follow Venezuela’s status closely (from media on a broad range of the political spectrum) see Venezuelan emmigrants daily and have met quite a few Venezuelans, and yet Lemmy is the only place I’ve ever seen with people really convinced that Venezuelans love Maduro, and the current situation of the country is because of the sanctions.

    It feels almost surreal, and reminds me when some people on Reddit were convinced they knew better than me what’s my country’s political status, all while mistakenly calling the country “Columbia”.

    I’m not trying to argue that you should blindly trust my opinions here, but really, really, Venezuela is in a bad spot, nobody likes Maduro’s dictatorship, and the sanctions are not the main causes of any of that (but they do help). Either that or somehow almost everybody in whole Latin American has a very biased opinion from first-hand experiences, and only people from other continents can see that.